Home / Learn / Data & AI / Offline AI / Lesson 3.10
All lessons

Module 3: Tokens: the unit of everything · Lesson 3.10 · about 20 min

Measure tokens and speed on your own machine

Get exact token counts and speeds from your own engine, with one command and a short Python script, so every estimate in this course becomes a measurement.

Key points, meanings and formulas on one page: choose "Save as PDF" in the print box.

Module 3 · about 20 minutes · read first: 3.5 and 11.2 (or come back after 11.2). You’ll be able to measure exact token counts and speeds for any prompt on your machine.

Your engine already counts every token and times every answer; read those numbers a few times and every estimate becomes a measurement you can plan with.

1. Why this matters

Rules of thumb get you started; measurements let you plan. With real counts you can say “this job is 2.1 million tokens and will take 9 hours”, and check that a change actually helped.

2. The quick way

ollama run gemma4:e4b --verbose

Ask something. After the answer you will see lines such as prompt eval count (input tokens), prompt eval rate (reading speed), eval count (output tokens) and eval rate (writing speed). llama.cpp prints similar timings after each answer, and LM Studio shows tokens per second under each reply.

It is like the trip computer in a car: it already knows the distance and the time; you just have to look.

3. The script way

import ollama

def measure(prompt: str, model: str = "gemma4:e4b") -> None:
    r = ollama.chat(model=model, messages=[{"role": "user", "content": prompt}])
    tin, tout = r["prompt_eval_count"], r["eval_count"]
    secs = r["eval_duration"] / 1e9
    print(f"input {tin} tokens, output {tout} tokens, writing {tout / secs:.1f} tokens per second")

for _ in range(3):
    measure("Explain what a context window is in 80 words.")

Run it three times: the first includes loading the model. Swap in your own prompts to measure your real jobs, and log the results (Lesson 9.7).

Here is what the script does with one reply.

The reply arrives with numbers attached: prompt_eval_count 42, eval_count 96, eval_duration 4,800,000,000.

Counts. 42 input tokens were read and 96 output tokens written.

Convert the time. Durations are in nanoseconds (billionths of a second): 4,800,000,000 ÷ 1,000,000,000 = 4.8 seconds of writing.

Speed. 96 ÷ 4.8 = 20 tokens per second.

Repeat and take the middle. Three runs give 14 (model loading), 20 and 21. The median is 20: that is your machine's real writing speed for this model.

Maths and principle

Principle: measure several runs and use the middle value; single runs mislead.

Equation: speed = tokens ÷ seconds; job time ≈ items × (input ÷ read speed + output ÷ write speed).

Analogy: timing a commute: one day’s traffic jam does not tell you the usual time; a week of trips does.

Try it: runs of 18, 21 and 20 tokens per second. Which is the middle (median)? (20.)

items
how many files, emails or questions the job will process
input, output
measured tokens read and written for a typical item
read speed, write speed
your measured prompt eval rate and eval rate, in tokens per second
job time ≈ items × (input ÷ read + output ÷ write)
how long the whole run will take, in seconds
Physical meaning
Measure a few typical items and you can say "this job takes 1 hour 50 minutes" before you start it, then check that a change really made it faster rather than guessing.

“The first answer shows how fast my machine is.” The first request also loads the model from disk into memory, which can take many seconds, so it looks slow. Ignore it, or take the middle of three runs. Speeds also drop a little with very long prompts, so measure with prompts like your real ones.

4. Worked example

Made-up numbers. Before a 600-file job, Zoe measures three typical files: about 1,800 input and 110 output tokens each, reading at 450 and writing at 16 tokens per second. Each file takes about 4 + 7 = 11 seconds, so the job needs about 1 hour 50 minutes. She starts it before lunch.

5. Practice

  1. Run --verbose and note your reading and writing speeds.
  2. Run the script on a short and a long prompt. How do the speeds change?
  3. Estimate the time for 1,000 items at 1,000 input and 80 output tokens, with your measured speeds.
Answers
  1. Keep them in your log; they are your machine’s real numbers.
  2. Writing usually slows a little as the context grows.
  3. Items × (1,000 ÷ read speed + 80 ÷ write speed).

New words: eval count (output tokens), prompt eval count (input tokens), median (the middle value).

  • ollama run <model> --verbose prints input and output token counts and both speeds after each answer.
  • In code, prompt_eval_count is input tokens, eval_count is output tokens, and durations are in nanoseconds.
  • Writing speed = eval_count ÷ (eval_duration ÷ 1,000,000,000).
  • Run three times and take the median; the first run includes loading the model.
  • Job time ≈ items × (input ÷ read speed + output ÷ write speed).

Check yourself

1In Ollama's reply, what is eval_count?
2eval_duration is in nanoseconds. How do you get tokens per second?
3Why measure the same prompt three times?

Sources

Your ticks stay in this browser. Save them as a file on the course home.

Updated 2026-11-22. Prices and model names change fast: check the official source before you buy.