Module 3: Tokens: the unit of everything · Lesson 3.10 · about 20 min
Measure tokens and speed on your own machine
Get exact token counts and speeds from your own engine, with one command and a short Python script, so every estimate in this course becomes a measurement.
Module 3 · about 20 minutes · read first: 3.5 and 11.2 (or come back after 11.2). You’ll be able to measure exact token counts and speeds for any prompt on your machine.
1. Why this matters
Rules of thumb get you started; measurements let you plan. With real counts you can say “this job is 2.1 million tokens and will take 9 hours”, and check that a change actually helped.
2. The quick way
ollama run gemma4:e4b --verbose
Ask something. After the answer you will see lines such as prompt eval count (input tokens), prompt eval rate (reading speed), eval count (output tokens) and eval rate (writing speed). llama.cpp prints similar timings after each answer, and LM Studio shows tokens per second under each reply.
It is like the trip computer in a car: it already knows the distance and the time; you just have to look.
3. The script way
import ollama
def measure(prompt: str, model: str = "gemma4:e4b") -> None:
r = ollama.chat(model=model, messages=[{"role": "user", "content": prompt}])
tin, tout = r["prompt_eval_count"], r["eval_count"]
secs = r["eval_duration"] / 1e9
print(f"input {tin} tokens, output {tout} tokens, writing {tout / secs:.1f} tokens per second")
for _ in range(3):
measure("Explain what a context window is in 80 words.")
Run it three times: the first includes loading the model. Swap in your own prompts to measure your real jobs, and log the results (Lesson 9.7).
Here is what the script does with one reply.
The reply arrives with numbers attached: prompt_eval_count 42, eval_count 96, eval_duration 4,800,000,000.
Counts. 42 input tokens were read and 96 output tokens written.
Convert the time. Durations are in nanoseconds (billionths of a second): 4,800,000,000 ÷ 1,000,000,000 = 4.8 seconds of writing.
Speed. 96 ÷ 4.8 = 20 tokens per second.
Repeat and take the middle. Three runs give 14 (model loading), 20 and 21. The median is 20: that is your machine's real writing speed for this model.
Maths and principle
Principle: measure several runs and use the middle value; single runs mislead.
Equation: speed = tokens ÷ seconds; job time ≈ items × (input ÷ read speed + output ÷ write speed).
Analogy: timing a commute: one day’s traffic jam does not tell you the usual time; a week of trips does.
Try it: runs of 18, 21 and 20 tokens per second. Which is the middle (median)? (20.)
- items
- how many files, emails or questions the job will process
- input, output
- measured tokens read and written for a typical item
- read speed, write speed
- your measured prompt eval rate and eval rate, in tokens per second
- job time ≈ items × (input ÷ read + output ÷ write)
- how long the whole run will take, in seconds
- Physical meaning
- Measure a few typical items and you can say "this job takes 1 hour 50 minutes" before you start it, then check that a change really made it faster rather than guessing.
“The first answer shows how fast my machine is.” The first request also loads the model from disk into memory, which can take many seconds, so it looks slow. Ignore it, or take the middle of three runs. Speeds also drop a little with very long prompts, so measure with prompts like your real ones.
4. Worked example
Made-up numbers. Before a 600-file job, Zoe measures three typical files: about 1,800 input and 110 output tokens each, reading at 450 and writing at 16 tokens per second. Each file takes about 4 + 7 = 11 seconds, so the job needs about 1 hour 50 minutes. She starts it before lunch.
5. Practice
- Run
--verboseand note your reading and writing speeds. - Run the script on a short and a long prompt. How do the speeds change?
- Estimate the time for 1,000 items at 1,000 input and 80 output tokens, with your measured speeds.
Answers
- Keep them in your log; they are your machine’s real numbers.
- Writing usually slows a little as the context grows.
- Items × (1,000 ÷ read speed + 80 ÷ write speed).
New words: eval count (output tokens), prompt eval count (input tokens), median (the middle value).
ollama run <model> --verboseprints input and output token counts and both speeds after each answer.- In code, prompt_eval_count is input tokens, eval_count is output tokens, and durations are in nanoseconds.
- Writing speed = eval_count ÷ (eval_duration ÷ 1,000,000,000).
- Run three times and take the median; the first run includes loading the model.
- Job time ≈ items × (input ÷ read speed + output ÷ write speed).
Check yourself
Sources
- Ollama API documentation on GitHub (response fields and durations in nanoseconds) https://github.com/ollama/ollama/blob/main/docs/api.md
- Ollama Python library https://github.com/ollama/ollama-python
- llama.cpp, official repository (llama-bench and per-answer timings) https://github.com/ggml-org/llama.cpp
- Ollama API response fields (prompt_eval_count, eval_count, eval_duration), https://docs.ollama.com
Your ticks stay in this browser. Save them as a file on the course home.
Updated 2026-11-22. Prices and model names change fast: check the official source before you buy.