Module 7: Getting and choosing models · Lesson 7.3 · about 25 min
Model families today and what each is good at
The main open-weight model families in late 2026, their sizes, licences and strengths, and how to choose one for your machine and your job.
Module 7 · about 25 minutes · read first: 4.1 and 7.1. You’ll be able to name the main families, what fits your machine, and which to try first for your job.
1. Why this matters
New models appear every few weeks, so names date quickly. What lasts is knowing the families, their size classes and their licences. A family is a set of models from one maker that share a design and a tokenizer, like car models from one brand. Once you know the families, you can place any new release in seconds.
2. The families you can run at home (November 2026)
| Family | Sizes you can run | Licence | Good at |
|---|---|---|---|
| Gemma 4 (Google) | E2B, E4B, 12B, 26B A4B (MoE), 31B | Apache 2.0 | All-rounder; images in, and audio on the smaller sizes; long context (256K for 31B); phones to desktops |
| Qwen3.8 (Alibaba) | 27B dense, plus larger MoE models | Apache 2.0 for 27B; others vary | Strong reasoning and coding for its size; long context (262K) |
| gpt-oss (OpenAI) | 20B, 120B (MoE, 5.1B active) | Apache 2.0 | Reasoning and tool use; 120B is fast for its size on 128 GB machines |
Two words in that table:
- Dense: every parameter is used for every token, like a choir where everyone sings every note.
- MoE (Mixture-of-Experts): the model is split into many “experts” and only a few work on each token (Lesson 2.5). “26B A4B” means 26 billion parameters in total, about 4 billion active per token.
3. The big ones (data-centre size)
Kimi K3 (Moonshot AI, 2.8T parameters), DeepSeek-V4.1-Flash (MIT licence), GLM-5.3 (Z.ai), Meta’s Muse Spark Open and others are open too, but need hundreds of GB of memory. They matter for teams that rent servers (Lesson 15.7) and as a sign of where home-sized models will be in a year or two.
4. Choosing
- Fit: pick the biggest size class that fits your fast memory with room for context (Lesson 4.1).
- Job: coding and maths favour strong reasoners; document work and many languages favour broad all-rounders; images or audio need a multimodal model (one that reads pictures or sound as well as text).
- Licence: check it matches your use (Lesson 7.2).
- Try two: run the same five real tasks on two candidates and keep the one that does your work better. Benchmarks guide; your own tasks decide (Lesson 7.4).
Watch it happen: placing a new release in four reads
The announcement. "New model: 60B A6B, Apache 2.0, 128K context." Three numbers and a licence. Here is what each tells you.
Memory from the total. All 60B parameters must sit in memory. At about 4.8 bits each: 60 × 4.8 ÷ 8 = 36 GB. That needs a 48 to 64 GB machine.
Speed from the active part. Each token reads only the 6B active weights: 6 × 4.8 ÷ 8 = 3.6 GB per token. It writes about as fast as a 6B model, ten times less reading than a 60B dense model.
Licence and context. Apache 2.0: fine for paid work. 128K context: long reports fit, if you have memory left for it (Lesson 8.4).
Verdict. A good fit for a 64 GB Mac; too big for a 16 GB laptop. Shortlist it, then test it on your own five tasks.
Maths and principle
Principle: a Mixture-of-Experts model needs memory for all its parameters but reads only its active ones per token.
Equation: memory ∝ total parameters; writing speed ∝ 1 ÷ active parameters.
Analogy: a hospital with many specialists. The building must hold them all, but each patient sees only two or three, so patients move through quickly.
Try it: gpt-oss-120b has 117B total and 5.1B active. How many times fewer weights are read per token than a 117B dense model? (117 ÷ 5.1 ≈ 23 times.)
- Ptotal
- all the parameters, in billions: the "26B" in "26B A4B"
- Pactive
- the parameters actually used for each token: the "A4B"
- b
- bits per weight after quantisation, about 4.5 to 4.8 for a typical 4-bit file
- bandwidth
- how many GB per second your memory can be read
- Physical meaning
- The total decides whether the model fits in the building; the active part decides how fast patients get through. An MoE model is a big building with short queues.
“An MoE model with 4B active needs only as much memory as a 4B model.” It needs memory for all its parameters, because any expert may be chosen for the next token. The small active part makes it fast, not small.
5. Worked example
Made-up numbers. A small accounting firm has a 32 GB Mac. They try Gemma 4 31B and Qwen3.8-27B on five real tasks (two summaries, a spreadsheet formula, an email draft, a policy question). Qwen wins the formula; Gemma wins the summaries and emails. They keep both and choose per job.
6. Practice
- Which family and size would you try first on your machine?
- Write five real tasks from your own work to test models with.
- A new model is announced as “60B A6B, Apache 2.0, 128K”. Place it: memory class, speed, licence.
Answers
- Use your result from the simulation in 4.1.
- Keep them; you will reuse them in Lesson 7.4 and the capstone.
- Memory like a 60B model (about 36 GB at 4-bit, so a 48 to 64 GB machine); speed like a 6B model; free for commercial use.
New words: family (models from one maker that share a design and tokenizer), dense (every parameter used per token), MoE (only some experts used per token), multimodal (reads images or audio as well as text).
- The home-sized families in late 2026: Gemma 4, Qwen3.8 and gpt-oss, all with Apache 2.0 options.
- Memory follows the total parameters; writing speed follows the active parameters.
- “26B A4B” means 26 billion in total, about 4 billion active per token.
- Choose by fit, job and licence, then test two candidates on your own tasks.
Check yourself
Sources
- Open models, sizes, licences and context lengths: Morph, "Best Open Source LLMs (November 2026)", https://www.morphllm.com/best-open-source-llm
- Download sizes and Ollama tags, read 16 November 2026 via BenchLM, https://benchlm.ai/best/ollama-models
- gpt-oss-120b model card: OpenAI on Hugging Face (official page), https://huggingface.co/openai/gpt-oss-120b
- Shazeer, N. et al. (2017), Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer https://arxiv.org/abs/1701.06538
Your ticks stay in this browser. Save them as a file on the course home.
Updated 2026-11-22. Prices and model names change fast: check the official source before you buy.