Home / Learn / Data & AI / Offline AI / Lesson 1.3
All lessons

Module 1: Welcome, and why run AI yourself · Lesson 1.3 · about 20 min

What "offline" really means

Download once, run forever. What needs the internet during setup, what never does, and what always will.

Key points, meanings and formulas on one page: choose "Save as PDF" in the print box.

Module 1 · about 20 minutes · read first: 1.2. You’ll be able to say exactly which steps need the internet, and plan a setup that keeps working when the connection does not.

You need the internet once, to download the app and the model; after that, answering questions happens entirely on your own machine.

1. Why this matters

“Offline” is the reason many people want local AI: on a plane, at a remote site, in a secure office, or simply when the broadband drops. But “offline” has fine print. Knowing it saves you from finding out at the worst moment that one piece still wanted the internet.

2. Two phases: setup and use

Local AI has two phases.

Setup (online, once). You download two things: the program that runs models (an app such as LM Studio or Ollama) and one or more model files. Sizes:

  • a small model is 1 to 3 GB;
  • a good laptop model is about 5 to 10 GB;
  • the largest you might run at home are 60 GB or more.

Downloading them is the only heavy internet use.

Use (offline, every day). You type. The program loads the model into memory, and your own processor works out the answer. No connection is needed and nothing is sent anywhere.

It is like buying a cookbook. You need the shop once. After that you can cook from it in a cabin with no signal for years.

Diagram

Online, once

  • Download the app (about 0.1 to 1 GB)
  • Download one or more models (about 2 to 80 GB each)
  • Updates, when you choose

Offline, every day

  • Your questions and the answers
  • Your files and documents
  • Code, summaries, translations

Nothing leaves the machine.

Still needs the internet

  • Searching the web or reading live news
  • Sending email or messages for you
  • Getting a newer model
"Offline" means the thinking happens on your machine. Getting the files there, and anything that has to reach the outside world, still needs a connection.

3. What still needs the internet

A model is frozen at the moment it was trained, like a printed encyclopedia. It cannot know what happened after that date (its knowledge cut-off). It also cannot reach out to the world unless you connect it to something that can. So these still need a connection:

  • searching the web, reading live prices, news or weather;
  • sending email or messages on your behalf;
  • downloading a newer model or an update to the app.

Later modules show how to bring some of these inside your machine: an offline copy of an encyclopedia, your own documents (Module 12), offline speech (13.4).

4. Traps that break “offline”

Some setups look offline but quietly are not:

  • Apps that check for updates or send usage reports. Many let you switch this off; Lesson 9.6 shows how to test by pulling the cable.
  • Plug-ins and extensions that call a cloud service, for example a “web search” tool switched on by default.
  • Models that are not fully downloaded. Some programs fetch parts of a model the first time you use it. Run each model once before you go offline.
  • Python libraries that download extra files on first use. Lesson 9.6 shows how to pre-fetch them.

“My local model is on my computer today, so it knows today’s news.” It doesn’t. The model knows only what was in its training data up to its knowledge cut-off, which is often months before it was released. Being on your machine changes where it runs, not what it knows. For anything recent, give it the facts yourself (paste them in, or use your own documents as in Module 12).

Maths and principle

Principle: download time depends on file size and connection speed; it happens once, so plan it on a fast connection.

Equation: time (seconds) = size (GB) × 8,000 ÷ speed (Mbit/s). (1 GB = 8,000 megabits.)

Analogy: filling a water tank. The tank is the model size, the pipe is your connection. Once the tank is full, you can draw water for years without the pipe. It breaks down because some apps keep a small pipe open for updates unless you close it.

Try it: an 8 GB model on a 50 Mbit/s connection? (8 × 8,000 ÷ 50 = 1,280 seconds, about 21 minutes.)

size (GB)
how big the files are, in gigabytes; the model file is nearly all of it
× 8,000
turns gigabytes into megabits: 1 byte is 8 bits, and 1 GB is 1,000 MB
speed (Mbit/s)
your connection speed, in megabits per second, the number broadband plans quote
time (s)
how long the download takes in seconds; divide by 60 for minutes
Physical meaning
The download is the one moment you need the internet. Do it at home or the office on a fast line, check everything works with Wi-Fi off, and then the setup works anywhere for as long as you like.

5. Worked example

Made-up numbers. Tom is a field engineer who spends weeks on a site with no signal. Watch him get ready.

Add up the files. An app (0.5 GB), a general model (7 GB) and an offline speech-to-text model (1.5 GB): 0.5 + 7 + 1.5 = 9 GB.

Turn gigabytes into megabits. 9 × 8,000 = 72,000 megabits.

Divide by the connection speed. His home line is 100 Mbit/s: 72,000 ÷ 100 = 720 seconds, which is 12 minutes.

Test with Wi-Fi off. He runs each model once (so nothing is still half-downloaded), switches Wi-Fi off and checks every tool answers.

On site. He dictates notes, and the laptop turns them into tidy reports, with nothing sent anywhere.

6. Practice

  1. How long would a 20 GB model take on a 30 Mbit/s connection?
  2. Name two things you do with AI that would still need the internet.
  3. Your laptop has 256 GB of storage with 40 GB free. Could you keep three models of 8 GB each and still have room to work?
Answers
  1. 20 × 8,000 ÷ 30 ≈ 5,333 seconds, about 89 minutes.
  2. Anything live: today’s prices, news, flight times, or anything that sends messages.
  3. Yes: 24 GB for models leaves 16 GB. It is tight; keep at least 10 to 20 GB free so the system runs smoothly.

New words: knowledge cut-off (the last date the model learned about), setup phase and use phase, Mbit/s (megabits per second, how internet speed is quoted).

  • Setup needs the internet once: the app and the model files (1 to 10 GB for most laptops, 60 GB or more for the biggest).
  • Everyday use needs no connection; the answer is worked out by your own processor.
  • Live news, web search, sending messages and updates still need the internet.
  • Download time (s) = size (GB) × 8,000 ÷ speed (Mbit/s).
  • Before you rely on it, run each model once, then test everything with Wi-Fi off.

Check yourself

1You have downloaded a model and its app. The Wi-Fi goes down. What happens when you ask it a question?
2Which of these still needs an internet connection, even with a local model?
3About how big is a typical download for a model that runs well on a 16 GB laptop?

Sources

Your ticks stay in this browser. Save them as a file on the course home.

Updated 2026-11-22. Prices and model names change fast: check the official source before you buy.