Module 15: Scaling up: from one laptop to a team server · Lesson 15.7 · about 20 min
Renting graphics cards by the hour
The middle path between buying hardware and paying for a subscription. How hourly GPU rental works, what it costs, what it does and does not protect, and when it beats both.
Module 15 · about 20 minutes · read first: 14.3 and 15.2. You’ll be able to decide when to rent a GPU by the hour instead of buying one or paying for a plan.
1. Why this matters
Some jobs need a big card for a few hours: fine-tuning a model (Module 14), processing a backlog of 50,000 documents, or trying a 120B model before spending thousands on hardware. Renting gives you that power for the length of the job, with your own choice of open model and software.
2. How it works
- Choose a provider: GPU-rental services (for example RunPod, Vast.ai, Lambda) or the large clouds (AWS, Google Cloud, Azure, and regional providers in India, Canada and the UK).
- Pick a card (memory first, as always) and a ready-made image with vLLM, Ollama or a notebook.
- Start it. Billing begins. Load your model, run your job or serve your team over a secure link.
- Save results to your own storage, then stop or delete the machine. Stored disks may still cost a little.
On-demand machines are yours until you stop them. Spot or interruptible machines are cheaper but can be taken back with little warning: fine for batch jobs that save progress, bad for a live service.
The research group from section 5, hour by hour (made-up prices):
Choose. An 80 GB card with a ready-made training image, at 3.00 an hour on demand.
Start. The meter starts the moment the machine is on, whether you are using it or not.
Train. The fine-tune runs for 6 hours: 6 × 3.00 = 18.
Copy home, then delete. The adapter (tens of megabytes) is copied to their own storage and the machine is deleted. The meter stops.
The costly mistake. Had they forgotten it over a weekend, another 48 hours would have cost 48 × 3.00 = 144: eight times the job itself.
3. What it protects and what it does not
You control the model and software, so there are no per-message limits and no provider reading prompts to train a model. But your data does leave your building, travels over the internet and sits on someone else’s hardware. Read the provider’s data terms, choose a region that fits your rules (for example, data kept in Canada or India), and treat rented GPUs as cloud for any data policy from Lesson 15.6. Marketplace providers that rent out other people’s home machines are cheaper still and trusted less; keep private data off them.
4. Prices
Hourly prices vary by card, provider, region and week, and moved a lot in 2026 with the memory shortage (Lesson 5.7). A consumer-class 24 to 32 GB card usually rents for well under US$1 an hour, data-centre cards with 80 GB or more for a few dollars an hour. Check the providers’ pricing pages on the day; the numbers in this lesson’s examples are made up.
Maths and principle
Principle: rent when the hours are few, own when the hours are many.
Equation: months for owning to beat renting ≈ hardware price ÷ (hours per month × hourly rent − monthly power cost).
Analogy: a van. Hire one for moving day; buy one if you deliver every day.
Try it: a card costing 2,000, rented equivalent 0.50 an hour, used 20 hours a month, power 5 a month? (20 × 0.50 − 5 = 5 a month saved by owning: 400 months. Rent.) Used 200 hours a month? (100 − 5 = 95: about 21 months. Owning starts to make sense.)
- hardware price
- what buying a similar card (or computer) would cost once
- hours per month × hourly rent
- what renting the same power would cost each month
- monthly power cost
- the electricity your own card would use for those hours
- months to break even
- hardware price ÷ (monthly rent − monthly power): after this, owning is cheaper
- Physical meaning
- A van. Hire one for moving day; buy one if you deliver every day. Few hours a month means the purchase never pays off; many hours a month means the rent quickly adds up to the price of your own.
“A rented GPU is the same as running AI offline.” You choose the open model and the software, so there are no message limits and no provider training on your prompts. But the machine, its disks and its network belong to someone else, and your data travels there over the internet. Treat it as cloud in your data policy (Lesson 15.6).
5. Worked example
Made-up numbers. A university research group wants to fine-tune a 12B model on 10,000 labelled lab notes. On their own 16 GB card it would take days and might not fit. They rent an 80 GB card for 6 hours at 3.00 an hour (18 in total), train the adapter, copy it home and delete the machine. Daily use then runs on their own card with the new adapter (Lesson 14.5).
6. Practice
- Name two jobs in your work that suit renting rather than buying.
- Why is a spot machine a poor choice for your team’s live chat server?
- Using the formula, how many hours a month make owning a 1,500 card better within two years if renting costs 0.40 an hour and power is 4 a month?
Answers
- For example: a one-off fine-tune, a big document backlog, testing a model before buying hardware.
- It can be taken back at any moment, cutting off everyone mid-answer.
- 1,500 ÷ 24 ≈ 62.5 a month needed: hours × 0.40 − 4 ≥ 62.5, so about 166 hours a month or more.
New words: on-demand, spot (interruptible) instance, region, image (a ready-made software set-up).
- Renting suits short bursts: a fine-tune, a document backlog, or testing a big model before buying.
- Billing runs while the machine is on, used or not: save results, then stop or delete it.
- Spot machines are cheaper but can be taken back at any moment; fine for batch jobs that save progress, not for live chat.
- Months to break even ≈ hardware price ÷ (hours per month × hourly rent − monthly power); few hours favour renting.
- A rented GPU is cloud for your data policy: data leaves your building, so read the terms and pick a region.
Check yourself
Sources
- RunPod pricing (official page), https://www.runpod.io/pricing
- Vast.ai pricing (official page), https://vast.ai/pricing
- Lambda GPU cloud pricing (official page), https://lambda.ai/pricing
- vLLM documentation: running in Docker (official page), https://docs.vllm.ai
Your ticks stay in this browser. Save them as a file on the course home.
Updated 2026-11-22. Prices and model names change fast: check the official source before you buy.