All lessons

Module 3: Cleaning data · Lesson 3.3 · about 20 min

Data types and converting them

Check each column's type and fix the wrong ones: text to numbers with to_numeric (stripping $ and %), numbers to text for codes, categories for repeated labels, and booleans.

Key points, meanings and formulas on one page: choose "Save as PDF" in the print box.

Module 3 · about 20 minutes · uses Lesson 1.6. You’ll be able to fix columns that were read with the wrong type.

Each column has a type (number, text, date, category, True/False), and the wrong type silently breaks sums, sorting and filters.

In everyday terms: numbers stored as text are like prices written on paper tags: you can read them, but the till cannot add them. Converting the type is typing them into the till.

1. Spot the wrong types

Pythonorders_messy.csv

Unit Price and Discount should be numbers but are text (str), because some cells contain $ or %. You can’t sum or average them until they’re converted.

2. Text to numbers

Pythonorders_messy.csv

If some values can’t be read, errors='coerce' turns them into NaN, and you can see which:

Python

3. Numbers to text, categories and booleans

Python
  • astype(str) for codes you’ll never do maths on (ids, postcodes, phone numbers).
  • An ordered category sorts in a meaningful order and saves memory on repeated labels.
  • map with a dictionary turns labels into True/False (or any values).
  • astype(int) fails if the column has NaN; use astype('Int64') (capital I), which allows missing whole numbers.
int64 / float64
Whole numbers / decimals.
object or string
Text.
datetime64
Dates and times.
category
Text with few repeated values, stored compactly.
pd.to_numeric(errors='coerce')
Convert text to numbers; anything unreadable becomes NaN for you to inspect.
Physical meaning
The type decides what operations mean. "10" + "5" is "105" as text and 15 as numbers; "10" sorts before "9" as text. Fix types right after reading.

“If it looks like a number, pandas treats it as one.” One ”$” or “n/a” in a column turns the whole column into text. info() shows object instead of float64: that is your clue.

Watch it happen, one step at a time:

The column. price = ['$4.50', '$12', 'n/a', '$7.25'] (object).
Strip symbols. .str.replace('$', '') → ['4.50', '12', 'n/a', '7.25'].
Convert. pd.to_numeric(..., errors='coerce') → 4.5, 12.0, NaN, 7.25 (float64).
Inspect what failed. The NaN came from 'n/a': a real gap, not a typo.
Now maths works. Sum = 23.75.

4. Your turn

Your turnorders_messy.csv
Your turn
How it works

Principle: a pandas column holds one type. If even one value can’t be read as a number, read_csv keeps the whole column as text. Converting is deliberate: you decide how to treat the odd values instead of pandas guessing.

Types you’ll meet: int64, float64 (numbers), bool, str (text, pandas 3), datetime64[ns] (dates, next lessons), category, and the nullable Int64, boolean.

5. Practice

  1. Convert '1,250.50' style text to numbers.
  2. Why keep customer ids as text even if they look like numbers?
  3. What does astype('Int64') allow that astype(int) doesn’t?
Answers
  1. pd.to_numeric(s.str.replace(',', '', regex=False)).
  2. You never add them up, leading zeros matter, and some systems mix letters in later.
  3. Missing values (shown as <NA>) in a whole-number column.

New words: dtype, to_numeric, coerce, astype, category, ordered category, nullable integer.

  • Check dtypes with info() after reading.
  • Clean symbols, then pd.to_numeric(errors=‘coerce’) and inspect new NaN.
  • Use astype for safe conversions; category for repeated text; bool for flags.
  • Text numbers add and sort as text: “10” + “5” = “105”.

Check yourself

1Prices arrive as text like '$12.50'. How do you make them numbers?
2What does errors='coerce' do in pd.to_numeric?
3Why store a column like store size (small/medium/large) as category?

Sources

Your ticks stay in this browser. Save them as a file on the course home.

Updated 2026-11-22. All datasets are made up. Python runs in your browser (the first run downloads it once, about 20 MB); nothing you type is sent anywhere.