Module 1: Getting started · Lesson 1.6 · about 15 min
A first look: head, info, describe
The five commands to run on every new table: head and sample to see rows, info for types and missing values, describe for numbers, value_counts for categories, and nunique for keys.
Module 1 · about 15 minutes · uses Lesson 1.5. You’ll have a routine for the first five minutes with any new dataset.
In everyday terms: a mechanic walks round a car before opening the bonnet: tyres, lights, dents. info, describe and value_counts are that walk round, and they catch most problems early.
1. See some rows
display() shows a table mid-box; the last line is shown anyway.
2. Types and missing values: info
Check: is each column the type you expect (dates as datetime64, amounts as numbers)? Does any column have fewer non-null values than there are rows?
3. Numbers: describe
Read the min and max for impossible values, and compare mean with median (50 %) for skew (Statistics Lesson 1.2).
4. Categories and keys
There are more rows than distinct order ids, so a row is an order line (one product within an order), not an order. Knowing what one row means (the grain) prevents double counting later.
- head(), sample()
- See real rows.
- info()
- Column types and non-missing counts.
- describe()
- Count, mean, SD, min, quartiles, max for numbers.
- nunique(), value_counts()
- How many distinct values, and which are most common.
- Physical meaning
- Each summary answers a "could this be wrong?" question: a negative quantity, a price of 9,999, a store that appears once, a date in 1970. Spotting them now saves wrong conclusions later.
“describe() covers everything.” It skips text columns by default and says nothing about duplicates or keys. Add describe(include=‘object’), duplicated().sum() and value_counts() on the key columns.
Watch it happen, one step at a time:
orders.head(): the columns look as expected.orders.info(): 9,717 rows, qty int64, date datetime, no missing values.orders.describe(): qty from 1 to a small maximum, discount from 0 to 0.3. Nothing impossible.orders['store_id'].value_counts(): 7 stores, none tiny.5. Your turn
How it works: the first-look checklist
df.shape: as many rows and columns as you expected?df.head(),df.sample(5): what does one row mean (the grain)?df.info(): right types? missing values?df.describe(): impossible or extreme numbers?df[col].value_counts()anddf.nunique(): odd categories, spelling variants, unique keys?
Write down what surprises you; each surprise is a question for the data owner or a cleaning step (Module 3).
6. Practice
- Suppose
info()saiddiscounthad 9,000 non-null values out of 9,717 rows. What does that tell you? - Why use
sample()as well ashead()? - What is the grain of
orders.csv?
Answers
- 717 rows have no discount recorded: missing values to understand (zero discount? not recorded?).
- The first rows are often unusual (sorted by date, test rows). A random sample is more typical.
- One order line: one product in one order (an order can have several rows).
Try it on your own data: CSV profiler runs this whole checklist on a CSV in one go.
New words: head, tail, sample, info, describe, value_counts, nunique, grain.
- head/sample for rows; info for types and gaps; describe for ranges.
- value_counts and nunique for categories and keys.
- Look for impossible values, odd types, rare categories and duplicates.
- Write down what you found before analysing.
Check yourself
Sources
- pandas documentation, User Guide: Essential basic functionality (descriptive statistics) https://pandas.pydata.org/docs/user_guide/basics.html
- NIST/SEMATECH e-Handbook, 1.1 EDA Introduction https://www.itl.nist.gov/div898/handbook/eda/section1/eda1.htm
Your ticks stay in this browser. Save them as a file on the course home.
Updated 2026-11-22. All datasets are made up. Python runs in your browser (the first run downloads it once, about 20 MB); nothing you type is sent anywhere.