Module 2: Selecting and filtering · Lesson 2.3 · about 20 min
Filtering rows with conditions
Keep the rows you need with conditions: comparisons, & for and, | for or, ~ for not, brackets around each part, and filters on text and dates.
Module 2 · about 20 minutes · uses Lesson 1.3. You’ll be able to filter rows with one or many conditions, on numbers, text and dates.
In everyday terms: a sieve with holes shaped by your rule. Pour the table through and only the rows that fit the rule stay.
1. One condition
A condition makes a True/False Series (a mask); putting it in the brackets keeps the True rows.
Comparisons: ==, !=, >, >=, <, <=.
2. Combining conditions
& and, | or, ~ not. Wrap each condition in brackets.
Long filters read better as named masks: is_web = ..., is_big = ..., then orders[is_web & is_big].
3. Text and dates
- df['qty'] > 2
- A mask: one True/False per row.
- df[mask]
- Rows where the mask is True.
- &, |, ~
- And, or, not; wrap each condition in brackets.
- Physical meaning
- A mask is a column of ticks and crosses. Combining masks is combining ticks: & needs both ticked, | needs either. The filter never changes the rows it keeps.
“Use and / or between conditions.” Python’s and works on single values, not columns, and raises an error. Use & and |, with brackets: df[(df.qty > 2) & (df.store_id == ‘S01’)].
Watch it happen, one step at a time:
| store | qty |
|---|---|
| S01 | 3 |
| WEB | 1 |
| S01 | 1 |
| S02 | 4 |
qty > 2 → True, False, False, True.store == 'S01' → True, False, True, False.| store | qty |
|---|---|
| S01 | 3 |
4. Your turn
How it works
Principle: a filter is two steps: build a mask (one True/False per row), then index with it. Because masks are Series, you can store, combine, count (mask.sum()) and average them (mask.mean() is the share of True).
Why the brackets: in Python & is evaluated before >=, so df.qty >= 2 & df.store == 'S01' is read as df.qty >= (2 & df.store) == 'S01' and fails. Brackets make the order explicit.
Why not and: and asks “is this whole Series true?”, which has no single answer, so pandas raises “The truth value of a Series is ambiguous”.
5. Practice
- Write a filter for qty between 2 and 4, inclusive.
- What share of lines have a discount? (One expression.)
- What does the error “The truth value of a Series is ambiguous” usually mean?
Answers
df[(df['qty'] >= 2) & (df['qty'] <= 4)]ordf[df['qty'].between(2, 4)].(df['discount'] > 0).mean().- You used
and/or(orif) with a Series; use&/|with brackets.
New words: mask, boolean indexing, and (&), or (|), not (~).
- Conditions make boolean masks; df[mask] keeps True rows.
- Combine with & (and), | (or), ~ (not), each condition in brackets.
- mask.sum() counts matching rows; mask.mean() gives the share.
- Filtering never changes values, only which rows are kept.
Check yourself
Sources
- pandas documentation, User Guide: Indexing and selecting data (boolean indexing) https://pandas.pydata.org/docs/user_guide/indexing.html#boolean-indexing
- McKinney, W. (2022), Python for Data Analysis, 3rd edition, O'Reilly (open access) https://wesmckinney.com/book/
Your ticks stay in this browser. Save them as a file on the course home.
Updated 2026-11-22. All datasets are made up. Python runs in your browser (the first run downloads it once, about 20 MB); nothing you type is sent anywhere.