Module 7: Retail media, adtech and measurement · Lesson 7.2 · about 30 min
Adtech plumbing: how an ad gets served
Ad servers, auctions, real-time bidding, DSPs and SSPs, identity and consent, frequency caps and pacing, clean rooms, and the event data every step creates.
Module 7 · about 30 minutes · read first: 7.1, 4.3. You’ll be able to walk through an ad request from page load to impression, name the pieces (ad server, DSP, SSP, exchange, clean room), work out who wins an auction and what they pay, check a campaign’s pacing, and design the topics and stores behind real-time ad controls.
1. An ad request in about 100 milliseconds
A shopper opens the grocery app’s search results for “cola”.
- The app sends an ad request to the ad server with the context: page, placement, search term, a user or device ID, and consent flags.
- For sponsored products, the retailer’s own auction picks eligible ads (cola brands bidding on “cola”, in stock, with budget left, under their frequency caps), scores them by bid × predicted click rate (often with a relevance factor too) and returns the winners.
- For display or offsite ads, the request may go to an SSP and an ad exchange, which run a real-time bidding (RTB) auction: many DSPs receive a bid request, decide within tens of milliseconds whether and how much to bid, and the best bid wins.
- The ad appears. The device fires an impression event; a viewability script fires a viewable impression when enough of the ad is on screen for long enough; a tap fires a click.
- Later, an order or store purchase is linked back to that exposure for attribution (Lesson 7.3).
Each step writes events: requests, bids, wins, losses, impressions, clicks, conversions. A busy network creates billions of events a day.
Think of a lightning-fast auction house. Each page load puts one seat at a show up for sale (an impression). The auctioneer (exchange) calls out the seat’s details; buyers’ agents (DSPs) have a tenth of a second to bid for their clients (advertisers); the seller’s agent (SSP) wants the best price; the box office (ad server) prints the ticket. It happens billions of times a day, and every call and bid is written in the ledger.
2. How the auction picks a winner
Ranking only by bid would fill the page with ads nobody clicks, which earns the retailer nothing on a pay-per-click deal and annoys shoppers. So the ad server ranks by expected value: bid × predicted click rate (CTR). The predicted CTR comes from a machine learning model fed by past impressions and clicks, which is one reason clean event data matters.
Many sponsored-products auctions use a second-price rule: the winner pays just enough to have beaten the runner-up, not its full bid. In a ranking by score, that price is the runner-up’s score divided by the winner’s CTR, plus one cent.
3. Who’s who
| Piece | Job |
|---|---|
| Ad server | Decides and records which ad shows in each placement (publisher ad servers, retailer sponsored-products engines) |
| DSP (demand-side platform) | Lets advertisers buy across many publishers with targeting |
| SSP (supply-side platform) and exchange | Lets publishers sell their ad space in auctions |
| CDP (customer data platform) | Builds audience segments from first-party data |
| Clean room | Privacy-safe joins between partners |
| Measurement and verification | Independent checks of viewability, fraud and outcomes |
4. Identity and consent
To link an ad view to a purchase, you need to know it was the same person. Methods, from strongest to weakest:
- Logged-in first-party IDs (loyalty ID, account): the most accurate, and the core advantage of retail media.
- Hashed emails matched between partners, inside clean rooms. “Hashed” means turned into a fixed scramble that cannot easily be turned back.
- Device IDs on phones, limited by the platforms’ privacy controls.
- Contextual targeting needs no identity at all: the search word “cola” is enough.
Privacy laws (such as Canada’s PIPEDA and Québec’s Law 25, Europe’s GDPR and US state laws) decide what may be collected and used. Platforms store consent flags with every event and enforce them in the pipelines. Check the current rules for each country you work in.
5. Real-time controls that need streaming
- Budget pacing: compare spend so far with the plan every minute; slow down or speed up bidding.
- Frequency capping: “no more than 3 impressions per user per day” needs a fast counter per user and campaign (Bigtable or Redis, from Bigtable and wide-column stores).
- Out-of-stock suppression: stop ads for products not available at the shopper’s store, using stock events from Kafka.
- Bot filtering: drop invalid traffic before it is billed.
Pacing is simple arithmetic run every minute against a live spend counter:
- Ad request
- The app asking "which ad goes here?", with the page context and consent flags.
- Score
- Bid × predicted click rate: what an impression is expected to earn.
- Second price
- The winner pays just enough to have beaten the runner-up.
- Pacing
- Keeping spend in line with the plan so the budget lasts to the end date.
- Frequency cap
- A limit on how often one person sees one campaign.
- Physical meaning
- When you search "cola" and the first result is a sponsored bottle you might actually buy, not the highest bidder's, an auction weighed bids by how likely you were to click, checked a counter so you would not see the same ad a tenth time today, and wrote the whole decision down in under a tenth of a second.
“The brand with the biggest bid always gets the top spot.” Ad servers rank by bid × predicted click rate (and often relevance), because an ad nobody clicks earns nothing. A brand with a well-liked product and a modest bid can win and pay less than its bid.
6. Data clean rooms
Brands want to combine their data (their customer list, their other campaigns) with the retailer’s shopper data, and neither side may share raw personal data. A clean room lets both put data into a controlled space where only approved queries run: aggregates above a minimum audience size, no row-level export. On Google Cloud, BigQuery data clean rooms do this with analysis rules; other clouds and specialist vendors offer similar products.
7. The event data model
A minimal impression table:
| column | example |
|---|---|
| event_id | imp_7f3c… |
| event_ts | 2026-11-04 10:02:07.311 |
| campaign_id, line_item_id, creative_id | C123, L9, CR55 |
| placement, page_type, search_term | search_top, search, cola |
| user_key (hashed), device_type | u_9ab…, ios |
| price_paid, auction_type | 0.61, second_price |
| is_viewable, consent_ad | true, true |
Clicks and conversions refer back to event_id or user_key and time, which is how attribution joins them in the next lesson.
8. Try it
Design the Kafka topics, keys and downstream stores for (a) pacing, (b) frequency capping, (c) advertiser dashboards and (d) monthly billing. For each, give the delay the business can accept and the store you would use.
One design, and what to check
Topics ads.impressions.v1 and ads.clicks.v1, keyed by campaign_id so each campaign’s events stay in order, plus ads.spend.v1.
- (a) Pacing: Flink sums spend per campaign per minute into Redis or Bigtable counters that the bidder reads (seconds).
- (b) Frequency caps: Flink or the ad server increments the key
user#campaign#datein Bigtable or Redis (milliseconds). - (c) Dashboards: Druid reads impressions and clicks from Kafka with a one-minute rollup (seconds to a minute).
- (d) Billing: Kafka Connect to Cloud Storage, then Spark removes duplicates and invalid traffic, then BigQuery billing tables, reconciled monthly (daily batch, exact).
Check: each control has a delay in plain units, the fastest ones use a key-value store, and billing uses the exact batch path, not the live counters.
New words: ad server, ad request, real-time bidding, DSP, SSP, exchange, predicted click rate, second-price auction, pacing, frequency cap, clean room.
- An ad request triggers an auction in about 100 ms; every step (request, bid, win, impression, click) is an event.
- Sponsored-products auctions rank by bid × predicted click rate, and second-price rules charge the winner just enough to beat the runner-up.
- Pacing, frequency caps, stock suppression and bot filtering need streaming and fast key-value counters.
- Identity links exposure to purchase, consent decides what may be used, and clean rooms let partners join data without sharing rows.
Check yourself
Sources
- IAB Tech Lab: OpenRTB specification (iabtechlab.com)
- Edelman, B., Ostrovsky, M. and Schwarz, M. (2007), Internet Advertising and the Generalized Second-Price Auction, American Economic Review 97(1), 242-259 https://doi.org/10.1257/aer.97.1.242
- Google Cloud documentation: BigQuery data clean rooms (cloud.google.com/bigquery/docs)
Your ticks stay in this browser. Save them as a file on the course home.
Updated 2026-11-22. All company data in the examples is synthetic. Tool versions and prices change: check the official docs before you rely on them.