Why Radio Ratings Sample Sizes Are So Low

Why Radio Ratings Sample Sizes Are So Low

by Tracy Johnson

The key to sustainable ratings success is to attract a large, loyal fan base that actually participate in the ratings process. Active participants share many traits, which are detailed here. But to win the Ratings Game, broadcasters should understand the ratings process, beginning with how a listener becomes a ratings respondent. Warning: the following information about woefully small sample sizes may disturb, shock and frustrate you.

Ratings companies have a difficult mission. In today’s over-communicated world, everyone is asking consumers to respond to a survey or poll. It’s probably happened to you several times just this week.

It’s become such a problem, finding a valid sample for research has become a major issue for every industry. That’s especially the case for radio surveys.

Finding Radio Ratings Respondents: The Sample Size

Most respondents are contacted using one of three methods. Each has obvious flaws.

Phone Calls: Historically, this has been the preferred method of finding respondents. But as Caller ID has become universal, unwanted solicitations have skyrocketed and landlines have become extinct, it’s more difficult to recruit ratings respondents by telephone. Would you actually answer a call from a number you don’t recognize? Neither do most others.

Direct Mail: Some ratings companies try to find respondents by sending a letter or packet of information to households. This is not only expensive, but response rates have become microscopic. Most pieces end up being recycled with grocery ads and other unwanted solicitations.

Knocking on Doors: When a stranger comes to your door offering cash or gifts to participate in a survey, most of us would probably be suspicious. But ratings companies continue this method of recruitment because of the struggle in the other tactics.

So ratings companies have a big problem. It’s hard to attract valid sample sizes to participate.

Why Radio Ratings Sample Sizes Are So Low

In addition to contacting listeners, convincing them to become a ratings respondent is even more difficult. Assuming a ratings company is able to find a potential participant, here’s what they’re asked to do:Nielsen PPM (Personal People Meter)

  • Carry a meter… for up to two years… everywhere they go. Every. Single. Day.
  • It must be carried with them. It can’t be left in a purse or on a desk. It has to remain “active.”
  • Therefore, the device resembles a pager that clips in a pocket or belt. Remember pagers? From the 90s?
  • Place that meter in a dock each night to upload their radio listening and recharge the meter.
  • Ensure participation from each member of the household. The main contact is responsible for his or her family.

If a diary market, the respondent is asked to carry a paper and pencil to write down every radio station they listen to, including Radio Ratings Diarystart and stop times. Then mail it back in. Since most listening takes place in the car, how do you think that is going to work out?

You might also like:  When Bad Ratings Happen to Good Stations

No wonder sample sizes are inadequate to the point of being ridiculous. And, it’s no surprise that each respondent expects to be paid for their time.

Incentives to Participate in Radio Ratings Surveys

And they do get paid. Some of them significantly.

Each respondent receives regular cash compensation. The amount is a bit of a mystery, but I know of first-hand instances of hard-to-recruit demographics getting $1,000 per month plus bonuses.

Most ratings companies have a harder time finding young participants. Male audiences are also notoriously more difficult to recruit. The harder it is to fill the quota, the greater the payoff.

It’s also common to offer even more prizes if the respondent listens to more radio! Some are awarded points for more listening. The points can be redeemed for prizes in an exclusive online catalog. This inflates actual radio listening, of course. But then, this is not the only problem with the sample.

For some, the incentive to play the Ratings Game is so strong, a user actually asks friends or coworkers to carry their meter for them. I know of one 22-year-old male that said:

I don’t listen to the radio or even like it really. But they’re paying me and my dad so much, I can’t afford to say no. My friends and I switch off carrying the meter every week or so. We’ve been doing it for about a year.

Consequences of Poor Radio Rating Sample Sizes

The end result is that sample sizes are painfully low. No responsible manager would make decisions based on this data, yet millions of dollars a year are spent advertising to listeners based on it.

There are thousands of horror stories about the ratings process. I’m not going to recount them, but here are a couple of highlights… or rather, lowlights:

  • A client in a Top 20 market saw their 25-54 ratings dip from Top 3 to #10 overnight. After several months of investigation and hand-wringing, we discovered what happened: a single heavy listener had a birthday. He turned 55, aging out of the demographic we tracked.
  • Another client saw a sudden spike in online streaming ratings for 18 to 34-year-old women. Their overall share increased by 15% in one month. The ratings gain came from a single 31-year-old woman who listened at work online. The gain was enough to raise the station’s rank from #3 to #1 in the market.
  • A Classic Hits station targeting 35 to 54-year-olds suddenly shot from #18 to #1 in 18 to 34-year-old men. The source? You guessed it. A single meter from a heavy user. The ratings increase stayed with the station for nearly a year, until the respondent was replaced in the panel.
You might also like:  What To Do About Nielsen's Headphones Bonus

Solutions for Small Sample Sizes

Ratings services understand the problem, though they are careful about admitting how bad it has become. Much research has gone into alternatives, but none have been adopted. Each new idea introduces more potential problems.

Many have suggested using smartphones to track listening. That certainly solves some problems for PPM markets, as an app-based meter would theoretically record listening taking place on headphones. And it would be far easier to convince participation than requiring them to carry a pager-like device!

It may also solve some of the recruitment issues, with in-app and social media advertising seeking respondents. Yet, it introduces other issues. For example, even though smartphones have become much more common, it would still exclude a percentage of the potential population. This may increase sample sizes but reduce reliability of the type of person that would participate.


There are many problems, but small sample sizes may be the single biggest issue with radio ratings today. The entire process is flawed. But until broadcasters and the advertising community become more vocal and proactive, major changes are unlikely.

How Radio Ratings Work

The Components of Radio Ratings: Understanding The Math

Why Radio Ratings Sample Sizes Are So Low

Pros and Cons of PPM Ratings

Radio Ratings: Higher Shares Are Driven By Daily Cume.

Why Do Radio Contests Work? Because Ratings Respondents Play

Ratings Case Study: If You Made Decisions Based On This Information, You Would Be Fired…

Double Your Ratings Seminar on Demand

Opt In Image
Topics - Bits - Features - Games - More

"The service is really outstanding. I love the moral dilemma situations each day. We get tons of feedback from listeners. The topics in Personality Magnet drive real listener calls." - Andreas Hinsberger, Programming Chief, Antenne Vorarlberg, Austria

Just $24/month or $249/year.




Fill Your Phone Lines Seminar on Demand
Webinar, $19.00

FILL YOUR PHONE LINES: It’s harder than ever for radio shows to generate phone calls today. But that shouldn’t stop you from filling your phone lines. In this seminar, I’ll give you practical, useful […]

The New Insiders Radio Network

Is This Really Radio's Most Valuable Resource For Personalities Programmers and Promotion Managers?