100 Percent of Pollsters Say Their Job is More Difficult
May 11, 2011
Doing accurate research through random telephone polling is important. It can help assess the public’s stand on key issues, rate the effectiveness of government services, assess attitudes about new and existing products and help predict the outcome of elections.
Our company oversees many, many surveys each year, and we stand by the accuracy of these instruments, as would researchers who have tested their veracity in a variety of ways. But getting people on the phone with pollsters is getting more and more difficult. And expensive.
Tony Parker, whose Birmingham-based firm the Parker Group oversees calling for most of our firm’s surveys, says that when he started doing calls for surveys more than 20 years ago, he was able to get one in every six people to agree to answer a list of questions. Today, only one in every 20-30 calls results in a completed survey.
The reasons for the declining success rate:
- Cell phone-only households, which comprise 20 percent of the U.S. population.
- Caller ID, which often warns the home’s residents that a survey-taker or an unknown caller is ringing.
- Voicemail, to which some households with “land line” phones send all calls for screening.
- Incomplete databases of cell phone numbers. People who list their cell phone number for credit applications, driver’s license applications or for White Pages listings probably will have their numbers added to databases. But people who list their land line phone on these applications are probably not going to have their cell phone included in publicly accessible lists.
Younger citizens – those under 35 – are especially hard to reach, Parker said. Because these people are the least likely to vote, their inaccessibility is less of an issue for a political pollster than for someone testing the market for blue jean preferences.
Does that mean that Parker, who does calling for a living, or McNeely Pigott & Fox, which writes and analyzes surveys for clients, are giving up and going home? Certainly not. They just have to work harder to get to the right survey participants.
Though they are not complete, there are many good cell phone lists available – for a hefty price – that yield not only better success rates, but also a generally higher percentage of younger interviewees. These lists can be ordered and crafted to produce a better overall demographic sample.
Timing calls to when your target interviewees are available is also a key.
While Parker has noted a trend among some pollsters toward combining the results of online surveys with those of randomly generated telephone interviews, doing anything other than random phone interviews (such as online or mailed written surveys) can negatively affect the accuracy of the product.
We, too, strongly advise our clients against putting much stock in self-selecting online or written surveys because they just aren’t very accurate.
Bottom line: A scientific random survey of 400 people 20 years ago produced a “margin of error” of about 4.88 percent. Today, that same size survey generates exactly the same rate of accuracy. It’s just more expensive and difficult to get there.