A fake poll tested the media and poll trackers. Who passed? The Los Angeles Times landed a big polling scoop Tuesday: | A poll claiming Los Angeles Mayor Karen Bass was leading Councilmember Nithya Raman by a wide margin — and picking up support from most voters who backed Spencer Pratt in the mayoral primary — was bogus, the company behind the poll acknowledged Monday. | | | | The fake polling firm fessed up after L.A. Times inquiries, posting a statement saying “Median Strategies was created as a short term social experiment to examine how purported polling information could enter and spread through the political information ecosystem without independent verification.” The group did not name any individuals or real organizations involved, leaving the effort shrouded in mystery. Before the poll was exposed, the California Post and Fox 11 Los Angeles featured it in their reporting. Mayor Karen Bass (D) touted the favorable poll results on X, and two anonymous X accounts, @PollTracker2024 and @OpenSourceZone, shared it to their more than 50,000 followers. Neither the California Post, Fox 11 nor @PollTracker2024 responded to a request for comment about their coverage of the fake poll. In response to questions, a spokesman for @OpenSourceZone said in a statement to The Post that it “relies on publicly available polling information and does not independently verify the underlying data of every survey. Median Strategies presented itself as a polling organization, and its polling appeared legitimate based on the information available to us at the time.” The site added that “we’re reviewing our standards for evaluating newer polling organizations, including what qualifies as an established pollster, following this incident.” Other outlets and poll trackers didn’t bite The L.A. Times’s Noah Goldberg sniffed out the bogus poll within a week of it gaining significant attention. “I found the poll suspect immediately because the firm was so new, had so few followers, and did not have an actual human pollster listed on its website,” Goldberg wrote in an email to The Post. “We have pretty high standards at The Times when it comes to writing about a poll. A polling outfit must have cross tabs available for us and they have to have a pollster! If there is no human associated, that is a pretty big red flag in my opinion.” Median Strategies also was not listed by three major poll-tracking websites, including the New York Times, RealClearPolitics and FiftyPlusOne. I asked each organization how it evaluated the Median Strategies poll, with two providing detailed answers: | “When the poll was released, it lacked basic information about how it was conducted and who was behind it. Releasing this type of information is standard practice for pollsters, and because that information was not provided, we did not include it. After it received attention by other media outlets, we attempted to contact the pollster to get more information, as is our typical practice for an unknown pollster. They never responded.” William P. Davis, director of election data analytics at the New York Times | | | | | “For every new pollster, we conduct some vetting to ensure that the surveys meet our methodological standards. ... For Median Strategies, their website looked plausible enough, but didn’t include key information about the principals and their background. We do not publish anything if we don’t have names behind the research. I reached out to them with questions about their organization and did not hear back.” Mary Radcliffe, head of research at FiftyPlusOne | | | | “Bravo,” said longtime pollster Mark Blumenthal, who helped create pollster.com, one of the first big poll-tracking websites in the mid-2000s. “The fact that common aggregators have put transparency standards into place helped keep this bogus data out of their aggregates.” Blumenthal said his bigger concern is “fly-by-night X and other accounts with no standards, no details, often without a link. … I sincerely wish journalists and other people with a following would stop sharing those bare-bones tweets.” More vetting of polls is good The scrutiny described by the L.A. Times, New York Times and FiftyPlusOne showcases the unglamorous, rarely seen work of poll vetting. In my first job on the ABC News Polling Unit, I helped maintain spreadsheets tracking 2008 presidential primary polls. Whenever a new pollster released a survey, I called to ask for the full questionnaire, how the sample was drawn and any other details if they were not described in the report. Based on those answers, the polling team decided whether the poll was “airworthy.” The Washington Post also vets each poll cited in our coverage — as do several other news organizations — but the task has gotten bigger with more than 200 organizations conducting polls ahead of the 2024 presidential election. The American Association for Public Opinion Research’s (AAPOR) Transparency Initiative has led many firms to report more about their polls, yet many election pollsters have not committed to it. The Median Strategies episode showcased the strengths and weaknesses of the current polling ecosystem. A fraudulent poll in a prominent election got some attention, but it was shunned by mainstream poll aggregators and exposed by a reporter. How can you tell if a poll is legit? Fake polls are rare, although the quality of published polls varies widely. If you read a poll and wonder whether you should trust its results, here are four questions to ask: - Is there a survey report with detailed survey methods, including question wording, sampling methods, interviewing modes and weighting (not just “conducted online”)?
- Was it conducted by a named researchers with a website describing their qualifications?
- Was the survey conducted independently of a political party, campaign or advocacy group?
- Is the firm a member of AAPOR’s Transparency Initiative, or are its polls cited by poll aggregators like the New York Times, FiftyPlusOne and RealClearPolitics? Silver Bulletin is a useful source for data on historical poll accuracy.
These don’t cover everything, but a “no” to any of these should warrant caution and will identify the most common problems. AAPOR’s cheat sheet for understanding polls is also a useful resource. A Gallup poll released Thursday found 54 percent of Americans drink alcohol occasionally, tying last year’s record low in polling since 1939 and down from 58 percent in 2024 and 62 percent in 2023. |
