Northwestern University Political scientists Benjamin Page and Lawrence Jacobs have written a paper arguing that the deliberative forums put together this week by America Speaks were “unreliable” measures of public opinion, and that the public has already made itself clear on these matters, opposing benefit cuts to Social Security and preferring lifting the cap on payroll taxes to strengthen its finances.

George Washington University political scientist John Sides from The Monkey Cage summarizes the results of the paper. In addition to the demographic difficulties of finding a true random sample based on people who decide affirmatively to show up to a day-long meeting, there is the problem, which I noted in my report on the America Speaks event in LA, of biased presentation:

A second problem is that the value of deliberative forums depends entirely on the quality of the information presented. One specific concern, among others: if the information accentuates a particular problem, respondents may be more likely to see it as important. At the America Speaks forums, the emphasis — unsurprisingly, given the Peterson Foundation’s mission — is the deficit. Page and Jacobs write:

“A focus on the “challenge” of deficit reduction (repeatedly emphasized in the America Speaks briefing book), if it temporarily distracts forum participants from other important concerns, could lead them to say they would tolerate cuts in Social Security benefits that most Americans – even the forum participants themselves – would strongly oppose if asked about them in the normal way at home or at work.”

Page and Jacobs also argue that a large amount of evidence from public opinion surveys suggests, well, that the public doesn’t necessarily agree with the Peterson Foundation:

“The bottom line is that many Americans express concern about budget deficits, but many more see other issues (especially jobs and economic growth) as the top priority. Most Americans do not favor cutting popular programs like Social Security (or education or health care) in order to reduce budget deficits. Support for Social Security is strong and widespread across the population, including among young people. Many more Americans want to increase spending on Social Security than want to decrease it, and that has been true for decades. Virtually any sort of benefit cut is opposed by substantial majorities of Americans.”

These findings actually held in the America Speaks results, and I think part of this is due to a higher proportion of liberals actually coming out to the meetings. However, the options for deficit reduction – and the foregrounding of deficit reduction over economic growth – narrowed the range of results. In one meeting in Portland, participants demanded to be allowed a vote on single-payer health care as a deficit reduction strategy, for example. But that wasn’t on the menu.

Andrew Gelman, a political scientist from Columbia University, concurred with his colleagues that the forums were “a pretty useless way to assess public opinion,” and focused on the role of the presentation materials as well, and the Peterson Foundation which funded them:

As Page and Jacobs note, it’s not just that Peter Peterson is a fiscally conservative Republican, it’s that his foundation is specifically focused on deficit reduction, which in turn is the topic of their study.

The right way to theink about the America Speaks forum, I think, is not as an attempt to measure public opinion but rather as an attempt to influence public opinion. As Jacobs and Shapiro write in Politicians Don’t Pander, political actors often view public opinion not as a fixed constraint but instead as a tool that they can use to influence policy.

Gelman said that the event had three goals: securing publicity for the notion of cutting spending as a deficit reduction maneuver; testing political messages in a large-group setting, as research for future public ads; and using the results (which America Speaks “balanced” in its press release as encompassing spending cuts as well, even though those options were far less supported than the more progressive measures which came in first) to influence elite opinion. Public opinion, in the latter case, becomes a means to an end.

The paper from Page and Jacobs is quite interesting, if you’re interested in this topic it’s worth reading. America Speaks created an artificial “fish bowl” of faux-public opinion, narrowed the range of options, and then touted the results as more “deliberative” than a poll. This is flawed political science, and it’s good to see that community recognize it.