# Posted 6:24 PM by Ariel David Adesnik  

HOW THE NYT MANUFACTURES PUBLIC OPINION: Take a look at the front page of today's NY Times. When you're done looking at the stunning photo of colorfully-attired entrepreneurs in Bangladesh, look over to the headline at the top left that reads "Support For War Is Down Sharply, Poll Concludes".

That's some pretty big news, especially given that last weeks WaPo/ABC poll resulted in the headline "Poll Shows New Gains For Bush". So what happened? According to the first three grafs of the NYT article,

Support for the war in Iraq has eroded substantially over the past several months, and Americans are increasingly critical of the way President Bush is handling the conflict, according to the latest New York Times/CBS News Poll.

After initially expressing robust backing for the war, the public is now evenly divided over whether the United States military should stay for as long as it takes to stabilize Iraq or pull out as soon as possible, the poll showed.

Asked whether the United States had done the right thing in taking military action against Iraq, 47 percent of respondents said it had, down from 58 percent a month earlier and 63 percent in December, just after American forces captured Saddam Hussein. Forty-six percent said the United States should have stayed out of Iraq, up from 37 percent last month and 31 percent in December.

The first thing you need to know about the NYT poll is that there was a single question (#56) which asked

Looking back, do you think the United States did the right thing in taking military action against Iraq, or should the US have stayed out?

If you don't think that the US did the right thing, then you have to answer that it should've stayed out (or just not answer the question). I think that's somewhat of a misleading question, since there are probably a good number of Americans who believe that we did was right but, in hindsight, was a mistake. However, in order to get a fuller sense of why the NYT poll is misleading, you have to take a look at the sidebar entitled "Different Poll Results But Much in Common". The sidebar explains that

Wording differences among polls can [also] have a significant effect. In the ABC News/Washington Post poll, 51 percent said that the war in Iraq had been worth fighting, "all in all, considering the costs to the United States versus the benefits to the United States." That number was down from 56 percent in January and 59 percent in December. Forty-seven percent said it was not worth fighting, up from 41 percent in January and 39 percent in December...

Every polling organization has different ways of wording questions and of conducting surveys, but those methods tend to remain consistent over time within an organization. For this reason, the trends within an organization's polling are generally viewed as more relevant than the results from polls by different organizations within a short period of time

Now, I agree in principle that trends over time are an extremely important indicator of public opinion. But depending on what window of time you're looking at, trend lines can have a very different message. For example, the NYT started asking its 'done the right thing/should've stayed out' question (#56) in mid-December 2003. Thus, the 17 point decline among those who answered 'done the right thing' reflects the artificial high in support for the war that followed Saddam's capture. Moreover, there was only a 6 point decline from December to March, then an 11 point decline from March to April. Does that kind of single observation merit its own headline?

[CORRECTION APPENDED 5/9: JH points out that the NYT asked Question #56 both immediately before (Dec. 10-13) and immediately after (Dec. 14-16) Saddam's capture. There was only a one point difference between the answers. Thus, I was wrong to say that the 63 percent benchmark represented an artificial post-capture high.

One should point out, however, that the NYT had never asked Question #56 before Dec. 10-13/Dec. 14-16 and did not ask it again for a number of months. So first of all, there is no real benchmark against which to measure the mid-December results. This is especially important, since the second mid-December survey took place while the news of Saddam's capture had begun to spread. Second of all, one cannot confirm whether or not the December results were accurate since no further samples were taken until March.

By way of comparison, Question #13 in the WaPo survey (see below), which is analogous to Quesiton #56, was asked consistently over time. It demonstrated that there was, in fact, a post-capture high in support for the war, one which gradually dissipated thereafter.]

Perhaps. Before answering that question, take a look at the answer to Question #13 in the WaPo/ABC poll, which asked

All in all, considering the costs to the United States versus the benefits to the United States, do you think the war with Iraq was worth fighting, or not?

On April 18, there was a 51-47 margin in favor of 'It was worth fighting'. But what's really interesting is that the Post has been asking the exact same question at regular intervals since last April. By last July, support for the war had already fallen below 60 percent. In November if tell to 52 percent. It then rose to 59 after Saddam's capture before falling back to 51 percent today.

So if the NYT staff is so well aware of the important of trends over time, how did they manage to ignore the most important evidence against their conclusion that support for the war is dramatically down? Who knows.

Anyhow, I also think its extremely interesting that the NYT poll doesn't have a question like #11 in the WaPo poll, which asks respondents to say whether Bush or Kerry would deal better with a specific issue (taxes, healthcare, etc.). Instead, the NYT poll asks separate questions (#46 and #51) about whether Bush and Kerry will be able to handle the situation in Iraq effectively. Surprisingly, both men get around a 60-40 vote of confidence, with Kerry doing just slightly better. When it comes to fighting terrorism (Questions #47 and #51), both men get around a 75-25 vote of approval, with Bush supporters more likely to have a lot of confidence in the candidate.

Given that Bush continues to beat out Kerry by 30-point margins whenever voters are asked who will do a better job of fighting the war on terror, I think it's fair to say that asking separate questions about the candidates ability says very little about whom voters prefer. After all, the best way to explain why voters disapprove of how Bush is handling the situation in Iraq but still want to vote for him is that they think Kerry would do even worse.

In closing, I'd like to take a quick look at the second statistic that the NYT uses to demonstrate that suppor for the war has fallen sharply. According to Question #69, the public is evenly split at 46-46 about whether the US should stay in Iraq until it is a "stable democracy" or whether it should pull out "even if Iraq is not completely stable".

Again, the trend line on this question only goes back to last November, when the split was 49-43. It rose to 56-35 after Saddam's capture. Then, for some reason, the NYT didn't ask the question again until this past week. As a result, there was a sudden 10 percent drop in support for sticking it out.

The most comparable question to that in the WaPo/ABC poll is #17, whether the US should stay in Iraq until it restores order, even if that means taking more casualties. Surprisingly, the margin on that one is 66-33 in favor. But again, the question hadn't been asked since last fall, so there isn't much of a trend line to look out.

All in all, I'd say the NYT has a lot to learn about interpreting poll results and even more to learn about writing headlines.