Contact Us Search our Site Home Page
The Maureen and Mike Mansfield Foundation
Japanese site About the Foundation Asian Opinion Polls Mansfield Fellowships Foundation Programs Publications and Outreach Resource Links

The Mansfield Asian Opinion Poll Database


Assessing Public Opinion Data: What to look for, and what to look past

by William Watts, President, Potomac Associates
February 7, 2007

The Mansfield Asian Opinion Poll Database is an important contribution to the growing field of Asia-related public opinion research. With the rapid emergence of the region as a global economic powerhouse, and with the many and varied security and other issues that challenge the region’s political and economic structure, this new resource is one that policy makers, scholars, journalists, and others will come to rely upon in their several endeavors. In the daily process of separating the informational wheat from the speculative chaff, this is a valuable addition. We look forward to seeing the Database expand its network of participating polling organizations in the future.

Given the enormous volume of survey research efforts now available, it may be useful to offer a few comments on the nature of such findings. What are the special values that are associated with opinion research, what are some of the pitfalls, and what are the most useful lessons to be drawn from individual surveys?

Trends, not predictions
It is crucial to remember that any given survey is simply a snapshot, a picture of a slice in time. It represents the views and attitudes of a selected sample of the public, captured at a specific moment, and always subject to change driven by events or personal feelings. Just as last year’s family photograph may look very different from one taken yesterday, so too will earlier surveys almost certainly differ from latest findings. While this may seem quite obvious, one must be wary of the temptation to attribute to this morning’s readings of opinions any unwarranted permanence.

Surveys do not predict. All too often, individual survey results are used by their interpreters to make assertions about what is likely to come next, about what the future is likely, if not certain, to hold. This hidden snare is one to be carefully avoided. What is valuable, even key, is to watch for trends. When we are looking at attitudes of individuals toward other nations, for example, it is definitely instructive to see how those attitudes change over time. Fortunately, many survey organizations make a point of repeating certain trend questions verbatim, which does permit this kind of trend analysis. As the database builds up, these trends do provide a better opening to try to look at a more likely future picture – a measured ability to come close to predicting.

My use of the word verbatim is intentional. For any trend question to be truly useful, the wording must be consistent. Otherwise, statistical reliability is lost, and any meaningful trend comparisons go out the window.

In looking at trends in questions related to foreign affairs and foreign policy issues, it is worth noting that opinions on such themes tend to change very slowly. Attitudes toward other countries, for example, in terms of “warmth,” “like/dislike,” or “friend/enemy” generally don’t vary much from year to year. It takes major events to move the larger public consensus in a measurably different direction. A good case in point was the collapse of Japanese opinion toward China, which fell sharply after the Tiananmen massacre, and has since then never fully recovered. Absent such dramatic events, opinion tends to move slowly over time.

The importance of a single variable
In addition, it is critical to note that good survey questions contain only one variable. If, for example, a Japanese respondent is asked to give his/her attitudes toward North Korea in light of Pyongyang’s nuclear program and kidnapping of Japanese citizens, the responses are of only marginal value to the reader or analyst: Is the nuclear issue the driving force in the respondent’s opinion, is it the kidnapping issue, or is it some combination of both? Any realistic analysis is moot.

A specific and relevant example is contained in the 2005 Dong-a Ilbo survey, contained in the Mansfield database. Korean respondents were asked: “Do you feel in your daily life that South Korea’s economic relations with China and Japan have been deepened over the last 5 years?”. Which country is the Korean being interviewed referring to in his/her answer? China? Japan? Both? There is no way we can be sure.

The danger of composite findings
Another key factor to keep in mind relates to surveys that contain composite findings that may mask significant differences below the surface. Take, for example, the theme of “red” versus “blue” states in U.S. politics. It is common to see generic political maps of the United States broken down by so-called “red” and “blue” patterns. That may be at least superficially interesting, but it masks a much more complicated reality. If you take each state and break down the voting outcomes by individual voting districts, you find a remarkably different picture. The most accurate characterization is a bad case of measles, with most states impressively pock-marked – red and blue – from border to border.

Most survey results do contain a range of demographic or other breakdowns, and differences that emerge among the various subgroups are always ripe for analysis and discussion. One such subset that is frequently looked at by commentators is age groupings. Older age cohorts frequently offer views that can be seen as more conservative than their younger counterparts. This is, also frequently, interpreted as meaning that a new or differing attitudinal landscape is emerging – a predictor of changes in things to come. But that predictor may also be overtaken by the passage of time itself.

Measuring attitudes
This may be especially true in trying to get a fix on the growth (or lack thereof) of anti-Americanism. There is at least a pair of factors at work here. First, it is important to realize that today’s younger cohort will fall into tomorrow’s middle-aged or older age group. Readers may recall that many younger Japanese, who in June 1960 were storming the U.S. embassy in downtown Tokyo in protest against the forthcoming visit of President Dwight D. Eisenhower (and ultimately forcing cancellation of that visit), have since become pillars of the Japanese establishment and leading proponents of close U.S.-Japanese ties.

In addition, views that on one hand are seen as “anti-American” can easily move into a more “pro-American” mode, depending on the issue being addressed. In an earlier study of the views of younger Koreans who were seen as likely future leaders (not included in the Mansfield data base, but relevant to this discussion), respondents in this group ranked the United States first in expressions of positive, as opposed to negative, personal feelings, with China, Japan and, especially, Russia lagging behind.

But this broadly positive reading contrasted sharply with specifics about the “biggest problems in relations with the United States.” Random complaints included: arrogance; unilateralism; Pax Americana” and “hegemonism;” excessive economic pressure; cultural insensitivity; “flunkeyism” and Korean dependency; the impact of the U.S. military presence; and aggressive U.S. policies that heighten South-North tensions. These more negative personal feelings seemed to contradict the warmer outlooks expressed about the United States as a whole. What we see here is a differentiation between generalized attitudes and the specifics of policy-driven views.

To take this one step further, we are all witness at the present time to the sharp decline around the world in attitudes toward the United States. While at least some of this has eroded the overall image many abroad hold of America, it is clear that much of the anti-U.S. feeling is directed against specifics of American policy, rather than representing a condemnation of America per se. This is demonstrated in material in the Mansfield database. Japanese, for example, rank the United States ahead of China, South Korea, and several other countries both in terms of “having an affinity for,” or thinking of relations as “good.” There is, in effect, a disparity between critical “noise” and underlying feelings that reach beyond current complaints or feelings of animosity.

One closing comment is in order, intended particularly for American readers of material in the Mansfield data base. In looking at survey research results, it is important to remember that the United States has interests all around the globe. The American radar screen is filled with blips throughout the entire 360 degrees of the radar band sweep. For most other countries around the world, and this probably applies to many within the Mansfield database universe, the United States represents a magnified blip on their screens. This means that their focus on the U.S., its policies, and its impact, is especially intense, and can give rise to higher expectations and sharper, more pointed, evaluations. One thing Americans need to understand is that their lesser focus on developments in other countries and other parts of the world is clearly outmatched by the greater attention that others pay to what we do. This is an added lesson, and bonus, in what the Mansfield database brings to American readers. And it is a reminder of the relative insularity in which many, perhaps most, Americans go about their daily rounds.


About the Foundation | Asian Opinion Polls | Mansfield Fellowships
Foundation Programs
| Publications & Outreach | Support the Foundation
Contact Us | Search Site | Home Page

© 2005-2009 The Maureen and Mike Mansfield Foundation