Design and pre-test questionnaires: Designing the questionnaire carefully and then pre-testing it before fielding it to your entire sample is crucial to getting data that are valid and reliable. For example, careful questionnaire design and pre-testing can help reduce the chance that respondents may interpret the meaning of questions differently. Future posts in this series will tackle these important steps in much greater detail.
One of the most common formats used in survey questions is the “agree-disagree” format. In this type of question, respondents are asked whether they agree or disagree with a particular statement. Research has shown that, compared with the better educated and better informed, less educated and less informed respondents have a greater tendency to agree with such statements. This is sometimes called an “acquiescence bias” (since some kinds of respondents are more likely to acquiesce to the assertion than are others). A better practice is to offer respondents a choice between alternative statements. A Pew Research Center experiment with one of its routinely asked values questions illustrates the difference that question format can make. Not only does the forced choice format yield a very different result overall from the agree-disagree format, but the pattern of answers among better- and lesser-educated respondents also tends to be very different.