Research Methods: Questionnaires

A questionnaire, or social survey, is a popular research method that consists of a list of questions.

If administered directly by the researcher to the subject in person then this is the same as a structured interview; however, questionnaires can also be completed independently (self-completion questionnaires) and therefore administered in bulk, through the post or electronically for example. The method can use closed or open questions or indeed a mixture of the two, depending on what sort of data is desired and how the researcher intends to analyse it.

Reliability and Validity of Questionnaires

In the context of research, the reliability of a method refers to the extent to which, were the same study to be repeated, it would produce the same results. For this to be the case, samples need to be representative, questions or processes need to be uniform and data would generally need to be quantitative. Researchers need to be confident that if they repeat the same research and the result is different that what they are studying has genuinely changed and not just that their original method was not sufficiently reliable. If you take the example of opinion polls on people's voting preferences: if the support for parties changes by several points, the researchers (and their "customers") need to be confident that this is because people are really changing their minds about how they intend to vote; that it is not simply that the research method is unreliable and therefore changes between polls are likely and unpredictable. If that were the case it would render their data useless.

Questionnaires are generally considered to be high in reliability. This is because it is possible to ask a uniform set of questions. Any problems in the design of the survey can be ironed out after a pilot study. The more closed questions used, the more reliable the research.

Valid research reveals a true picture. Data that is high in validity tends to be qualitative and is often described as "rich". It seeks to provide the researcher with verstehen - a deep, true understanding of their research object. The validity of data produced by questionnaires can be undermined by the use of closed questions which limit respondents' answers.

In a questionnaire (or structured interview) it is possible to ask open questions or closed questions. Closed questions are those with a limited number of possible responses, often "yes" or "no". Closed questions help to make data easier to analyse and more reliable. This is because closed questions produce quantitative data. However, restricting responses can impact validity. To try to overcome this, sociologists often broaden possible responses to closed questions, by, for example, ranking possible responses or indicating the degree of agreement with a statement. The latter is known as the Likert Scale, and is a way of quantifying qualitative data for ease of analysis. It is also possible to mix closed questions with an open "other (please specify)" option.

Open questions do not limit the possible answers that the responder can give, producing qualitative data which is generally considered to be higher in validity. This is because it can be detailed and the respondent can give their own views, rather than be limited by the assumptions of the researcher. However, such data can be very difficult to analyse. There is also the danger that options are simply limited during analysis rather than design (ie. the researcher puts the wide range of responses into a smaller number of categories in order to analyse them). This depends on the researcher's interpretation of the respondent's response which could be affected by subjectivity or the researcher's values.

Because questionnaires are usually used to produce quantitative data, they are generally thought to be more reliable than valid. However, they do have the advantage of being able to produce a mixture of reliable and valid data, known as triangulation.