[MUSIC] Hi, in this module, I'm going to talk about some of the practical considerations that come up in conducting a survey. Assuming that you've already figured out how you're going to do your sampling. When we think about surveys, there's actually several different formats, several different types, which are typically associated with specific applications. One of the earliest forms, which is not so common now, are self-administered surveys. You remember the literary digest poll that we talked about in a previous module. Where the magazine literary digest mailed postcards to millions of people. Asking them to indicate their preference for president and then send those post cards back. That was an example of a self administered survey. Nowadays, they are a lot less common. Sometimes you see them, you receive one in a mail. If it's a marketing survey. Actually the most common though that are still being used are for government surveys. So the census at least at the very recently was typically delivered by mail. And then people filled out census forms at home, on their own, and mailed it in. And there are some other government household surveys and so forth that are still in a self-administered format. In fact, involving mail. Now another type of survey, another format that you've probably encountered are in person surveys at venues. Now these represent convenient samples, which you're unlikely to use in academic research. But they're fairly common for marketing. You've probably been at some point at an airport where perhaps somebody came up with a clipboard. To ask you questions about what you thought about the airport. That's an example of an In-person survey again, at a venue with a convenient sample. A more academic type of survey is an in-home survey, conducted in person. Where a interviewer or a team of interviewers actually goes to visit homes and sits down with the members of the household or a specific member of the household to interview them. Again for the really big longitudinal surveys, the big panel surveys that I've often talked about. This is the most common approach because sometimes the questionnaire takes several hours. It may be quite complex and require the work of a trained interviewer to really go through the questionnaire. Make sure that the questions are asked correctly. Of course, there are telephone surveys. For decades these were extremely important for opinion polling. Because it was possible with a telephone survey to very quickly accumulate a set of responses on opinions about important issues. So political polling and so forth, as well as marketing, made heavy use of telephone surveys. They're still in use today. But they're becoming quite difficult. Because many people no longer answer their phones, or they don't trust incoming calls. Finally, an area of that's been getting a lot of attention recently is online surveys. In many ways, the online surveys that you are most likely to have seen probably are more for entertainment than for any serious use. Where you maybe sent a link to indicate your preference for some band or some movie or your casual opinion about some topic. In fact though, there are some efforts to make serious use of online surveys. There's debate, that I'll talk about later, in terms of the representativeness of surveys. But there are other approaches, including hybrid surveys, that include, say, in-person visit, at the outset with follow up via online surveys. I want to talk a little bit about some practical considerations that come up when you're designing and testing a questionnaire. In fact, most people learn how to design questionnaires by joining projects, and so forth. Where they gain hands-on experience, in the design of questionnaires. So here I can only highlight a few things that you need to pay attention to, some general guidelines. First thing to keep in mind is that designing a questionnaire is a major effort. It's not trivial, it requires a lot of work. Now, one way in which you can help yourself is to keep in mind that you don't have to reinvent the wheel. When you're designing a questionnaire. In fact, it's very typical for people designing questionnaires, when they are finding ways to ask about, especially basic information, demographic characteristics, employment and so forth. To follow or use questions that have already appeared in other questionnaires. So there are thousands of questionnaires from surveys already conducted. They're already online. And normally, especially for collecting really basic information, like age, sex, race, employment, marital histories and so forth. It's fairly typical to reuse questions that appear in other questionnaires. Now there are a few situations, especially with psychological batteries, where there may be copyright issues. But for the most part, questions are expected to be reused. And in fact, reusing questions from other questionnaires, helps enhance comparability. Makes the results easier to compare to other studies. So again, do yourself a favor. Don't re-invent the wheel. Especially for the basic sorts of information that are collected in every survey. So, again, following the standards helps enhance comparability. And increases the chances that perhaps some results that you get from your sales survey could be compared to some results from a similar survey in another country, for some kind of international comparison. Now, another consideration that you have to keep in mind is that almost all questionnaires go through multiple rounds of pre-tests. To verify the quality of the wording of the questions. To make sure that the wording is clear. That people understand how to answer the questions. And then when they respond, they give the right sorts of responses. This typically begins informally. Perhaps by circulating a questionnaire to colleagues, students, coworkers, friends simply to give feedback on the wording of the questionnaire. Even if they're not necessarily members of the target population. Now for a very large study, in which the eventual target is perhaps thousands or tens of thousands of people or households. Then a pre-test maybe a formal part of the project where in fact interviewers go out into the field and then conduct the pre-test. To make sure that respondents in the population that is the target of the study, actually can respond in a meaningful way to the questions. Another area where I want to offer some general thoughts or considerations with respect to interviewers. Anything other than a small survey requires interviewers. It's unlikely that if your interview seeks more than 100 or 200 people that you will actually do all the interviewing yourself. You'll have to get interviewers. Interviewers have to be trained. They have to be paid. And they have to be monitored. Poorly trained and poorly monitored interviewers may make mistakes or make terrible mistakes. In fact, they may even cut corners. So there are cases over the recent decades of interviewers where they were perhaps as part of a sampling design. Assigned on a particular day to go visit perhaps two or three households to conduct their interviews. But maybe when they went there the people weren't home. Or, they couldn't gain access to the building. Or they thought the neighborhood was dangerous. So, they went to a coffee shop and then filled out the interviews themselves. This is called in fact coffee shopping where interviewers simply, rather than conduct an interview, make up the answers. Now, that's an extreme problem. But there may be more innocuous problems that interviewers may have if they're not trained properly. So, for example, sometimes interviewers, especially if they're pressed for time, may try to answer on behalf of respondents. So, for example, if the questionnaire includes a question about how many children the respondent has. But perhaps the interviewer feels the interview is lagging. That the interviewee can't think or respond clearly and they notice there's a photo on the wall of the interviewee with three children. They may just go ahead and fill in three without asking the question. That's not appropriate. And interviewers have to be trained to avoid those sorts of shortcuts. Skilled interviewers can make the difference between a successful and an unsuccessful study. Now, another phenomenon is that interviewer's appearance and their demeanor may affect the respondent's answers. So if the interviewer appears to be very high status, speaks with a particularly high status accent, that may intimidate interviewees. They may be constrained, in terms of giving answers that they might feel would embarrass them in front of a high status person. So this is an area that's gotten a lot of attention in recent decades. Now I've been talking a lot about interviewers, but I want to turn in just a second, to some more general issues that reflect all questions and questionnaires.
