The art of designing questionnaires

Margaret Grosh and Paul Glewwe, in their book Designing Househld survey Questionnaires for Developing Countries, outlined the ideal process of survey design:

First, survey planners must define the fundamental objectives of the survey and decide on the overall design of the survey in light of these objectives. Second, within this general framework the survey planner must choose which modules to include in the questionnaires, the objective of each of these modules, and the approximate length of each module. Third, the planners must work out the precise design of each module, question by question, in light of the module’s specific objectives and approximate length. Fourth, the modules must be integrated with each other and combined into a complete set of draft questionnaires (household, community, price, and, in some cases, facilities). Fifth, the draft questionnaire should be translated (if applicable) and field tested. Ideally, the five steps should be completed in chronological order.

But they also noted that:

However, in practice, implementing any given step may reveal information that requires survey designers to rethink a previous step.

Designing questionnaires is great fun. But your questionnaire only makes sense when you have a clear goal in mind and phrase your questions in local vocabulary. When I first started designing my household survey to be carried out in Uganda this summer, I simply wrote down several open-ended questions about the topics I want to investigate. Only after I went over several large-scale household surveys did I realize how naive my idea was. Survey questionnaires are designed to close off logic holes and elicit the maximal information from respondents. This requires designers to have a good idea of the survey target.

I spent several afternoons tabulating data from Uganda Household Panel Survey 09/10 wave. This is part of the Living Standards Measurement Survey (LSMS) surveys designed by the World Bank. I limit my observations to central district — where I will be going — whenever possible. Several patterns stand out.

1. Polygamy is not uncommon. Females who are household heads are often married polygamous. Risk sharing behaviors might be very different in polygamous families than in monogamous families as a result of more relatives.

2. Insurance coverage is extremely low. Less than 1 percent of households in the national sample have household members covered by insurance. Now, instead of asking “do you have a health insurance” (for which I will likely get a “no”), I need to ask why they do not purchase health insurance. Do people Is there a need for formal health insurance? Are there available products? Are the products affordable? Is trust an issue?

3. Government health facilities are much less accessible than private clinics and traditional healers. But government hospitals are usually packed with patients while private clinics often have extra beds.

4. Response rate for the question “how much do you have in your bank account” is only moderate. My advisor suggested a reason for this: we are generally better at keeping track of flows, rather than stock values, in our personal assets.

Another young researcher who is doing household survey in Tanzania described her focus group discussions as “the most valuable” and said they are a good way to explore research questions as well as revise the language of the questionnaire. Going door to door for interviews is not completely new for me. I have conducted two surveys in secondary school, one on rural-to-urban workers and the other on newspaper vendors. Neither groups were friendly to interviewers, and the latter was pretty mobile. It would be fascinating to continue my survey work in Africa.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s