3.5.3 Gamification

Standard surveys are boring for participants, but that can and must change.

So far, I’ve told you about new approaches to asking that are facilitated by computer-administered interviews. However, one downside of computer-administered interviews is that there is no human interviewer to help induce participation. This is a problem because surveys are both time-consuming and boring. Therefore, in the future, survey designers are going to have to design around their participants and make the process of answering questions more enjoyable and game-like. This process is sometimes called gamification.

To illustrate what a fun survey might look like, let’s consider Friendsense, a survey that was packaged as a game on Facebook. Sharad Goel, Winter Mason, and Duncan Watts (2010) wanted to estimate how much people think they are like their friends and how much they are actually like their friends. This question about real and perceived attitude similarity gets directly at people’s ability to accurately perceive their social environment and has implications for political polarization and the dynamics of social change. Conceptually, real and perceived attitude similarity is an easy thing to measure. The researchers could have just asked lots of people about their opinions and then asked their friends about their opinions (this allows for measurement of real attitude agreement) and they could have asked lots of people to guess their friends’ attitudes (this allows for measurement of perceived attitude agreement). Unfortunately, it is logistically very difficult to interview both a respondent and her friend. Therefore, Goel and colleagues turned their survey into a Facebook app that was fun to play.

After a participant consented to be in a research study, the app selected a friend from the respondent’s Facebook account and asked a question about the attitude of that friend (Figure 3.9). Intermixed with questions about randomly selected friends, the respondent also answered questions about herself. After answering a question about a friend, the respondent was told whether her answer was correct or, if her friend had not answered, the respondent was able to encourage her friend to participate. Thus, the survey spread in part through viral recruitment.

Figure 3.9: Interface from the Friendsense study (Goel, Mason, and Watts 2010). The researchers turned a standard attitude survey into a fun, gamelike experience. The app asked participants both serious questions and more lighthearted questions, such as the one shown in this image. Image used with permission from Sharad Goel.

The attitude questions were adapted from the General Social Survey. For example, “Does [your friend] sympathize with the Israelis more than the Palestinians in the Middle East situation?” and “Would [your friend] pay higher taxes for the government to provide universal health care?” On top of these serious questions, the researchers mixed in more lighthearted questions: “Would [your friend] rather drink wine over beer?” and “Would [your friend] rather have the power to read minds, instead of the power to fly?” These lighthearted questions made the process more enjoyable to participants and also enabled an interesting comparison: would attitude agreement be similar for serious political questions as lighthearted questions about drinking and superpowers?

There were three main results from the study. First, friends were more likely to give the same answer than strangers, but even close friends still disagree on about 30% of the questions. Second, respondents over-estimate their agreement with their friends. In other words, most of the diversity of opinions that exists between friends is not noticed. Finally, participants were as likely to be aware of disagreements with their friends on serious matters of politics than lighthearted issues about drinking and superpowers.

Although the app is no longer available to play (unfortunately), it was a nice example of how researchers can a standard attitude survey into something enjoyable. More generally, with some creativity and design work, it is possible to improve the user experience for survey participants. So, next time you are designing a survey, take a moment to think about what you could do to make the experience better for your participants. Some may fear that these steps toward gamification could hurt data quality, but I think that bored participants are a far greater risk to data quality.

The work of Goel and colleagues also illustrates the theme of the next section: linking surveys to other data sources. In this case by linking the survey with Facebook the researchers automatically had access to a list of the participants’ friends. In the next section, we will consider the linkages between surveys and other data sources in greater detail.