We are always going to need to ask people questions.
Given that more and more of our behavior is captured in big data sources, such as government and business administrative data, some people might think that asking questions is a thing of the past. But, it’s not that simple. There are two main reasons that I think researchers will continue to ask people questions. First, as I discussed in chapter 2, there are real problems with the accuracy, completeness, and accessibility of many big data sources. Second, in addition to these practical reasons, there is more fundamental reason: there are some things that are very hard to learn from behavioral data—even perfect behavioral data. For example, some of the most important social outcomes and predictors are internal states, such as emotions, knowledge, expectations, and opinions. Internal states exist inside people’s heads, and sometimes the best way to learn about internal states is to ask.
The practical and fundamental limitations of big data sources, and how they can be overcome with surveys, are illustrated by Moira Burke and Robert Kraut’s (2014) research on how the strength of friendships was impacted by interaction on Facebook. At the time, Burke was working at Facebook so she had complete access to one of the most massive and detailed records of human behavior ever created. But, even so, Burke and Kraut had to use surveys in order to answer their research question. Their outcome of interest—the subjective feeling of closeness between the respondent and her friend—is an internal state that only exists inside the respondent’s head. Further, in addition to using a survey to collect their outcome of interest, Burke and Kraut also had to use a survey to learn about potentially confounding factors. In particular, they wanted to separate the impact of communicating on Facebook from communication through other channels (e.g., email, phone, and face to face). Even though interactions through email and phone are automatically recorded, these traces were not available to Burke and Kraut so they had to collect them with a survey. Combining their survey data about friendship strength and non-Facebook interaction with the Facebook log data, Burke and Kraut concluded that communication via Facebook did in fact lead to increased feelings of closeness.
As the work of Burke and Kraut illustrates, big data sources will not eliminate the need to ask people questions. In fact, I would draw the opposite lesson from this study: big data sources can actually increase the value of asking questions, as I will show throughout this chapter. Therefore, the best way to think about the relationship between asking and observing is that they are complements rather than substitutes; they are like peanut butter and jelly. When there is more peanut butter, people want more jelly; when there is more big data, I think people will want more surveys.
