We are always going to need to ask people questions.
Given that more and more of our behavior is capture in government and business administrative data, some people might think that asking questions is a thing of the past. But, its not that simple. It is certainly true that researchers will ask less about behavior in the future, but, as I discussed in Chapter 2, there are real problems with the accuracy, completeness, and accessibility of many big data sources. Therefore, I expect that the problems with these data sources mean that researchers will continue to ask respondents about their behavior for the foreseeable future.
In addition to these practical reasons, there is also a more fundamental reason to ask: behavior data—even perfect behavior data—is limited. Some of the most important social outcomes and predictors are internal states, such as emotions, knowledge, expectations, and opinions. Internal states exist only inside people’s heads, and sometimes the best way to learn about internal states is to ask.
The practical and fundamental limitations of big data sources, and how they can be overcome with surveys, are illustrated by Moira Burke and Robert Kraut’s (2014) research on how the strength of friendships was impacted by interaction on Facebook. At the time, Burke was working at Facebook so she had complete access to one of the most massive and detailed records of human behavior ever created. But, even still, Burke and Kraut had to use surveys in order to answer their research question. Their outcome of interest—how close the respondent feels to specific friends—is an internal state that only exists inside the respondent’s head. Further, in addition to using a survey to collect their outcome of interest, Burke and Kraut also had to use a survey to learn about other potentially confounding factors. In particular, they wanted to separate the impact of communicating on Facebook from communication through other channels (e.g., email, phone, face-to-face). Even though interactions through email and phone are automatically recorded, these traces were not available to Burke and Kraut. Combining their survey data about friendship strength and non-Facebook interaction with the Facebook log data, Burke and Kraut concluded that communication via Facebook did in fact lead to increased feelings of closeness.
As the work of Burke and Kraut illustrates, big data sources will not eliminate the need to ask people questions. In fact, I would draw the opposite lesson from this study: big data actually increases the value of asking questions, as I will show throughout this chapter. Therefore, the best way to think about the relationship between asking and observing is that they are complements rather than substitutes; they are like peanut butter and jelly. When there is more peanut butter, people want more jelly; when there is more big data, people want more surveys.
