Researchers can chop up big surveys and sprinkle them into people’s lives.
Ecological momentary assessment (EMA) involves taking traditional surveys, chopping them up into pieces, and sprinkling them into the lives of participants. Thus, survey questions can be asked at an appropriate time and place, rather than in a long interview weeks after the events have occurred.
EMA is characterized by four features: (1) collection of data in real-world environments; (2) assessments that focus on individuals’ current or very recent states or behaviors; (3) assessments that may be event-based, time-based, or randomly prompted (depending on the research question); and (4) completion of multiple assessments over time (Stone and Shiffman 1994). EMA is an approach to asking that is greatly facilitated by smartphones with which people interact frequently throughout the day. Further, because smartphones are packed with sensors—such as GPS and accelerometers—it is increasingly possible to trigger measurements based on activity. For example, a smartphone could be programmed to trigger a survey question if a respondent goes into a particular neighborhood.
The promise of EMA is nicely illustrated by the dissertation research of Naomi Sugie. Since the 1970s, the United States has dramatically increased the number of people that it imprisons. As of 2005, about 500 in every 100,000 Americans were in prison, a rate of incarceration higher than anywhere else in the world (Wakefield and Uggen 2010). The surge in the number of people entering prison has also produced a surge in the number leaving prison; about 700,000 people leave prison each year (Wakefield and Uggen 2010). These people face severe challenges upon leaving prison, and unfortunately many end up back there. In order to understand and reduce recidivism, social scientists and policy makers need to understand the experience of people as they re-enter society. However, these data are hard to collect with standard survey methods because ex-offenders tend to be difficult to study and their lives are extremely unstable. Measurement approaches that deploy surveys every few months miss enormous amounts of the dynamics in their lives (Sugie 2016).
In order to study the re-entry process with much greater precision, Sugie took a standard probability sample of 131 people from the complete list of individuals leaving prison in Newark, New Jersey. She provided each participant with a smartphone, which became a rich data collection platform, both for recording behavior and for asking questions. Sugie used the phones to administer two kinds of surveys. First, she sent an “experience sampling survey” at a randomly selected time between 9 a.m. and 6 p.m. asking participants about their current activities and feelings. Second, at 7 p.m., she sent a “daily survey” asking about all the activities of that day. Further, in addition to these survey questions, the phones recorded their geographic location at regular intervals and kept encrypted records of call and text meta-data. Using this approach—which combines asking and observing—Sugie was able to create a detailed, high-frequency set of measurements about the lives of these people as they re-entered society.
Researchers believe that finding stable, high-quality employment helps people successfully transition back into society. However, Sugie found that, on average, her participants’ work experiences were informal, temporary, and sporadic. This description of the average pattern, however, masks important heterogeneity. In particular, Sugie found four distinct patterns within her participant pool: “early exit” (those who start searching for work but then drop out of the labor market), “persistent search” (those who spend much of the period searching for work), “recurring work” (those who spend much of the period working), and “low response” (those who do not respond to the surveys regularly). The “early exit” group—those who start searching for work but then don’t find it and stop searching—is particularly important because this group is probably the least likely to have a successful re-entry.
One might imagine that searching for a job after being in prison is a difficult process, which could lead to depression and then withdrawal from the labor market. Therefore, Sugie used her surveys to collect data about the emotional state of participants—an internal state that is not easily estimated from behavioral data. Surprisingly, she found that the “early exit” group did not report higher levels of stress or unhappiness. Rather, it was the opposite: those who continued to search for work reported more feelings of emotional distress. All of this fine-grained, longitudinal detail about the behavior and emotional state of the ex-offenders is important for understanding the barriers they face and easing their transition back into society. Further, all of this fine-grained detail would have been missed in a standard survey.
Sugie’s data collection with a vulnerable population, particularly the passive data collection, might raise some ethical concerns. But Sugie anticipated these concerns and addressed them in her design (Sugie 2014, 2016). Her procedures were reviewed by a third party—her university’s Institutional Review Board—and complied with all existing rules. Further, consistent with the principles-based approach that I advocate in chapter 6, Sugie’s approach went far beyond what was required by existing regulations. For example, she received meaningful informed consent from each participant, she enabled participants to temporarily turn off the geographic tracking, and she went to great lengths to protect the data that she was collecting. In addition to using appropriate encryption and data storage, she also obtained a Certificate of Confidentiality from the federal government, which meant that she could not be forced to turn over her data to the police (Beskow, Dame, and Costello 2008). I think that because of her thoughtful approach, Sugie’s project provides a valuable model to other researchers. In particular, she did not stumble blindly into an ethical morass, nor did she avoid important research because it was ethically complex. Rather, she thought carefully, sought appropriate advice, respected her participants, and took steps to improve the risk-benefit profile of her study.
I think there are three general lessons from Sugie’s work. First, new approaches to asking are completely compatible with traditional methods of sampling; recall that Sugie took a standard probability sample from a well-defined frame population. Second, high-frequency, longitudinal measurements can be particularly valuable for studying social experiences that are irregular and dynamic. Third, when survey data collection is combined with big data sources—something that I think will become increasingly common, as I’ll argue later in this chapter—additional ethical issues can arise. I’ll treat research ethics in more detail in chapter 6, but Sugie’s work shows that these issues are addressable by conscientious and thoughtful researchers.
