Le mea e faitau i le isi

• Faatomuaga (vaega 4.1)

Fesili e uiga i le mafuaʻaga i suʻesuʻega faʻaagafesootai e tele lava ina faigata ma faigata. Mo se faʻasalaga faʻavae i mafuaʻaga e faʻavae i luga o graphic causal, tagaʻii le Pearl (2009) , ma mo se faʻavae faavae e faʻavae i taunuʻuga talafeagai, silasila Imbens and Rubin (2015) . Mo se faʻatusatusaga i le va o nei auala e lua, tagaʻii le Morgan and Winship (2014) . Mo se auala aloaia i le faʻamalamalamaina o se tagata faʻamaonia, vaai VanderWeele and Shpitser (2013) .

I totonu o lenei mataupu, ua ou fatuina mea na foliga mai o se laina malamalama i le va o lo tatou mafai ona faia ni fua faʻatatau mai faʻataʻitaʻiga ma le leai o faʻamatalaga. Ae ui i lea, ou te manatu o le mea moni, o le eseʻesega ua sili atu ona le mautonu. Mo se faʻataʻitaʻiga, e talia e tagata uma le ulaula e mafua ai le kanesa, e ui lava e leai se faʻataʻitaʻiga faʻapitoa e faʻamalosia ai tagata e ulaula ua uma ona faia. Mo le sili ona lelei togafitiga o tusi-length i luga o le faia o fuainumera o mafuaʻaga mai i le leai o ni faʻataʻitaʻiga, see Rosenbaum (2002) , ( ??? ) , Shadish, Cook, and Campbell (2001) , ma Dunning (2012) .

O mataupu 1 ma le 2 o le Freedman, Pisani, and Purves (2007) ofoina atu se folasaga manino i eseesega i le va o suʻega, suʻega faʻapitoa, ma suʻesuʻega faʻasolosolo faʻatonutonu.

Manzi (2012) loʻo tuʻuina mai ai se faʻamatalaga mataʻina ma mafai ona mafai ona tuʻuina atu i lalo o le faʻafilosofia ma le fuainumera o faʻataʻitaʻiga o suʻesuʻega faʻapitoa. E maua ai foi faʻamatalaga matagofie o le lalolagi i le mana o faʻataʻitaʻiga i pisinisi. Issenberg (2012) maua ai se folasaga mataʻina i le faʻaaogaina o faʻataʻitaʻiga i faiga faʻapolokiki.

• O a ni faʻataʻitaʻiga? (vaega 4.2)

Box, Hunter, and Hunter (2005) , @ casella_statistical_2008, ma Athey and Imbens (2016b) tuʻuina atu ni faʻamatalaga lelei i vaega faʻafuainumera o le faʻataʻitaʻiga ma le auʻiliʻiliga. Gata i lea, o loo i ai togafitiga e sili ona lelei o le faaaogaina o suesuega i le tele o fanua eseese: tamaoaiga (Bardsley et al. 2009) , Sociology (Willer and Walker 2007; Jackson and Cox 2013) , mataupu tau le mafaufau (Aronson et al. 1989) , Saienisi faaupufai (Morton and Williams 2010) , ma faiga faʻavae lautele (Glennerster and Takavarasha 2013) .

O le taua o le auai o tagata auai (faataitaiga, sampling) e masani ona le talisapaiaina i suʻesuʻega suʻesuʻe. Ae peitai, afai o le aafiaga o togafitiga e eseese i le faitau aofaʻi, o le sampling e taua tele. Longford (1999) loʻo faʻamalamalama manino lenei taimi pe a fautua atu mo tagata suʻesuʻe e mafaufau i suʻega e fai ma suʻesuʻega o le faitau aofaʻi faʻataʻitaʻiga.

• E lua vaega o le suʻega: vaʻalele-faʻafanua ma analog-numera (vaega 4.3)

Ua ou fautua atu o loʻo i ai se faʻagasologa i le va o suʻesuʻega ma suʻesuʻega faʻapitoa, ma isi tagata suʻesuʻe ua tuʻuina atu ni faʻamaumauga sili atu auʻiliʻili, aemaise lava vaega e vavae ese ai ituaiga o suʻesuʻega faʻapitoa (Harrison and List 2004; Charness, Gneezy, and Kuhn 2013) .

O le tele o pepa ua faʻatusatusaina le galuega a le fale ma faʻataʻitaʻiga faʻapitoa i le tala otooto (Falk and Heckman 2009; Cialdini 2009) ma iʻuga o taunuʻuga faʻapitoa i suʻesuʻega faʻapolokiki (Coppock and Green 2015) , tamaoaiga (Levitt and List 2007a, 2007b; Camerer 2011; Al-Ubaydli and List 2013) , ma le mafaufau loloto (Mitchell 2012) . Jerit, Barabas, and Clifford (2013) ofo atu se mamanu suʻesuʻega lelei mo le faʻatusatusaina o taunuuga mai suʻesuʻega ma faʻataʻitaʻiga. Parigi, Santana, and Cook (2017) loʻo faʻamatalaina le auala e mafai ai e faʻataʻitaʻiga faʻapitonuʻu ona tuʻufaʻatasia nisi o uiga o suʻesuʻega ma faʻataʻitaʻiga.

O popolega e uiga i tagata o loʻo auai e suia a latou amio ona latou te iloa o loʻo vaʻavaʻavaʻavaʻaia, o nisi taimi e taʻua o aafiaga o manaoga , ma ua suʻesuʻeina i le mafaufau (Orne 1962) ma le tamaoaiga (Zizzo 2010) . E ui o le tele o fesoʻotaʻiga faʻatasi ma suʻesuʻega a le fale, o nei lava mataupu e mafai ona mafua ai faafitauli mo suʻesuʻega faʻapitoa. O le mea moni, o aʻafiaga o aʻafiaga e taʻua i nisi taimi o aafiaga o Hawthorne , o se vaitaimi e maua mai ai faʻamatalaga faʻalauiloa faʻalauiloa na amata i le 1924 i le Hawthorne Works o le Western Electric Company (Adair 1984; Levitt and List 2011) . O aʻafiaga uma o aʻafiaga ma aʻafiaga a Hawthorne e fesoʻotaʻi vavalalata ma le manatu o fua o le gaioiga o loʻo talanoaina i le mataupu 2 (silasila foi Webb et al. (1966) ).

O suʻesuʻega faʻapitoa o loʻo i ai se talafaasolopito umi i le tamaoaiga (Levitt and List 2009) , suʻesuʻega faʻapolokiki (Green and Gerber 2003; Druckman et al. 2006; Druckman and Lupia 2012) , mafaufau faʻapitoa (Shadish 2002) , ma faiga faʻavae lautele (Shadish and Cook 2009) . O le tasi itu o le sosaiete faʻasaienisi lea na vave ona taʻutaʻua ai faʻataʻitaʻiga faʻapitoa i le atinaʻe faavaomalo Mo se iloiloga lelei o lena galuega i totonu o le tamaoaiga, vaai i le Banerjee and Duflo (2009) , ma mo se iloiloga taua, ia Deaton (2010) . Mo se toe iloiloga o lenei galuega i suʻesuʻega faʻapolokiki, silasila i Humphreys and Weinstein (2009) . O le mea mulimuli, o le faʻaaogaina o luitau faʻapitoa na maua mai suʻesuʻega faʻapitoa na suʻesuʻeina i le tulaga o suʻesuʻega faaupufai (Humphreys 2015; Desposato 2016b) ma le atinaʻeina o tamaoaiga (Baele 2013) .

I lenei vaega, sa ou fautua atu e mafai ona faʻaaogaina faʻamatalaga muamua togafitiga e faʻaleleia ai le saʻo o le fua faatatau o togafitiga, ae o loʻo i ai le finauga e uiga i lenei auala; tagai Freedman (2008) , W. Lin (2013) , Berk et al. (2013) , ma Bloniarz et al. (2016) mo nisi faʻamatalaga.

Ma le mea mulimuli, e lua isi ituaiga o faʻataʻitaʻiga na faia e saienitisi faʻasaienisi e le fetaui lelei i le tulaga o le galuega: suʻesuʻega suʻesuʻega ma suʻesuʻega faʻaagafesootai. O suʻega o suʻesuʻega o suʻega e faʻaaoga ai le tele o suʻesuʻega o loʻo i ai nei ma faʻatusatusa tali i isi suiga o fesili lava ia (o nisi o suʻega suʻesuʻega o loʻo tuʻuina atu i le Vaega 3); mo nisi faʻamatalaga suʻesuʻega matamata i le Mutz (2011) . O suʻesuʻega faʻapitoa o suʻega lea o togafitiga o nisi faiga faʻavae lautele lea e mafai ona faʻaaoga e se malo. O suʻesuʻega faʻapitoa e fesoʻotaʻi vavalalata ma le iloiloga o polokalame. Mo nisi faʻamatalaga faʻavae, tagaʻi Heckman and Smith (1995) , Orr (1998) , ma le @ glennerster_running_2013.

• Agai i tua atu o faʻataʻitaʻiga faigofie (vaega 4.4)

Ua ou filifilia e taulaʻi atu i mataupu e tolu: faʻamaoni, faʻalauteleina o aʻafiaga togafiti, ma metotia. O nei manatu e eseese igoa i vaega eseese. Mo se faʻataʻitaʻiga, e masani ona faʻagasolo i luga o suʻesuʻega faʻapitoa e le au suʻesuʻe o le mafaufau e ala i le taulai atu i tagata faufautua ma tagata faʻatau (Baron and Kenny 1986) . O le manatu o le aufaasālalau e maua i le mea ou te taʻua o meafaigaluega, ma o le manatu o tagata faʻapitoa o loʻo maua e le mea ou te taʻua o le faʻamaonia mai fafo (faʻataʻitaʻiga, o le a ese le iʻuga o le suʻega pe a fai e taʻalo i tulaga eseese) ma le eseesega o togafitiga ( faʻataʻitaʻiga, o aʻafiaga e sili atu mo nisi tagata nai lo isi).

O le faataitaiga a Schultz et al. (2007) faʻaalia pe faʻapefea ona faʻaaogaina aʻoaʻoga faʻale-agafesoʻotaʻi e mamanuina ai faʻamatalaga lelei. Mo se finauga sili atu lautele e uiga i le matafaioi o le mafaufau i le mamanuina o fegalegaleaiga lelei, vaai Walton (2014) .

• Tulaga aloaia (vaega 4.4.1)

O faʻamatalaga i totonu ma fafo atu o le faʻamuamua na faʻamuamua e Campbell (1957) . Vaai Shadish, Cook, and Campbell (2001) mo se auiliiliga auiliili ma faʻamalamalama ma le faʻaeteete le faʻataunuʻuina o fuainumera fuainumera, aloaia o le lotoifale, faʻamaonia le aoga, ma le faʻamaonia mai fafo.

Mo se aotelega o mataupu e fesoʻotai i le fuainumera fuainumera o faʻataʻitaʻiga, tagaʻi Gerber and Green (2012) (mai se vaaiga i le vafealoaloai) ma Imbens and Rubin (2015) (mai se fuainumera fuainumera). O nisi o mataupu o le faʻamaoniaina o fuainumera fuainumera e tulaʻi mai faapitoa i faʻasalalauga i luga o le initaneti e aofia ai mataupu e pei o auala faʻaleleia lelei mo le fatuina o tafaoga (Bakshy and Eckles 2013) faʻamaumauga faalagolago (Bakshy and Eckles 2013) .

E mafai ona faigata le faʻamaoniaina i totonu i le lotoifale e faʻamautinoa i faʻalavelave faigata faʻapitoa. Vaʻai, mo se faʻataʻitaʻiga, Gerber and Green (2000) , Imai (2005) , ma Gerber and Green (2005) mo felafolafoaiga e uiga i le faʻatinoina o se faʻalavelave faʻalavelave faʻapitoa e uiga i le palota. Kohavi et al. (2012) ma Kohavi et al. (2013) tuʻuina atu se faʻatagaga i luitau o le vavalalata taimi i faʻasalalauga faʻapitoa i luga o le initaneti.

O se tasi lamatiaga ogaoga i totonu o le lotoifale o le avanoa lea e le manuia ai. O se tasi o auala mafai e iloa ai faafitauli ma le faʻatauvalea o le faʻatusatusaina lea o kulupu togafitiga ma taʻitaʻia i luga o uiga vaaia. O lenei ituaiga o faʻatusatusaga e taua o le siakiina paleni . Vaai Hansen and Bowers (2008) mo se fuainumera faamaumauina e paleni siaki ma Mutz and Pemantle (2015) mo popolega e uiga i siaki paleni. Mo se faʻataʻitaʻiga, i le faʻaaogaina o siaki paleni, Allcott (2011) maua ai ni faʻamaoniga e le o faʻatinoina le faʻataunuʻuina i totonu o le tolu o iloiloga a le Autower (vaai laulau 2, nofoaga 2, 6, ma le 8). Mo isi auala, tagaʻii le mataupu 21 o Imbens and Rubin (2015) .

O isi popolega tetele e fesoʻotaʻi ma le faʻamaoniga o le lotoifale, o: (1) le tasi itu, o le mea e le o tagata uma i le vaega o togafitiga na maua moni togafitiga, (2) le maliliega e lua, pe a le maua e tagata uma ile togafitiga togafitiga ma nisi tagata o le vaega o loʻo pulea le togafitiga, (3) faʻatusa, pe afai e le fuaina taunuuga mo nisi tagata auai, ma le (4) faʻalavelave, pe a fai o le togafitiga e solo atu mai tagata i tulaga togafitiga i tagata i le tulaga e pulea ai. Vaai mataupu 5, 6, 7, ma le 8 o Gerber and Green (2012) mo nisi faʻamatalaga i nei mataupu taʻitasi.

Mo nisi faʻamatalaga i luga o le aoga, silasila Westen and Rosenthal (2003) , ma mo nisi faʻamatalaga i luga o le fausia o faʻamaumauga tele, Lazer (2015) ma le mataupu 2 o lenei tusi.

O se tasi o itu o le faʻamaonia mai fafo o le nofoaga lea e faʻataʻitaʻiina ai se mea e fai. Allcott (2015) loʻo tuʻuina atu ma le faʻaeteete togafitiga faʻapitoa ma togafitiga faʻapitoa o le vaʻaia le filifilia o nofoaga. O lenei lomiga e talanoaina foi e Deaton (2010) . O le isi itu o le faʻamaonia mai fafo o le faʻaaogaina lea o galuega faatino tutusa o le gaioiga tutusa o le ai ai foi aafiaga faapena. I lenei tulaga, o se faatusatusaga i le va o Schultz et al. (2007) ma Allcott (2011) loʻo faʻaalia ai o le atinaʻeina o le Atower na i ai sona aafiaga laʻititi laʻititi nai lo uluaʻi suʻesuʻega a Schultz ma ana uo (1.7% e tusa ma le 5%). Allcott (2011) o suʻesuʻega na mulimuli mai na i ai se aafiaga itiiti ona o auala e ese ai togafitiga: o le emoticon lima tusilima e avea o se vaega o se suʻesuʻega na lagolagoina e se iunivesite, faʻatusatusa i se emoticon lolomi e avea o se vaega o se fatuga tele lipoti mai se kamupani malosiaga.

• Eseesega o togafitiga (vaega 4.4.2)

Mo se vaaiga sili ona lelei o le eseesega o togafitiga i togafitiga faʻapitoa, tagaʻii le mataupu 12 o Gerber and Green (2012) . Mo faʻamatalaga i le eseesega o togafitiga i togafitiga faafomaʻi, silasila Kent and Hayward (2007) , Longford (1999) , ma Kravitz, Duan, and Braslow (2004) . O iloiloga o le faʻasesega o togafitiga faʻapitoa e taulai atu i eseesega e faavae i uiga o togafitiga muamua. Afai e te fiafia i le faʻalauteleina e faʻavae i luga ole taunuʻuga o togafitiga, ona sili atu lea ona manaʻomia ni auala faʻapitoa, e pei o le (Frangakis and Rubin 2002) autu (Frangakis and Rubin 2002) ; tagaʻi i le Page et al. (2015) mo se toe iloiloga.

O le tele o tagata suʻesuʻe e faʻatusatusa le faʻalauteleina o togafitiga faʻapitoa e faʻaaogaina ai le faasologa o laina, ae o auala fou e faʻalagolago i le aʻoaʻoina o masini; vaai, mo se faataitaiga, Green and Kern (2012) , Imai and Ratkovic (2013) , Taddy et al. (2016) , ma Athey and Imbens (2016a) .

O loʻo i ai ni masalosaloga e uiga i sailiiliga o le faʻaaloalo o aʻafiaga ona o le tele o faafitauli o faʻatusatusaga ma le "fagota." E tele ituaiga o fuainumera faʻataʻitaʻi e mafai ona fesoasoani e foia ai popolega e uiga i faʻatusatusaga e tele (Fink, McConnell, and Vollmer 2014; List, Shaikh, and Xu 2016) . O se tasi o auala e faatatau i le "fagota" o le muaʻi resitalaina, lea ua avea ma tagata e sili ona masani i le mafaufau (Nosek and Lakens 2014) , saienisi faaupufai (Humphreys, Sierra, and Windt 2013; Monogan 2013; Anderson 2013; Gelman 2013; Laitin 2013) , ma le tamaoaiga (Olken 2015) .

I le suʻesuʻega a Costa and Kahn (2013) naʻo le afa o aiga i totonu o le suʻega e mafai ona fesoʻotaʻi atu i faʻamatalaga faʻamalamalamaga. O le au faitau e fiafia i nei auiliiliga e tatau ona faatatau i le pepa muamua.

• Faʻatinoga (vaega 4.4.3)

O mea e tutupu e matua taua tele, ae e foliga mai e matua faigata lava ona suesue. Suesuega e uiga i faiga e vavalalata vavalalata ma suʻesuʻega o tagata faufautua i mafaufauga faʻapitoa (ae vaʻai VanderWeele (2009) mo se faʻatusatusaga saʻo i le va o manatu e lua). O fuainumera faamaumauina i le sailia o auala, e pei o le auala na atinaeina i Baron and Kenny (1986) , e masani lava. O le mea e faanoanoa ai, e foliga mai o nei taualumaga e faalagolago i luga o ni manatu malolosi (Bullock, Green, and Ha 2010) ma mafatia ai pe a tele ni auala, e pei ona faamoemoeina i le tele o tulaga (Imai and Yamamoto 2013; VanderWeele and Vansteelandt 2014) . Imai et al. (2011) ma Imai and Yamamoto (2013) ofo atu nisi metotia faʻafuainumera. E le gata i lea, o VanderWeele (2015) ofoina atu se tusi-lōmi ma togafitiga e tele, e aofia ai se auala atoatoa e suʻesuʻeina ai le mafaufau.

O se isi auala e taulai atu i suʻega e taumafai ai e faʻaaogaina le masini (eg, tuʻuina atu o vailaʻau vitamin C). Ae paga lea, i le tele o tulaga faasaienisi lautele, e masani ona tele auala ma e faigata ona mamanuina togafitiga e suia ai le tasi e aunoa ma le suia o isi. O nisi o auala e faʻaaogaina ai suiga faʻapitoa o loʻo faamatalaina e Imai, Tingley, and Yamamoto (2013) , Ludwig, Kling, and Mullainathan (2011) , ma Pirlott and MacKinnon (2016) .

O tagata suʻesuʻe e suʻe faʻamalositino atoatoa o le a manaʻomia le popole e uiga i le tele o suʻega faʻapitoa; Vaʻai Fink, McConnell, and Vollmer (2014) ma List, Shaikh, and Xu (2016) mo nisi faʻamatalaga.

Mulimuli ane, o masini e iai foi se talafaasolopito umi i le filosofia o le faasaienisi e pei ona faamatalaina e Hedström and Ylikoski (2010) .

• Faaaogaina o siosiomaga oi ai (vaega 4.5.1)

Mo nisi faʻamatalaga e uiga i le faʻaogaina o suʻesuʻega faʻataʻitaʻitusi ma suʻega suʻega e faʻatatau ai le faʻaituau, tagaʻi Pager (2007) .

• Fausia lau oe lava faʻataʻitaʻiga (vaega 4.5.2)

O le auala taatele e faʻaaogaina ai tagata auai i faʻataʻitaʻiga e te fausiaina, o le Amazon Mechanical Turk (MTurk). Talu ai o le MTurk e faʻaaogaina vaega o faʻataʻitaʻiga masani a le fale-totogi tagata e faʻatumu galuega latou te le faia mo le saoloto-e tele tagata suʻesuʻe ua amata ona faʻaaogaina le au Turkers (tagata faigaluega i MTurk) e avea ma tagata suʻesuʻe, e mafua ai ona maua vave le aoina o faʻamaumauga nai lo mea e mafai ona ausia i suʻesuʻega faʻasaienisi masani (Paolacci, Chandler, and Ipeirotis 2010; Horton, Rand, and Zeckhauser 2011; Mason and Suri 2012; Rand 2012; Berinsky, Huber, and Lenz 2012) .

I se tulaga lautele, o le tele o tulaga lelei o le faʻaaogaina o tagata auai na maua mai le MTurk e faʻasino. E ui e mafai e suʻega a le fale ona faia ni vaiaso e tamoe ai ma faʻataʻitaʻiga faʻapitoa e mafai ona ave masina i le seti, o suʻega faʻatasi ma tagata o loʻo auai mai le MTurk e mafai ona tafe i aso. Mo se faʻataʻitaʻiga, na mafai e Berinsky, Huber, and Lenz (2012) ona suʻe 400 mataupu i le aso e tasi e auai i se suʻega e 8 minute. E le gata i lea, o nei tagata auai e mafai ona faʻaaogaina mo le toeitiiti lava o soʻo se faʻamoemoega (e aofia ai suʻesuʻega ma le galulue soosootauau, e pei ona talanoaina i mataupu 3 ma le 5). O lenei faigofie o le faʻafaigaluegaina o lona uiga e mafai e tagata suʻesuʻe ona faʻasolosolo faʻasalalauga o suʻega faʻapitoa i le vave vave.

A o leʻi faʻailoa mai tagata auai mai le MTurk mo au lava suʻega, e fa mea taua e te manaʻomia ona e iloa. Muamua, o le toʻatele o tagata suʻesuʻe e iai ni talitonuga le mautinoa o faʻataʻitaʻiga e aofia ai Turkers. Ona o lenei masalosalo e le o se mea patino, e faigata tele ona feteenai ma se faʻamaoniga. Ae ui i lea, ina ua mavae le tele o tausaga o aʻoga e faʻaaoga ai le au Turkers, ua mafai nei ona tatou manatu o lenei talitonuga masalosalo e le tauamiotonuina. E tele suʻesuʻega na faʻatusatusa ai le demographics o Turkers faatasi ma isi tagata ma le tele o suʻesuʻega e faʻatusatusa ai taunuuga o suʻega ma le Turkers faatasi ai ma i latou mai isi tagata. I le tele o nei galuega, ou te manatu o le auala aupito sili ona lelei mo oe e te mafaufau ai, o le Turkers o se faʻataʻitaʻiga talafeagai talafeagai, e pei lava o tamaiti aʻoga laiti ae eseese (Berinsky, Huber, and Lenz 2012) . O le mea lea, e pei lava o tamaiti aoga o se fuainumera talafeagai mo nisi, ae le o mea uma, suʻesuʻega, o Turkers o se faitau aofaʻi talafeagai mo nisi, ae le o mea uma, suʻesuʻega. Afai o le a oulua galulue ma le au Turkers, o lona uiga e tatau ona faitau le tele o nei faʻatusatusaga faʻatusatusaga ma malamalama io latou foliga.

Lona lua, ua atiaʻe e le au suʻesuʻega ni faiga sili ona lelei mo le faʻaleleia o le aoga o suʻesuʻega a MTurk, ma e tatau ona e aʻoaʻo ma mulimuli i nei faiga sili ona lelei (Horton, Rand, and Zeckhauser 2011; Mason and Suri 2012) . Mo se faʻataʻitaʻiga, e faʻamalosia tagata suesue e faʻaaogaina le 'au Turkers e faʻaaoga vailaau e aveese ai tagata e le o (Berinsky, Margolis, and Sances 2014, 2016) (ae faʻaali foi DJ Hauser and Schwarz (2015b) ma DJ Hauser and Schwarz (2015a) ). Afai e te le aveesea tagata e le o popole i le auai, ona mafai loa lea ona fufulu ese le malosi o togafitiga e ala i le pisapisao latou te faailoaina, ma i le faatinoga o le numera o tagata e le o popole i le auai e mafai ona tele. I le faʻataʻitaʻiga a Huber ma ana paʻaga (2012) , e tusa ma le 30% o tagata auai na le mafai ona suʻeina le au suʻesuʻega. O isi faʻafitauli e masani ona tulaʻi mai pe a faʻaaogaina le Turkers e le o ni nai tagata auai (Chandler et al. 2015) ma le faʻataunuʻu (Zhou and Fishbach 2016) .

Lona tolu, e fesoʻotaʻi ma isi ituaiga o faʻataʻitaʻiga numera, o suʻega a le MTurk e le mafai ona fua; Stewart et al. (2015) faʻatusatusa i soo se taimi lava e na o le 7,000 tagata i luga o le MTurk.

I le mea mulimuli, e tatau ona e iloa o le MTurk o se faʻalapotopotoga ma ona lava tulafono ma tulafono (Mason and Suri 2012) . I le auala lava e tasi o le a e taumafai e saili ai e uiga i le aganuu a se atunuu o le ae faia ai au suʻega, e tatau ona e taumafai e saili atili e uiga i aganuu ma masani a Turkers (Salehi et al. 2015) . Ma e tatau ona e iloa o le a talanoa tagata Turkers e uiga i lau suʻega pe afai e te faia se mea e le talafeagai pe leai foi (Gray et al. 2016) .

O le MTurk o se ala sili ona faigofie e faʻafaigaluegaina ai tagata auai i au suʻega, tusa lava pe tutusa a latou galuega, e pei o Huber, Hill, and Lenz (2012) , poʻo nisi faʻalapotopotoga pei o Mason and Watts (2009) , Goldstein, McAfee, and Suri (2013) , Goldstein et al. (2014) , Horton and Zeckhauser (2016) , ma Mao et al. (2016) .

• Fausia a oe lava oloa (vaega 4.5.3)

Afai o loʻo e mafaufau i le taumafai e fatu lau lava oloa, ou te fautuaina oe e faitau le fautuaga na ofoina mai e le vaega o MovieLens i Harper and Konstan (2015) . O se faʻamatalaga autu mai o latou poto masani, o le manuia o galuega faatino taʻitasi o loʻo i ai le tele o le tele o toilalo. Mo se faʻataʻitaʻiga, na faʻalauiloa e le vaega o le MovieLens isi oloa, e pei o GopherAnswers, o faʻauʻu atoa (Harper and Konstan 2015) . O se isi faʻataʻitaʻiga o se tagata suʻesuʻe ua toilalo aʻo taumafai e fau se oloa o le taumafaiga a Edward Castronova e fausia se taaloga i le initaneti e igoa ia Arden. E ui i le $ 250,000 i le faʻatupeina, o le poloketi o se flop (Baker 2008) . O mea e pei o GopherAnswers ma Arden e sili atu le taatele nai lo galuega pei o MovieLens.

• Paʻaga ma le malosi (vaega 4.5.4)

Ua ou faalogo i le manatu a le Pasteur's Quadrant e masani ona talanoa soo i kamupani faatekinolosi, ma fesoasoani i le faatulagaina o taumafaiga sailiili i Google (Spector, Norvig, and Petrov 2012) .

O suʻesuʻega a Bond ma a latou paʻaga (2012) loʻo taumafai foi e iloa le aʻafiaga o nei togafitiga i luga o uo a i latou oe na mauaina. Talu ai ona o le mamanu o le suʻega, o nei vailaʻau e faigata ona iloa mama; e tatau i tagata faitau faitau fiafia ona vaʻai ia Bond et al. (2012) mo se talanoaga sili atu ona maeʻa. O Jones ma ana paaga (2017) latou faia foi se suesuega talitutusa i le 2012. O nei faʻataʻitaʻiga o se vaega o se aga masani o suʻesuʻega i suʻesuʻega faʻapolokiki i taumafaiga e faʻamalosia le palota (Green and Gerber 2015) . O nei suʻega-le-palota o suʻega e taatele, i se vaega ona o latou i Pasteur's Quadrant. O lona uiga, e tele tagata ua faaosofia e faateleina le palota ma o palota e mafai ona avea ma se mea manaia e suʻeina ai le tele o aʻoaʻoga lautele e uiga i suiga o amioga ma aʻafiaga lautele.

Mo fautuaga e uiga i le faʻataʻitaʻiga o faʻataʻitaʻiga ma faʻalapotopotoga faʻapitoa e pei o vaega faaupufai, faʻalapotopotoga, ma pisinisi, tagaʻi Loewen, Rubenson, and Wantchekon (2010) , JA List (2011) , ma Gueron (2002) . Mo manatu e uiga i le auala e mafai ai e faiga faʻapaʻaga ma faʻalapotopotoga ona aʻafia ai suʻesuʻega suʻesuʻega, tagaʻii le King et al. (2007) ma Green, Calfano, and Aronow (2014) . Faiga faʻapaʻaga e mafai ona taʻitaʻia ai fesili faʻale-aganuʻu, e pei ona talanoaina e Humphreys (2015) ma Nickerson and Hyde (2016) .

• Fautuaga faufautua (vaega 4.6)

Afai o loʻo e alu e faia se fuafuaga faʻatalanoaga ao leʻi faia lau suʻega, ou te fautuaina oe amata i le faitauina o faʻamatalaga o lipoti. O faʻasalalauga (Tuufaatasia Tuʻupuina o Lipoti o Tofotofoga) na faʻavaeina i vailaʻau (Schulz et al. 2010) ma faʻaleleia mo suʻesuʻega lautele (Mayo-Wilson et al. 2013) . O se taʻiala faʻapitoa na faʻapupulaina e faatonu o le Tusitala o Faʻataʻitaʻiga Faʻapolokiki Faʻapolokiki (Gerber et al. 2014) (silasila i le Mutz and Pemantle (2015) ma Gerber et al. (2015) ). I le taimi mulimuli, ua faʻatautaia taiala i le mafaufau (APA Working Group 2008) , ma silasila foi Simmons, Nelson, and Simonsohn (2011) .

Afai e te fatuina se fuafuaga faʻatalanoaina, e tatau ona e mafaufau muamua ae e te lesitalaina ona o le muaʻi resitaraina o le a faʻaleleia ai le talitonuga o loʻo i ai i isi i au iuga. E le gata i lea, afai o loʻo e galulue ma se paʻaga, o le a faʻatapulaaina le mafai gafatia o lau paaga e suia le suʻega pe a uma ona e vaʻaia iuga. O le muaʻi resitaraina ua amata ona faateleina le masani i le mafaufau (Nosek and Lakens 2014) , saienisi faaupufai (Humphreys, Sierra, and Windt 2013; Monogan 2013; Anderson 2013; Gelman 2013; Laitin 2013) , ma le tamaoaiga (Olken 2015) .

O fautuaga faufautua faʻapitoa mo suʻega i le initaneti o loʻo tuʻuina atu i Konstan and Chen (2007) ma Chen and Konstan (2015) .

O le mea na ou taʻua o le fuafuaga a le armada e taʻua i nisi taimi o suʻesuʻega faʻavaomalo ; tagai Wilson, Aronson, and Carlsmith (2010) .

• Faʻatuina fuainumera tau tau fesuiaiga (vaega 4.6.1)

Mo nisi faʻamatalaga i le MusicLab, tagaʻi Salganik, Dodds, and Watts (2006) , Salganik and Watts (2008) , Salganik and Watts (2009b) , Salganik and Watts (2009a) , ma Salganik (2007) . Mo nisi faʻamatalaga i luga o maketi-maua-maketi uma, tagaʻi Frank and Cook (1996) . Mo nisi faʻamatalaga e uiga i le le fiafia ma le tomai sili atu ona masani ai, vaai Mauboussin (2012) , Watts (2012) , ma Frank (2016) .

O loʻo i ai se isi auala e faʻaitiitia ai tupe totogi a tagata auai e tatau i tagata suʻesuʻe ona faʻaoga ma le faʻaeteete: faʻatau. I le tele o faʻataʻitaʻiga i luga o le initaneti, o le aufaʻatasi o loʻo auai i suʻesuʻega ma e leai se taui. O faʻataʻitaʻiga o lenei auala e aofia ai le Restivo ma le van de Rijt's (2012) faʻataʻitaʻiga i taui i Wikipedia ma Bond ma le faʻataʻitaʻiga (2012) i le faʻamalosiau o tagata e palota. O nei suʻega e leai se tau e fesuisuiai-ae, e leai se tau fesuiaiga i tagata suʻesuʻe . I ia ituaiga o faʻataʻitaʻiga, e tusa lava pe o le tau i tagata auai taitoatasi e matua laʻititi lava, o le aofaʻiga tau e mafai ona telē tele. O tagata suʻesuʻe e faʻateleina suʻesuʻega i luga o le initaneti e tele lava ina tauamiotonuina le taua o le togafitiga ole togafitiga faʻatusatusa i le fai mai o nei aʻafiaga laiti e mafai ona taua pe a faʻaaogaina i le tele o tagata. O le manatu lava lea e tasi e faatatau i tau e faʻamalosia e tagata suʻesuʻe i tagata auai. Afai o lau suʻega e mafua ai le tasi miliona tagata e faʻatafunaina le tasi minute, o le suʻega e le afaina tele i soʻo se tagata faapitoa, ae i le tuufaatasiga ua faʻatafunaina toetoe lava lua tausaga.

O le isi auala e faʻavae ai le tau o le tau e fesuiaʻi i tagata auai o le faʻaaogaina lea o se lotoa, o se auala na faʻaaogaina i suʻesuʻega suʻesuʻega (Halpern et al. 2011) . Mo nisi faʻamatalaga e uiga i le mamanuina o le faʻafiafiaga o tagata faʻaoga, tagaʻi Toomim et al. (2011) . Mo nisi faʻamatalaga e uiga i le faʻaaogaina o kosi e fatuina ai ni fuainumera taugata fesuiaiga taugata, vaai ( ??? ) .

• Sui, Faʻamaʻoti, ma Faʻaitiitia (vaega 4.6.2)

O le tolu o R e pei ona fuafuaina muamua e Russell and Burch (1959) e faapenei:

"Le toe suia o lona uiga o le suia mo iloa manu maualuga ola o mea insentient. Faaitiitiga o lona uiga o le faaitiitia o le fuainumera o manu e faaaoga e maua mai ai faamatalaga o le a tuuina atu le aofaiga ma le saʻo atoatoa. Faamamaina o lona uiga o so o se faaitiitia o le incidence po o le mamafa o inhumane taualumaga faatatau ia i latou manu lea e tatau ona faaaoga pea. "

O mea e tolu o le R lea ou te fautuaina e le faʻafefeina ai mataupu faʻavae o loʻo faʻamatalaina i le mataupu 6. Nai lo lena, o se faʻamatalaga sili atu ona faʻamatalaina o se tasi o na mataupu faavae-le faʻamanuiaina-aemaise lava i le faatulagaina o faʻataʻitaʻiga a tagata.

E tusa ai ma le muamua R ("suitulaga"), faʻatusatusaina o le faʻamalositino lagona faʻafelagona (Kramer, Guillory, and Hancock 2014) ma le suʻesuʻega faʻaletalagona faalelagona (Lorenzo Coviello et al. 2014) loʻo ofoina mai ai nisi o lesona lautele e uiga i fefaʻatauaiga e aofia ai i le sola ese mai faʻataʻitaʻiga i suʻega masani (ma isi auala e pei o le fetaui ma taumafai e faʻatatau i suʻega i faʻamatalaga e le faʻapitoa; vaʻai le mataupu 2). I le faʻaopopo atu i faʻamanuiaga faʻale-aganuʻu, o le suia mai le faʻataʻitaʻiga i suʻesuʻega e le o faʻataʻitaʻiga e mafai ai foi e tagata suʻesuʻe ona suʻesuʻe togafitiga latou te le mafai ona faʻaoga. O nei tulaga faʻale-aganuʻu ma faʻamaumauga e maua i se tau, peitaʻi. Faʻamatalaga faʻapitoa e le maua ai e le au suʻesuʻe suʻesuʻega le pulea lelei o mea e pei o le faʻafaigaluegaina o tagata auai, faʻatau, ma le natura o togafitiga. Mo se faʻataʻitaʻiga, o le tasi tapulaa o timuga o se togafitiga, o le faʻateleina o le tulaga lelei ma faʻaitiitia ai le faʻaleagaina. I le suʻesuʻega suʻesuʻega, e ui lava i lea, na mafai e Kramer ma ana paʻaga ona faʻafetaui lelei le tulaga lelei ma le le mautonu. O le auala faapitoa na faʻaaogaina e Lorenzo Coviello et al. (2014) na faʻamatala atili e L. Coviello, Fowler, and Franceschetti (2014) . Mo se faʻamuamua i fesuiaiga o mea faigaluega, o le faʻaogaina lea e Lorenzo Coviello et al. (2014) , silasila Angrist and Pischke (2009) (itiiti le aloaia) poʻo Angrist, Imbens, and Rubin (1996) (sili atu ona aloaia). Mo se iloiloga masalosalo o meafaigaluega eseese, vaʻai Deaton (2010) , ma mo le faʻaofiina o meafaigaluega faʻapitoa i meafaiva vaivai (o le timu o se mea vaivai), silasila Murray (2006) . I le tele o taimi, o Dunning (2012) , o Rosenbaum (2002) , ( ??? ) , ma Shadish, Cook, and Campbell (2001) te ofoina atu ni manatu lelei e uiga i le fuafuaina o faʻasologa o aʻafiaga e aunoa ma ni faʻataʻitaʻiga.

I tulaga o le lona lua R ("faʻaleleia"), o loʻo i ai faʻataienisi faasaienisi ma faʻasinomaga pe a mafaufau e suia le mamanu o le Contagion o lagona mai le taofia o pou i pou faʻaleleia. Mo se faʻataʻitaʻiga, atonu o le tulaga lea o le faʻaaogaina faʻapitoa o le News Feed ua faigofie tele ai ona faia se suʻega lea e poloka ai pou nai lo le tasi o loʻo faʻamalosia ai i latou (ia maitauina e mafai ona faʻaaoga se suesuega e aofia ai le taofia o pou. e pei o se laulau i le pito i luga o le News Feed system e aunoa ma se manaʻoga mo suiga o le faiga faavae). Ae peitai, saienitisi, o le talitonuga na laugaina e le suʻega e leʻi manino le fautuaina o le tasi mamanu i le isi. Ae paga lea, ou te le o maitauina le tele o suʻesuʻega muamua e uiga i le taua o le taofia ma le faʻamalosia o mea i le News Feed. E le gata i lea, ou te lei vaai tele i suʻesuʻega e uiga i le faʻamamaina o togafitiga e faʻaleagaina ai; O se tasi o mea e ese mai ia B. Jones and Feamster (2015) , e manatu i le mataupu o le fuaina o le tusiga o le Initaneti (o se autu ou te talanoaina i le mataupu 6 i le mafutaga ma le Encore suesue (Burnett and Feamster 2015; Narayanan and Zevenbergen 2015) ).

E tusa ai ma le tolu R ("faaitiiti"), o faʻamatalaga lelei i suʻesuʻega faʻale-aganuʻu masani o loʻo tuʻuina atu e Cohen (1988) (book) ma Cohen (1992) (tusiga), ao Gelman and Carlin (2014) ofoina atu se vaaiga e ese le eseesega. E mafai ona aofia i totonu o le mamanu ma le auiliiliga o faʻataʻitaʻiga le muaʻi togafitiga; mataupu 4 o Gerber and Green (2012) maua ai se folasaga lelei i auala uma e lua, ma Casella (2008) maua ai le sili atu le loloto o togafitiga. O faʻaaogaga o loʻo faʻaaogaina lenei faʻamatalaga muamua i togafitiga i le faʻataunuʻuina e masani lava ona faʻalavelaveina faʻataʻitaʻiga faʻataʻitaʻiga poʻo mamanu faʻataʻitaʻiga faʻamaonia (e le o faʻaogaina faʻamalosi upu i nuʻu uma); o nei metotia e fesoʻotaʻi vavalalata ma metotia o sampling faʻataʻitaʻiina na faʻatalanoaina i le mataupu 3. Vaʻai Higgins, Sävje, and Sekhon (2016) mo nisi faʻamatalaga e uiga i le faʻaaogaina o nei mamanu i ni suesuega tetele. E mafai foi ona aofia i totonu o le auiliiliga o togafitiga muamua le togafitiga. McKenzie (2012) suʻesuʻeina le eseesega i le eseesega o le faʻavasegaina o suʻesuʻega faʻapitoa i auiliiliga. Vaai Carneiro, Lee, and Wilhelm (2016) mo nisi faʻamatalaga i luga o fefaʻatauaiga i le va o auala eseese e faʻateleina ai le saʻo i fua faʻatatau o togafitiga. Ma le mea mulimuli, pe a filifili pe taumafai e faʻaaofia faʻasalaga muamua-togafitiga i le mamanu poʻo le faʻatalanoaga (po o mea uma e lua), o loʻo i ai ni nai mea e mafaufau ai. I se nofoaga e fia iloa ai e le au suʻesuʻe latou te le "fagota" (Humphreys, Sierra, and Windt 2013) , o le faʻaaogaina o togafitiga muamua i togafitiga o loʻo mafai ona aoga (Higgins, Sävje, and Sekhon 2016) . I tulaga e taunuʻu ai sui usufono, aemaise lava faʻataʻitaʻiga i luga o le initaneti, o le faʻaaogaina o faʻamatalaga muamua i togafitiga i le mamanu faʻavae e ono faigata ona faʻatinoina; vaai, mo se faataitaiga, Xie and Aurisset (2016) .

E taua le faaopoopoina o sina malamalamaga e uiga i le mafuaʻaga e sili atu le lelei o le eseesega-i-eseesega nai lo se eseesega-i-o lona uiga tasi. O le tele o taunuʻuga i luga o le initaneti e matua maualuga lava le eseesega (faʻaf. RA Lewis and Rao (2015) , RA Lewis and Rao (2015) ma le Lamb et al. (2015) ) ma e tumau le mautu i le taimi. I lenei tulaga, o le numera o suiga o le ai ai le eseesega itiiti, ma faateleina le mana o le suʻega fuainumera. O se tasi o mafuaʻaga e le masani ona faʻaaogaina lenei faiga, o le a le masani ona i ai muamua muamua le togafitiga muamua aʻo leʻi oʻo i le tausaga numera. O se auala e sili atu ona lelei e mafaufau ai e uiga i lenei mea, o le mafaufau lea i se suʻega e fuafua pe o faʻaaogaina masani masani e mafua ai le paʻu o le mamafa. Afai e te faia se eseesega-i-o lona uiga o le faʻatulagaga, o lau faʻatusatusaga o le ai ai fesuiaiga e tulaʻi mai le fesuisuiai i fua o le faitau aofaʻi. Afai e te faia se eseesega-i-eseesega, e ui i lea, o le mea moni e tupu ai le eseesega o le mamafa o mea mamafa, ma e sili atu ona faigofie ona e iloa se eseesega e mafua mai i togafitiga.

Mulimuli ane, na ou mafaufau e faʻaopoopo le lona fa R: "toe faʻafou". O lona uiga, afai e maua e le au suʻesuʻe suʻesuʻega faʻamatalaga sili atu nai lo le mea latou te manaʻomia e folasia ai a latou fesili suʻesuʻe muamua, e tatau ona latou toe faʻafoʻi faamatalaga e fai ai ni fesili fou. Mo se faʻataʻitaʻiga, mafaufau i le faʻaaogaina e Kramer ma ana uo se eseesega-i-eseesega faʻatau ma maua ai i latou faʻamaumauga sili atu nai lo le latou manaʻomia e folasia ai a latou fesili suʻesuʻe. Nai lo le le faʻaaogaina o faʻamaumauga i se tulaga atoatoa, semanu e mafai ona latou suʻesuʻeina le tele o le aoga e avea o se galuega o faʻamatalaga faʻalagona muamua. E pei lava o Schultz et al. (2007) o le aafiaga o le togafitiga e ese mo le malamalama ma le mamafa o tagata faaaoga, atonu o aafiaga o le News Feed e ese mo tagata ua leva ona tausia le fiafia (pe faanoanoa) feau. O le toe faʻaaogaina e mafai ona taʻitaʻi atu ai i le "fagota" (Humphreys, Sierra, and Windt 2013) ma le "p-hacking" (Simmons, Nelson, and Simonsohn 2011) , ae o nei mea e sili atu ona mafai ona faʻasalalau ma se tuufaatasiga o lipoti faʻamaoni (Simmons, Nelson, and Simonsohn 2011) , muamua le lesitala (Humphreys, Sierra, and Windt 2013) , ma metotia faʻaaogaina masini e taumafai ai e aloese mai le faʻaogaina.