ukuhlaziya Okwengeziwe

Lesi sigaba yakhelwe ukuba lisetshenziswe njengeHholo reference, kunokuba ukuze ifundwe njengoba lelandzisako.

• Isingeniso (Isigaba 4.1)

Imibuzo mayelana causality e ucwaningo social ngokuvamile ziyinkimbinkimbi futhi okuyinkimbinkimbi. Ukuze uthole ndlela asisekelo ukuze causality incike kumagrafu esiyimbangela, bheka Pearl (2009) , kanye indlela asisekelo ngokusekelwe imiphumela engahle, bheka Imbens and Rubin (2015) (futhi isithasiselo lobuchwepheshe kulesi sahluko). Ukuze uqhathanise lezi zindlela ezimbili, bheka Morgan and Winship (2014) . Ukuze uthole ndlela ngokusemthethweni ngokuchaza confounder, bheka VanderWeele and Shpitser (2013) .

Esahlukweni, Ngazakhela ezazifikile umugqa elikhanyayo emkhatsini ikhono lethu lokwenza tilinganiso esiyimbangela kusuka idatha yokulinga futhi non-experiment. Esimweni esingokoqobo, ngicabanga ukuthi umahluko blurrier. Ngokwesibonelo, wonke umuntu uyawamukela ukuthi ukubhema kubanga umdlavuza ngisho nakuba siye awupheli a randomized ukuhlola ukuthi amabutho abantu ukuba ngibheme. Ngoba omuhle incwadi ubude zokwelapha ekwenzeni esiyimbangela ezakhishwa idatha non-yokulinga ukubona Rosenbaum (2002) , Rosenbaum (2009) , Shadish, Cook, and Campbell (2001) , kanye Dunning (2012) .

Izahluko 1 no-2 Freedman, Pisani, and Purves (2007) zinikeza isingeniso ecacile singene umehluko emkhatsini ucwaningo, kocwaningo, futhi randomized kocwaningo.

Manzi (2012) inikeza isingeniso elithakazelisayo futhi efundekayo singene isisekelo zefilosofi ezibalo randomized kocwaningo. Liphinde lunikeze izibonelo ezithakazelisayo wangempela wamandla zokucwaninga kwezamabhizinisi.

• Yiziphi nemaresiphi? (Isigaba 4.2)

Casella (2008) , Box, Hunter, and Hunter (2005) , Athey and Imbens (2016b) ahlinzeke izingeniso ezinhle namaphuzu ezibalo design yokulinga kanye nokuhlaziywa. Ngaphezu kwalokho, kukhona zokwelapha omuhle ukusetshenziswa ucwaningo emikhakheni eminingi ehlukene: ezomnotho (Bardsley et al. 2009) , Kwabantu (Willer and Walker 2007; Jackson and Cox 2013) , Psychology (Aronson et al. 1989) , Wezifundo zezombusazwe (Morton and Williams 2010) , kanye nenqubomgomo yezenhlalakahle (Glennerster and Takavarasha 2013) .

Ukubaluleka umhlanganyeli ekuqasheni (isib, izibonelo zalokho) kuyinto ngokuvamile ngaphansi-kangaka ucwaningo lokuhlola. Nokho, uma umphumela ukwelashwa kuyinto heterogeneous labantu, khona-ke ukusampula esibucayi. Longford (1999) uveza leli phuzu ngokucacile lapho likhuthaza abacwaningi becabanga ucwaningo nje ucwaningo labantu nge sampling ozenzekelayo.

• Ubukhulu Two ucwaningo: lab-field and analog-digital (Isigaba 4.3)

I dichotomy ukuthi nganikeza phakathi lab and field nemaresiphi kancane kwaba lula. Eqinisweni, abanye abacwaningi baye basikisela typologies eningiliziwe, ngamazibulo ikakhulukazi ehlukanisa izinhlobo ezihlukahlukene ucwaningo emkhakheni (Harrison and List 2004; Charness, Gneezy, and Kuhn 2013) . Ngaphezu kwalokho, kukhona ezinye izinhlobo ezimbili zocwaningo eyenziwa ngu ososayensi bezenhlalo ukuthi musa khaxa kuyo lab and field dichotomy. Inhlolovo kwenziwa ucwaningo kanye ucwaningo social Survey Ukuhlolwa kwenziwa ucwaningo besebenzisa ingqalasizinda vo ezikhona bese uqhathanisa izimpendulo izinguqulo ehlukile ye imibuzo efanayo (ezinye nemaresiphi inhlolovo ethulwa eSahlukweni 3); for more on inhlolovo kwenziwa ucwaningo ukubona Mutz (2011) . nemaresiphi Social kukhona nemaresiphi lapho uphathwa ezinye inqubomgomo yezenhlalakahle ongenziwa kuphela okumele zenziwe yi-uhulumeni. nemaresiphi Social zihlobene nesimiso nokuhlola. Ukuze uthole olunye ulwazi nemaresiphi inqubomgomo, bheka Orr (1998) , Glennerster and Takavarasha (2013) , kanye Heckman and Smith (1995) .

A eziningana amaphepha baye kuqhathaniswa lab and field ucwaningo abstract (Falk and Heckman 2009; Cialdini 2009) kanye mayelana nemiphumela nomthetho ucwaningo ethize e wezifundo zezombusazwe (Coppock and Green 2015) , ezomnotho (Levitt and List 2007a; Levitt and List 2007b; Camerer 2011; Al-Ubaydli and List 2013) kanye psychology (Mitchell 2012) . Jerit, Barabas, and Clifford (2013) unikeza nice ucwaningo design ngokuqhathanisa imiphumela evela lab and field ucwaningo.

Ukukhathazeka mayelana abahlanganyeli ekushintsheni ukuziphatha kwabo ngoba bayazi babekwa zakubhekisisa babizwe ngokuthi ukufuneka imiphumela, futhi baye watadisha kwengqondo (Orne 1962) kanye nakwezomnotho (Zizzo 2009) . Nakuba iningi labo ezihlobene ne lab kwenziwa ucwaningo, lezi zimpikiswano efanayo kungabangela izinkinga for ucwaningo emkhakheni kanye. Empeleni, funa imiphumela nazo ngezinye izikhathi obizwa ngokuthi imiphumela Hawthorne, igama sivela enza ucwaningo ensimini, ikakhulukazi odumile okukhanyisa ukuhlolwa kwaqala ngo-1924 ngesikhathi Hawthorne Works of the Western Electric Company (Adair 1984; Levitt and List 2011) . Bobabili imiphumela wokwakha Hawthorn imiphumela zihlobene umqondo wokumeda esisheshayo okuxoxwe ngayo eSahlukweni 2 (bheka futhi Webb et al. (1966) ).

Umlando ucwaningo emkhakheni iye yachazwa ezinhlwini ezingu ezomnotho (Levitt and List 2009) , isayensi yezombangazwe (Green and Gerber 2003; Druckman et al. 2006; Druckman and Lupia 2012) , Psychology (Shadish 2002) , kanye nenqubomgomo yomphakathi (Shadish and Cook 2009) . Enye indawo ye-social science lapho kwenziwa ucwaningo emkhakheni ngokushesha yayiya ivelela kuyinto ukuthuthukiswa international. Ukuze uthole ukubukezwa omuhle walowo msebenzi ngaphakathi ezomnotho ukubona Banerjee and Duflo (2009) , futhi luhlolo esibucayi ukubona Deaton (2010) . Ukuze uthole ukubukezwa kwezinqumo lo msebenzi wezifundo zezombusazwe ukubona Humphreys and Weinstein (2009) . Ekugcineni, zezinselele zezindinganiso abathintekayo ngokuhlolwa yasensimini baye lucwaningwa wezifundo zezombusazwe (Humphreys 2015; Desposato 2016b) kanye nokuthuthukiswa economics (Baele 2013) .

Esahlukweni, ngasikisela ukuthi ulwazi pre-ukwelashwa kungasetshenziswa ngcono ikwazi kulinganiselwa imiphumela ukwelashwa, kodwa kukhona abanye mpikiswano mayelana le ndlela: Freedman (2008) , Lin (2013) , kanye Berk et al. (2013) ; bheka Bloniarz et al. (2016) ukuze uthole ulwazi oluthe xaxa.

• Ukuthuthela ngaphesheya nemaresiphi elula (Isigaba 4.4)

Ngiye okhethe ukugxila kuwo imiqondo emithathu: kwemukeleka, heterogeneity imiphumela yokwelashwa, kanye netindlela. Lezi imiqondo anamagama ahlukene emikhakheni ehlukene. Ngokwesibonelo, odokotela bengqondo bathambekele ukudlulela ngale kwenziwa ucwaningo elula ngokugxila balamuli nabomengameli (Baron and Kenny 1986) . Umqondo we balamuli ethunjwa engikubiza ngokuthi izindlela, kanye nomqondo lalabalungisi ethunjwa engikubiza ngokuthi kwemukeleka yangaphandle (isib, ingabe imiphumela ukuhlola lingafani uma elaliphethwe ezimweni ezahlukene) kanye heterogeneity yemiphumela yokwelashwa ( isib, imiphumela okukhulu abanye abantu kunabanye abantu).

Ucwaningo of Schultz et al. (2007) ikhombisa ukuthi eyeza nezazi zezinkanyezi ezikhuluma social kungasetshenziswa ukudizayina izinhlelo zokungenelela ngempumelelo. Ukuze uthole i-agumenti okwengeziwe jikelele ngendima theory ukuklama setfule kungenelela ngempumelelo, bheka Walton (2014) .

• Kwetsembeka (Isigaba 4.4.1)

Imiqondo sokusebenza yangaphakathi nangaphandle base yethulwa ngo Campbell (1957) . Bheka Shadish, Cook, and Campbell (2001) ukuze uthole umlando eningiliziwe futhi kwenaba ngokucophelela of kwezibalo isiphetho kwemukeleka, kwemukeleka yangaphakathi, ukwakha kwemukeleka, nobuqiniso zangaphandle.

Ukuze ekuhlaziyeni nezindaba ezihlobene ezibalo esiphethweni kwemukeleka ucwaningo ukubona Gerber and Green (2012) (for a nombono social science) kanye Imbens and Rubin (2015) (for a nombono kwezibalo). Ezinye nezindaba of ezibalo isiphetho esisemthethweni kuphakama ikakhulukazi online ensimini kwenziwa ucwaningo zihlanganisa nezindaba ezifana nezindlela computationally ephumelelayo ngokudala ukuzethemba ngezikhathi ngedatha nasekuthengeni (Bakshy and Eckles 2013) .

Ukuba semthethweni kwangaphakathi kungaba nzima ukuqinisekisa e complex ensimini ucwaningo. Bheka, isibonelo, Gerber and Green (2000) , Imai (2005) , kanye Gerber and Green (2005) I-mpikiswano mayelana ukuqaliswa eyinkimbinkimbi ensimini experiment mayelana yokuvota. Kohavi et al. (2012) kanye Kohavi et al. (2013) isingeniso amehlo mayelana nezinselelo ezikhona sokusebenza isikhawu online ensimini ucwaningo.

Enye ukukhathazeka enkulu kwemukeleka yangaphakathi izinkinga ne eqenjini. Enye indlela eya ezingabangela izinkinga ne eqenjini ukuqhathanisa ekwelapheni kanye control amaqembu ezicini ebonakalayo. Lolu hlobo lokuqhathanisa ubizwa ngokuthi isheke balance. Bheka Hansen and Bowers (2008) indlela kwezibalo ukuba silinganisele amasheke, futhi ubone Mutz and Pemantle (2015) ngezinkathazo iqondene balance. Ngokwesibonelo, esebenzisa balance hlola Allcott (2011) bathola ukuthi kukhona abanye ubufakazi bokuthi eqenjini awuzange lwenteke kahle ezintathu ucwaningo kwezinye OPower nemaresiphi (bheka iThebula 2; sites 2, 6, 8). Ukuthola eminye kusondela, bheka Imbens and Rubin (2015) , Isahluko 21.

Okunye ukukhathazeka ezinkulu ezihlobene kwemukeleka ngaphakathi: 1) one-Amasa asekela non-ukuhambisana, lapho hhayi wonke umuntu eqenjini ukwelashwa empeleni wathola ukwelashwa, 2) ezimbili esehlanganyela non-ukuhambisana, lapho hhayi wonke umuntu eqenjini elashwa ukwelashwa kanye ezinye abantu eqenjini control ukuthola ukwelashwa, 3) ukuzithumela kwezinye izindawo nokufa, lapho imiphumela akulinganiswa for ezinye ababambiqhaza, futhi 4) nokugxambukela, lapho ukwelashwa lichitheke phezu esuka kubantu esimweni ukwelashwa abantu isimo control. Bheka Gerber and Green (2012) Izahluko 5, 6, 7, no-8 ukuze uthole okuthe xaxa ngayinye kulezi zindaba.

Ukuze uthole olunye ulwazi enza semthethweni, bheka Westen and Rosenthal (2003) , esingaphezu on enza ukuba semthethweni e big imithombo idatha, Lazer (2015) kanye eSahlukweni 2 sale ncwadi.

Esinye isici sokusebenza yangaphandle setting lapho i kokungenela lihlolwe. Allcott (2015) inikeza ukwelashwa ngokucophelela theory kanye enokwehla ukukhethwa site noma ukuchema kuzo. Lo magazini kuyaxoxwa ngaso Deaton (2010) . Ngaphezu kokuba ziphindaphindwe ezindaweni eziningi, ukungenelela Home Energy Umbiko nakho okuye ngokuzimela wafundela namaqembu ocwaningo amaningi (isib, Ayres, Raseman, and Shih (2013) ).

• Heterogeneity nemiphumela yokwelashwa (Isigaba 4.4.2)

Ukuze ekuhlaziyeni omuhle ka heterogeneity nemiphumela yokwelashwa ucwaningo emkhakheni, bheka iSahluko 12 of Gerber and Green (2012) . Ukuze izingeniso ukuze heterogeneity nemiphumela yokwelashwa ekulingweni yezokwelapha, bheka Kent and Hayward (2007) , Longford (1999) , kanye Kravitz, Duan, and Braslow (2004) . Heterogeneity nemiphumela yokwelashwa ngokuvamile ukugxila umehluko esekelwe pre-ukwelashwa izici. Uma unesithakazelo heterogeneity ngokusekelwe imiphumela post-ukwelashwa, khona-ke approachs eziyinkimbinkimbi ngokwengeziwe bayadingeka ezifana babhekane nezinga elikhulu uthishanhloko (Frangakis and Rubin 2002) ; bheka Page et al. (2015) sokuba kubuyekezwe.

Abacwaningi abaningi balinganisela heterogeneity nemiphumela yokwelashwa usebenzisa ekuhlehleni komugqa, kodwa izindlela newer ancike umshini ofunda izinto, isibonelo Green and Kern (2012) , Imai and Ratkovic (2013) , Taddy et al. (2016) , kanye Athey and Imbens (2016a) .

Kukhona ezinye zokungabaza okwatholakala heterogeneity nemiphumela ngenxa yezinkinga kuqhathanisa eziningi futhi "lokudoba." Kunezindlela ezihlukahlukene izindlela ezibalo ezingasiza ikheli ukukhathazeka ngokufana amaningi (Fink, McConnell, and Vollmer 2014; List, Shaikh, and Xu 2016) . Enye indlela ukuba ukukhathazeka mayelana "wokudoba" pre-ukubhaliswa, okuyinto kuya kuvama kwengqondo (Nosek and Lakens 2014) , isayensi ezombusazwe (Humphreys, Sierra, and Windt 2013; Monogan 2013; Anderson 2013; Gelman 2013; Laitin 2013) , nakwezomnotho (Olken 2015) .

Ekuhloleni Costa and Kahn (2013) kuphela cishe isigamu amakhaya ukuhlola sakwazi ukuba kuhlobene nolwazi lokubala ubuningi babantu. Abafundi abanesifiso sokwazi imininingwane kanye nezinkinga kungenzeka nale analysis kufanele ubheke leli phepha original.

• Izindlela (Isigaba 4.4.3)

Izindlela ngendlela emangalisayo ezibalulekile, kodwa ethubeni lokuba kube nzima kakhulu ukufunda. Ucwaningo mayelana netindlela esihlobene kakhulu kwisifundo sanoma balamuli kwengqondo (kodwa bheka futhi VanderWeele (2009) I-ukuqhathanisa eqondile phakathi imibono emibili). Tindlelanchubo Statistical umshini ekutholeni, ezifana ndlela athuthukiswe in Baron and Kenny (1986) , ivamile impela. Ngeshwa, kuvela ukuthi lezo zinqubo esithembele kuso ezinye ezicatshangwayo eqinile (Bullock, Green, and Ha 2010) kanye abhekana nazo lapho kuneendlela amaningi, njengoba ongakulindela ezimweni eziningi (Imai and Yamamoto 2013; VanderWeele and Vansteelandt 2014) . Imai et al. (2011) kanye Imai and Yamamoto (2013) anikele ezinye izindlela ezibalo improved. Ngaphezu kwalokho, VanderWeele (2015) inikeza ukwelashwa incwadi sobude nabantu abaningi imiphumela ebalulekile, kuhlanganise indlela olunzulu ukuze uzwela analysis.

Indlelanchubo ehlukile ligxile ukuhlolwa azame ukulawula indlela ngqo (isib, enikeza amatilosi vitamin C). Ngeshwa, ngo izilungiselelo eziningi social science kukhona ngokuvamile tindlela eziningi futhi kunzima ukuklama zokwelapha ukushintsha ngaphandle kokushintsha abanye. Abanye izindlela Ucwaningo tindlela ukuma zichazwe Imai, Tingley, and Yamamoto (2013) , Ludwig, Kling, and Mullainathan (2011) , kanye Pirlott and MacKinnon (2016) .

Ekugcineni, tindlela banomlando omude ifilosofi nesayensi sinjengoba sichazwa Hedström and Ylikoski (2010) .

• Ukusebenzisa izimo ekhona (Isigaba 4.5.1.1)

Ukuze uthole olunye on ukusetshenziswa izincwadi izifundo kanye nezifundo audit ukukala ukucwaswa ukubona Pager (2007) .

• Yakha sokuhlolwa kwakho siqu (Isigaba 4.5.1.2)

Indlela ejwayeleke kakhulu ekuqasheni ababambiqhaza kukubuka ukuthi ukwakha kuyinto Amazon Mechanical Turk (MTurk). Ngoba MTurk ulingisa izici bendabuko lab nemaresiphi okhokhela abantu ukuqedela imisebenzi babengeke benze for free-abaningi abacwaningi seziqalile usebenzisa Turkers (izisebenzi MTurk) abanengxenye tifundvo wesintu kwenziwa ucwaningo okuholele ekutheni ushibhile iqoqo kuka yendabuko idatha ngokushesha futhi ukulinga-campus laboratory (Paolacci, Chandler, and Ipeirotis 2010; Horton, Rand, and Zeckhauser 2011; Mason and Suri 2012; Rand 2012; Berinsky, Huber, and Lenz 2012) .

Amandla enkulu nemaresiphi ne iqhaza ukubuthwa MTurk yilezi yokudla: abavumela abacwaningi ukuthola ababambiqhaza ngokushesha futhi njengoba kudingeka. Nakuba kwenziwa ucwaningo lab kungathatha amasonto ukugijima kanye ucwaningo emkhakheni kungathatha izinyanga ukusetha-up, ucwaningo nge abahlanganyeli ukubuthwa MTurk kungenziwa run ngezinsuku. Ngokwesibonelo, Berinsky, Huber, and Lenz (2012) sakwazi ukuthola tifundvo 400 ngosuku olulodwa ukuze iqhaza ukuhlola ngomzuzu 8. Ngaphezu kwalokho, kungaba lezi ababambiqhaza waqoqa nganoma yisiphi injongo (kuhlanganise ukuhlola kanye mass ngokubambisana, njengoba kuxoxwa eZahlukweni 3 no-5). Le kalula ukunxenxa kusho ukuthi abacwaningi ungagijima ukulandelanisa ucwaningo ahlobene zilandelana.

Ngaphambi abaqashwayo ababambiqhaza MTurk for ucwaningo lwakho siqu, kukhona izinto ezine ezibalulekile ukuze ukwazi. Okokuqala, abacwaningi abaningi zokungabaza non-specific ucwaningo ezihilela Turkers. Ngoba lokhu kugxeka kwabo akuyona othize, kunzima yokulwisana ubufakazi. Nokho, ngemva kweminyaka eminingana izifundo besebenzisa Turkers, thina manje siphethe ngokuthi le zokungabaza akudingekile ikakhulukazi. Sekube izifundo eziningi ngokuqhathanisa nezibalo zabantu Turkers kwamanye imiphakathi kanye nezifundo eziningi ngokuqhathanisa imiphumela ucwaningo nge Turkers imiphumela kusuka kwamanye imiphakathi. Njengoba sazi konke lo msebenzi, ngicabanga ukuthi indlela engcono kuwe ukucabanga ngakho wukuthi Turkers kukhona enengqondo lula isampula, sifana abafundi kodwa kancane wehlukile (Berinsky, Huber, and Lenz 2012) . Ngakho, njengoba nje abafundi elinabantu enengqondo abanye kodwa hhayi bonke ucwaningo ocwaningayo, Turkers kukhona inabantu enengqondo abanye kodwa hhayi bonke ucwaningo. Uma uya ukusebenza nge Turkers, khona-ke kunengqondo ukuba ufunde eziningi zalezi zifundo eqhathanisa baqonde umehluko wemisindo nemiqondo yamagama abo.

Okwesibili, abacwaningi baye bavela best-imikhuba ngenxa yokwanda yiqiniso kwalesi zangaphakathi of Turk kwenziwa ucwaningo, futhi kufanele bafunde ngoJehova futhi ulandele lezi best-imikhuba (Horton, Rand, and Zeckhauser 2011; Mason and Suri 2012) . Ngokwesibonelo, abacwaningi usebenzisa Turkers bayakhuthazwa ukusebenzisa abezokuphepha ukususa abahlanganyeli ongatjhejisisiko (Berinsky, Margolis, and Sances 2014; Berinsky, Margolis, and Sances 2016) (kodwa bheka futhi DJ Hauser and Schwarz (2015b) kanye DJ Hauser and Schwarz (2015a) ). Uma ungenalo ukususa ababambiqhaza ongatjhejisisiko, iyiphi umphumela we ukwelashwa Pho, kungasindiswa igezwa umsindo kwethulwa esuka ababambiqhaza ongatjhejisisiko, futhi empeleni inani labamba iqhaza ongatjhejisisiko kungaba olutheni. Ekuhlolweni kwe-Huber kanye nozakwabo (2012) ngu-30% of ababambiqhaza akuphumelelanga ukunakwa eziyisisekelo abezokuphepha. Enye inkinga evamile nge Turkers kuyinto abahlanganyeli non-ukuzikhohlisa (Chandler et al. 2015) .

Okwesithathu, isihlobo ezinye nezinye izinhlobo ucwaningo digital, MTurk kwenziwa ucwaningo abakwazi ukucaca; Stewart et al. (2015) ilinganisela ukuthi ngasiphi isikhathi zikhona-7,000 kuphela abantu ku MTurk.

Ekugcineni, kufanele wazi ukuthi MTurk umphakathi nemithetho yawo siqu kanye nokuthuthukiswa (Mason and Suri 2012) . Ngendlela efanayo ukuthi bazozama ukuthola mayelana isiko ezweni lapho wawuhamba ukusebenzisa ukuhlolwa yakho, kufanele sizame ukuthola okwengeziwe mayelana namasiko kanye nemithetho Turkers (Salehi et al. 2015) . Futhi, kufanele wazi ukuthi Turkers uzobe ukhuluma ngani sokuhlolwa kwakho uma nenza into ezingafanelekile noma abangenasimilo (Gray et al. 2016) .

MTurk kuyindlela emangalisayo elula ekuqasheni ababambiqhaza kukubuka wakho, kungakhathaliseki ukuthi lab-like, ezifana Huber, Hill, and Lenz (2012) , noma insimu-like more, ezifana Mason and Watts (2009) , Goldstein, McAfee, and Suri (2013) , Goldstein et al. (2014) , Horton and Zeckhauser (2016) , kanye Mao et al. (2016) .

• Yakha umkhiqizo wakho siqu (Isigaba 4.5.1.3)

Uma ucabanga sokuzama ukwakha umkhiqizo wakho siqu, mina batusa ukuthi ufunda iseluleko olunikezwa MovieLens group in Harper and Konstan (2015) . A ukuqondisisa key kulokho okwenzeka kuwo wukuthi for ngamunye project yimpumelelo ziningi, ukwehluleka abaningi. Ngokwesibonelo, iqembu MovieLens umkhankaso neminye imikhiqizo efana GopherAnswers ukuthi nokuhluleka ephelele (Harper and Konstan 2015) . Esinye isibonelo umcwaningi kokuhluleka ngenkathi kuzanywa ukwakha umkhiqizo kuyinto umzamo Edward Castronova ukwakha umdlalo online esibizwa Arden. Naphezu $ 250,000 in ngezimali, project kwaba flop (Baker 2008) . Projects like GopherAnswers futhi Arden ngeshwa ezivamile lokungetulu kwekumane amaphrojekthi afana ne MovieLens. Ekugcineni, lapho sengithi hhayi ukuthi ikhona enye abacwaningi ukuthi yayiphumelele eyakhelwe imikhiqizo zokucwaninga okuphindaphindiwe akazange nanka criteria yami: 1) ababambiqhaza ukusebenzisa umkhiqizo ngoba lokho ibanikeze (isib, kungukuthi ikhokhwe futhi akuzona Izisebenzi zokuzithandela esiza nezesayensi) kanye 2) umkhiqizo liye lasetshenziswa ukuze experiment ezingaphezu kweyodwa abahlukene (ie, hhayi ukuhlolwa efanayo izikhathi eziningi ezahlukene umhlanganyeli amachibi). Uma wazi ezinye izibonelo, ngicela ungazise.

• Partner nge ezinamandla (Isigaba 4.5.2)

Ngalalela umqondo ka Quadrant Pasteur esikhulume ngazo kaningi izinkampani tech, futhi kuyasiza ekuhleleni imizamo yokuthola ucwaningo Google (Spector, Norvig, and Petrov 2012) .

Bond futhi isifundo nozakwabo ' (2012) futhi izama ukuthola umphumela walezi zokwelapha on abangane balabo abathola them. Ngenxa design ukuhlola, lezi spillovers kukhona nzima ukubona ehlanzekile; Abafundi abanesifiso sokwazi kufanele ubone Bond et al. (2012) ukuze uthole ingxoxo ngokuphelele. Lokhu experiment siyingxenye emasikweni akudala ucwaningo wezifundo zezombusazwe imizamo ukukhuthaza ukuvota (Green and Gerber 2015) . Lezi nemaresiphi zokwazi out-the-ivoti zivamile ngokwengxenye ngoba basuke Quadrant Pasteur sika. Okungukuthi, kunabantu abaningi abakhuluphele ushukumiseleka ukuba wandise yokuvota nokuvota kungaba ukuziphatha ezithakazelisayo ukuze ahlole imibono more jikelele mayelana ukushintsha indlela yokuziphatha nethonya social.

Abanye abacwaningi baklame benikeza nezeluleko mayelana egijima kwenziwa ucwaningo emkhakheni nezinhlangano ezifana amaqembu ezombusazwe, ama-NGO kanye namabhizinisi (Loewen, Rubenson, and Wantchekon 2010; List 2011; Gueron 2002) . Abanye baye banikela izeluleko zokuthi ungayisebenzisa kanjani nezinhlangano kungaba nomthelela ucwaningo imiklamo (Green, Calfano, and Aronow 2014; King et al. 2007) . Partnership kungabangela imibuzo zesimilo (Humphreys 2015; Nickerson and Hyde 2016) .

• Design iseluleko (Isigaba 4.6)

Uma uya ukudala uhlelo analysis ngaphambi egijima sokuhlolwa kwakho, mina asikisela ukuthi uqale ngokufunda imihlahlandlela yokubika. Indlovukazi (Consolidated Standard Ukubika Izilingo) ziqondiso zakhiwa kwezokwelapha (Schulz et al. 2010) kanye modified ucwaningo social (Mayo-Wilson et al. 2013) . Isethi ahlobene iziqondiso lakhiwe ngokuhlanganyela abahleli be-Journal of Kusahlolwa Political Science (Gerber et al. 2014) (bheka futhi Mutz and Pemantle (2015) kanye Gerber et al. (2015) ). Ekugcineni, imihlahlandlela nokubika kuye kwasungulwa kwengqondo (Group 2008) , futhi bheka futhi Simmons, Nelson, and Simonsohn (2011) .

Uma ukudala uhlelo analysis kufanele ucabangele pre-kokubhalisa it ngoba pre-ukubhaliswa kuyokwandisa ukuqiniseka ukuthi abanye bahlupheka ngaphezu kwimiphumela yakho. Ngaphezu kwalokho, uma usebenza nomlingani, it uzoba nomkhawulo enanini ikhono umlingani wakho ukushintsha ukuhlaziywa ngemva kokubona imiphumela. Oku-ukubhaliswa kuya kuvama kwengqondo (Nosek and Lakens 2014) , isayensi ezombusazwe (Humphreys, Sierra, and Windt 2013; Monogan 2013; Anderson 2013; Gelman 2013; Laitin 2013) , kanye nakwezomnotho (Olken 2015) .

Nakuba sakha pre-analysis plan yakho kufanele uqaphele ukuthi abanye abacwaningi ibuye isebenzise ukuhlehla kanye izindlela okuhlobene ngcono ngokunemba ezilinganiselwa ukwelashwa nomphumela, futhi kukhona amanye mpikiswano mayelana le ndlela: Freedman (2008) , Lin (2013) , kanye Berk et al. (2013) ; bheka Bloniarz et al. (2016) ukuze uthole ulwazi oluthe xaxa.

Design iseluleko ngqo ku-intanethi insimu nemaresiphi is yethuliwe Konstan and Chen (2007) kanye Chen and Konstan (2015) .

• Dala zero nedatha yezindleko okuguquguqukayo (Isigaba 4.6.1)

Ukuze uthole olunye on ucwaningo MusicLab, bheka Salganik, Dodds, and Watts (2006) , Salganik and Watts (2008) , Salganik and Watts (2009b) , Salganik and Watts (2009a) , kanye Salganik (2007) . Ukuze uthole olunye ulwazi winner-thatha-konke izimakethe, bheka Frank and Cook (1996) . Ukuze uthole olunye ulwazi inhlanhla untangling nangekhono more ngokuvamile, bheka Mauboussin (2012) , Watts (2012) , kanye Frank (2016) .

Kukhona enye indlela ukuqeda izinkokhelo obambe iqhaza abacwaningi kufanele usebenzise ngokuqaphela: izinsizwa. In eziningi online field nemaresiphi ababambiqhaza ngokuyisisekelo esebuthelwa ucwaningo never inzuzo. Izibonelo zalokhu ndlela zihlanganisa Restivo futhi van de Rijt sika (2012) ukuhlolwa imivuzo e Wikipedia kanye Bond nalowo asebenzisana sika (2012) ukuhlolwa ekhuthaza abantu ukuba bavote. Lezi nemaresiphi musa ngempela babe zero izindleko variable, banayo zero izindleko variable abacwaningi. Nakuba izindleko eziningi lokhu kucwaninga incane kakhulu ukuba umuntu ngamunye owayekhona, izindleko ezincane eziphoqelelwe isibalo omkhulu ababambiqhaza zingawenza ngokushesha. Abacwaningi egijima massive nemaresiphi online ngokuvamile bathethelele ukubaluleka encane ezigidini imiphumela yokwelapha bungane ngokuthi miphumela encane ibaluleke uma isicelo abantu abaningi. The ngqo ukucabanga esifanayo siyasebenza nakubafazi izindleko abacwaningi ukuphoqelela abahlanganyeli. Uma kwenziwa ucwaningo lwakho likwenza abantu esigidini esisodwa baya ezigidini ukuchitha umzuzu owodwa, ukuhlolwa ayiyona ingozi kakhulu kunoma yimuphi umuntu othize, kodwa ekuhlanganisweni ke udlale iminyaka ethi ayibe mibili isikhathi.

Enye indlela yokwenza zero variable izindleko zokukhokha abahlanganyeli ukusebenzisa lottery, indlela esiye yasetshenziswa nasezintweni ucwaningo Inhlolovo (Halpern et al. 2011) . Ekugcineni, for more mayelana ukuklama kujabulise umsebenzisi-nakho ukubona Toomim et al. (2011) .

• Esikhundleni Cwenga, futhi Nciphisa (Isigaba 4.6.2)

Nazi izincazelo original of the ezintathu R, kusukela Russell and Burch (1959) :

"Ukubuyiselwa kushiwo esikhundleni abazi ophilayo izilwane ephakeme material insentient. Ukuncishiswa kusho ukuncipha izinombolo izilwane esetshenziswa ukuthola ulwazi of imali inikezwe nokunemba. Intuthuko kushiwo noma yimuphi ukwehla izigameko noma kobukhulu izinqubo ngonya nakilabo izilwane kusadingeka ukuba kusetshenziswe. "

Laba bobathathu R ukuthi mina phakamisa musa ukukhipha nezimiso zokuziphatha ezichazwe eSahlukweni 6. Kunalokho, zingabantu inguqulo nekuchazwa elilodwa zalezo zimiso-beneficence-ngqo setting ucwaningo womuntu.

Lapho ucabangela Contagion Emotional, zikhona izingqinamba ezintathu non-zesimilo okufanele sihlale sizikhumbula lapho ehumusha lesi silingo. Okokuqala, akucaci ukuthi imininingwane langempela ukuhlola ukuxhuma okushiwo theory; ngamanye amazwi, kunemibuzo ephathelene enza ikhombe kwetsembeka. It Akucaci ukuthi emihle nemibi izwi okuzama empeleni inkomba ezinhle isimo esingokomzwelo ababambiqhaza ngoba 1) akucaci ukuthi amazwi abantu post kukhona inkomba imizwelo yabo futhi 2) akucaci ukuthi oluthile inhliziyo ukuhlaziya amasu ukuthi abacwaningi esetshenziswa uyakwazi thembeke ukuphetha imizwelo (Beasley and Mason 2015; Panger 2016) . Ngamanye amazwi, ingase ibe khona ngezinga elithile emibi isignali ekubandlululeni. Okwesibili, i-design kanye nokuhlaziywa ukuhlola lisitshela lutho mayelana owayebizwa nomthelela kakhulu (ie, akukho analysis of heterogeneity imiphumela yokwelashwa) futhi lokho mshini kungase kube. Kulokhu, abacwaningi kwadingeka ulwazi oluningi mayelana abahlanganyeli, kodwa baphindiselwa ngokuyisisekelo ziphathwe njengemfucuza amawijethi analysis. Okwesithathu, ubukhulu umphumela kule experiment lalilincane kakhulu; umehluko phakathi ukwelashwa kanye control nemibandela imayelana 1 ngamazwi 1,000. Ngo iphepha, Kramer kanye nozakwabo wenze icala umphumela lokhu size kubalulekile ngoba emakhulwini ezigidi zabantu ukufinyelela News yabo Feed usuku ngalunye. Ngamanye amazwi, bexabana ukuthi ngisho nemiphumela ezincane umuntu ngamunye they are big ekuhlanganisweni. Ngisho noma ubungase ukwamukela le ngxoxo, kusenzima akucaci noma umphumela lokhu size kubalulekile mayelana umbuzo more jikelele yisayensi ngezingozi esithathelwanayo ngokomzwelo. Ukuze uthole olunye on the izimo lapho imiphumela ezincane zibalulekile ukubona Prentice and Miller (1992) .

NgokoMthetho we-R wokuqala (Ukubuyiselwa), nokumadanisa Emotional Contagion experiment (Kramer, Guillory, and Hancock 2014) kanye esithathelwanayo ngokomzwelo experiment yemvelo (Coviello et al. 2014) unikeza ezinye izifundo jikelele mayelana ukudela okuthile bahilelekile babesuka nemaresiphi kukubuka yemvelo (kanye nezinye izindlela ezibonisa like ukumadanisa lowo mzamo ukuba zazikuphi ucwaningo idatha non-ocwaningayo, bheka iSahluko 2). Ngaphezu izinzuzo zokuziphatha, ukushintshela kumakhondomu yokulinga izifundo non-yokulinga godu kwenza abacwaningi ukutadisha zokwelapha ukuthi zingabantu logistically engakwazi phaka. Lezi izinzuzo zokuziphatha futhi yokudla Nokho ziyabiza,. Ngokuhlolwa zemvelo abacwaningi ukulawula kancane phezu izinto efana ukunxenxa ababambiqhaza, eqenjini, kanye nohlobo lwamathekisthi ukwelashwa. Ngokwesibonelo, omunye nemikhawuko imvula njengendlela yokwelapha wukuthi bobabili kwandisa unethemba kunciphe engemihle. Nokho, Ocwaningweni yokulinga, Kramer kanye nozakwabo sakwazi ukuba ulungise unethemba futhi engemihle ngokuzimela.

Le ndlela ethile olusetshenziswa Coviello et al. (2014) laphinda nekuchazwa e Coviello, Fowler, and Franceschetti (2014) . Ukuze isingeniso eziguquguqukayo instrumental ukubona Angrist and Pischke (2009) (ngaphansi lehlelekile) noma Angrist, Imbens, and Rubin (1996) (more lehlelekile). Ukuze sihlole uyangabaza eziguquguqukayo instrumental ukubona Deaton (2010) , futhi isingeniso eziguquguqukayo instrumental zomculo obuthakathaka (imvula liyithuluzi obuthakathaka), bheka Murray (2006) .

Ezimweni ezivamile, i-isingeniso esihle kukubuka yemvelo Dunning (2012) , kanye Rosenbaum (2002) , Rosenbaum (2009) , kanye Shadish, Cook, and Campbell (2001) zinikeza imibono emihle mayelana kulinganiswa imiphumela esiyimbangela ngaphandle ucwaningo.

NgokoMthetho we-R yesibili (intuthuko), kukhona zesayensi kanye yokudla ukudela okuthile lapho ucabangela ekushintsheni design of Contagion Emotional vinjwa izikhala ekufukuleni izikhala. Ngokwesibonelo, kungase kube icala ukuqaliswa lobuchwepheshe Okuphakelayo News kwenza kuba kakhulu kulula ukwenza i linga ngamasu ahlukahlukene ngenhloso luvimbe izikhala kunokuba thizeni ekufukuleni izikhala (phawula ukuthi i linga ngamasu ahlukahlukene ngenhloso luvimbe izikhala babengase luyasetshenziswa njengendlela ungqimba on top of the system News Feed ngaphandle kungadingeki nakancane ukuba kusetshenziswe ukushintshwa uhlelo ecashile). Ngokwesayensi, Nokho, inkolelo-mbono iphendulwa ukuhlola akazange ngokucacile basikisela design omunye phezu komunye.

Ngeshwa, mina angingowaleli ukuqaphela lobuKhristu ucwaningo ngaphambi mayelana nokuthi wayeqinisile yini uma isihlobo ukuvimba ekufukuleni okuqukethwe in the Okuphakelayo News. Futhi, angizange ngimbone ucwaningo okuningi mayelana ngcono zokwelapha ukubenza ayiyona ingozi; omunye Okungabalulwa Jones and Feamster (2015) , lapho icabanga Endabeni nesilinganiso ukucwaninga izici Internet (a sihloko I ngakho eSahlukweni 6 ebuhlotsheni cwaningo Encore (Burnett and Feamster 2015; Narayanan and Zevenbergen 2015) ).

NgokoMthetho we-R wesithathu (Ukuncishiswa), a isingeniso esihle bendabuko amandla analysis Cohen (1988) . Oku-ukwelashwa covariates kungafakwa esigabeni design kanye nokuhlaziywa esigabeni ucwaningo; Isahluko 4 soMthetho Gerber and Green (2012) inikeza isingeniso esihle ukuze kokubili kusondela, futhi Casella (2008) inikeza ukwelashwa more ngokujulile. Amasu ukusebenzisa lolu lwazi wangaphambi inqubo yokwelapha ngokwamaqiniso eqenjini zivame esibizwa noma ivinjiwe imiklamo yokulinga noma wakubo imiklamo zokuhlola (amagama alisetshenziswa njalo kuwo imiphakathi); kulezi zindlela kakhulu ezihlobene amasu izibonelo zalokho wakubo okuxoxwe ngayo eSahlukweni 3. Bheka Higgins, Sävje, and Sekhon (2016) ukuze uthole okuthe xaxa ngokusebenzisa miklamo ucwaningo okukhulu. Oku-ukwelashwa covariates kungabuye kufakwe analysis esiteji. McKenzie (2012) ichaza umehluko-in-umehluko approach to ehlaziya kwenziwa ucwaningo emkhakheni ngokuningiliziwe. Bheka Carneiro, Lee, and Wilhelm (2016) ukuze uthole okunye mayelana ukudela okuthile phakathi izindlela ezingafani ezisetshenziswa ukwandisa ekwakhiweni izilinganiso imiphumela yokwelashwa. Ekugcineni, lapho enquma ukuthi ukuzama zihlanganisa pre-ukwelashwa covariates at the design noma analysis esigabeni (noma kokubili), kukhona izinto ezimbalwa okumele ukucabange. Esimweni lapho abacwaningi bafuna ukukhombisa ukuthi ababona "wokudoba" (Humphreys, Sierra, and Windt 2013) , besebenzisa covariates pre-ukwelashwa esigabeni design ingaba wusizo (Higgins, Sävje, and Sekhon 2016) . Ezimweni lapho abahlanganyeli zifika ngokulandelana, ikakhulukazi online insimu nemaresiphi besebenzisa ulwazi pre-ukwelashwa esigabeni design kungase kube nzima logistically, bheka isibonelo Xie and Aurisset (2016) .

Kuwufanele enezela kancane umuzwa mayelana nokuthi kungani umehluko-in-ukungezwani kungaba kuphumelela kangcono kunokuthi umehluko-in-izindlela. ImiPhumela Jikelele Abaningi online babe ukuhluka kakhulu (bheka isib, Lewis and Rao (2015) kanye Lamb et al. (2015) ) futhi okuzinzile ngokuhamba kwesikhathi. Kulokhu, ushintsho score kuyodingeka ukuhluka kakhulu ezincane, okwandisa amandla test kwezibalo. Esinye isizathu sokuthi kungani lokhu wasondela alisetshenziswa kaningi wukuthi ngaphambi yobudala digital kwakungeyona evamile ukuba nemiphumela nomthetho pre-ukwelashwa. Indlela ukhonkolo esiningi sokucabanga ngawo ukuba zicabange UCWANINGO ukukala ukuthi i ukuzivivinya umzimba othize kubangela kwesisindo. Uma wenza umehluko-in-izindlela ndlela, isilinganiso sakho kuyodingeka variability ukuthi livela variability in izisindo labantu. Nokho, Uma wenza umehluko-in-umehluko ndlela, ukuthi ukushintshashintsha zemvelo e izisindo sisuswa futhi kalula thola umehluko odalwe ukwelashwa.

Enye indlela ebalulekile ukuze kuncishiswe inani labamba iqhaza ekuhlolweni kwakho kuyinto azokuqhubela analysis amandla, okuyinto Kramer kanye nozakwabo owayengenza esekelwe osayizi umphumela baphawula kusukela kuhlelo lokusebenza emvelo ngokuthulisa Coviello et al. (2014) noma ngaphambili ucwaningo non-yokulinga by Kramer (2012) (eqinisweni lezi yizinto ekupheleni kwalesi sahluko). Phawula ukuthi le ukusetshenziswa kwamandla analysis kancane ezahlukene ukwedlula ejwayelekile. In the age analog, ngokuvamile bacwaningi amandla ukuhlaziywa ukuze aqiniseke ukuthi ukutadisha kwabo kwakungeyona ezincane kakhulu (ie, ngaphansi-sikagesi). Nokho, Manje, abacwaningi okufanele ukwenze amandla ukuhlaziywa ukuze uqiniseke ukuthi ukutadisha kwabo akuyona engahlula (ie, phezu-sikagesi).

Ekugcineni, ngabhekisisa enezela R wesine: repurpose. Okungukuthi, uma abacwaningi bazithola ngedatha yokulinga okungaphezu kwalokho abakudingayo ukuze kubhekwane mbuzo wabo original ucwaningo, kufanele repurpose idatha ukubuza imibuzo emisha. Ngokwesibonelo, ake sithi Kramer kanye nozakwabo wayesebenzise umehluko-in-umehluko Estimator futhi bazithola ngedatha omningi kunalokho okudingekile ukuze kubhekwane mbuzo ucwaningo lwabo. Kunokuba akuthuthukiswa ukusebenzisa idatha ngokugcwele, babekwazi sengifunde ubukhulu nomphumela njengoba umsebenzi pre-ukwelashwa nkulumo ngokomzwelo. Njengoba nje Schultz et al. (2007) yathola ukuthi umphumela ukwelashwa ezahlukene ukukhanya ababhema, mhlawumbe imiphumela ye Okuphakelayo News zazihlukile for ntu kakade zazivame ukuba post imiyalezo happy (noma sad). Repurposing kungase kuholele ekusebenziseni "wokudoba" (Humphreys, Sierra, and Windt 2013) nesithi "p-nokugenca" (Simmons, Nelson, and Simonsohn 2011) , kodwa laba basuke ikakhulu addressable inhlanganisela yokubika othembekile (Simmons, Nelson, and Simonsohn 2011) , pre-ukubhaliswa (Humphreys, Sierra, and Windt 2013) , kanye nezindlela zokufunda umshini ukuthi azame ukugwema kuka-fitting.