ezimbalini ngakumbi

Eli candelo yenzelwe ukuba isetyenziswe njengesingqiniso, kunokuba kufundwe njengokuba ebalisayo.

• Intshayelelo (Section 4.1)

Imibuzo malunga kse kuphando zentlalo zidla entsonkothileyo ubugocigoci. Kuba indlela nolusisiseko kwi kse isekelwe kwiigrafu woko, bona Pearl (2009) , kunye indlela esisiseko esekelwe kwiziphumo ezinokwenzeka, bona Imbens and Rubin (2015) (kunye nesihlomelo yobugcisa kwesi sahluko). Kuba uthelekiso phakathi kwezi ndlela zimbini, bona Morgan and Winship (2014) . Kuba indlela esesikweni ekumiseleni ibe confounder, bona VanderWeele and Shpitser (2013) .

Kwisahluko, endiyidalele into ebonakala ngathi umgca eliqaqambileyo phakathi nakho kwethu ukwenza uqikelelo nakuyo evela data ngawovavanyo non-zovavanyo. Enyanisweni, ndicinga ukuba umahluko blurrier. Ngokomzekelo, wonke wamkela ukuba ukutshaya ebangela umhlaza nangona esingazange wenze kwelinge alawulwayo ukuba inyanzela abantu abatshayayo. Kuba unyango ubude incwadi esiluncedo ukwenza uqikelelo nakuyo evela data non-zovavanyo ukubona Rosenbaum (2002) , Rosenbaum (2009) , Shadish, Cook, and Campbell (2001) , kunye Dunning (2012) .

Isahluko 1 and 2 Freedman, Pisani, and Purves (2007) zinika ulwaziso ngokucacileyo kwi umahluko phakathi imifuniselo, imifuniselo elawulwayo, kwaye olungenamkhethe imifuniselo elawulwayo.

Manzi (2012) libonelela intshayelelo nezinomtsalane efundeka kungena ezixhasa bulumko kunye zobalo imifuniselo alawulwayo. Kwakhona inika imizekelo umdla wokwenene-ihlabathi ngamandla nokulinga kwishishini.

• Zintoni imifuniselo? (Section 4.2)

Casella (2008) , Box, Hunter, and Hunter (2005) , Athey and Imbens (2016b) ukubonelela iintshayelelo okulungileyo imiba zobalo yoyilo ngawovavanyo nohlalutyo. Ngaphezu koko, kukho unyango omhle ukusetyenziswa experimenti kwiinkalo ezininzi ezahlukeneyo: Economics (Bardsley et al. 2009) , Isosioloji (Willer and Walker 2007; Jackson and Cox 2013) , ngezengqondo (Aronson et al. 1989) , Kwinzululwazi yezopolitiko (Morton and Williams 2010) , kunye nomgaqo kwezentlalo (Glennerster and Takavarasha 2013) .

Ukubaluleka yokugaya nxaxheba (umzekelo, isampulu) luphantsi-kuthakazelelwa uphando uvavanyo. Kodwa ke, ukuba isiphumo unyango ngokweentlobo labemi, ngoko ukukhiwa kakhulu. Longford (1999) yenza oku ngokucacileyo xa elimemelela abaphandi ucinga experiments uphando abemi isampulu engalawulekiyo.

• Imilinganiso ezimbini experimenti: lab-intsimi analog-digital (Section 4.3)

Ezimbini, ukuba ndiye thaca phakathi elebhu kunye kunye entsimini experimenti isuntswana lula. Enyanisweni, abanye abaphandi baye typologies eneenkcukacha, kwi abo ingakumbi ezisahlula iintlobo ezahlukeneyo imifuniselo entsimini (Harrison and List 2004; Charness, Gneezy, and Kuhn 2013) . Ngaphezu koko, kukho ezinye iintlobo ezimbini imifuniselo eyenziwa ntle ezingangeniyo cocekileyo elebhu kunye kunye entsimini aphenjelelwe:. Imifuniselo uphando kunye amalinge zentlalo Survey imifuniselo ukuba imifuniselo usebenzisa iziseko iisaveyi ezikhoyo uze uthelekise iimpendulo iinguqulelo ezichaseneyo inye imibuzo (ezinye eksperimenti saveyi thaca kwiSahluko 3); vetshe eksperimenti uphando bona Mutz (2011) . iimvavanyo Social iyimifuniselo apho unyango nkqubo ezithile zentlalo ukuze kwenziwa kuphela ngurhulumente. imifuniselo Social ezinento kwiprogram novavanyo. Vetshe imifuniselo nkqubo, bona Orr (1998) , Glennerster and Takavarasha (2013) , kunye Heckman and Smith (1995) .

Inani lamaphepha baye kuthelekiswa elebhu kunye kunye entsimini eksperimenti bokoqobo (Falk and Heckman 2009; Cialdini 2009) kwaye ngokubhekiselele kwiziphumo eCebisayo ezithile kwinzululwazi yezopolitiko (Coppock and Green 2015) , Economics (Levitt and List 2007a; Levitt and List 2007b; Camerer 2011; Al-Ubaydli and List 2013) kunye nokusebenza kwengqondo (Mitchell 2012) . Jerit, Barabas, and Clifford (2013) inika uyilo lophando entle ngokuthelekisa iziphumo ukusuka elebhu kunye kunye entsimini imifuniselo.

Iinkxalabo malunga nxaxheba ekutshintshweni kokuziphatha kwabo kuba bayazi ukuba kwabonwa ngokusondeleyo ngamanye amaxesha kuthiwa iziphumo imfuno, yaye baye zisafundwa kwi bengqondo (Orne 1962) kunye Economics (Zizzo 2009) . Nangona enxulumene kakhulu kunye bekuhlolwa, le miba efanayo kungabangela iingxaki ukulungisela endle ngokunjalo. Enyanisweni, iziphumo imfuno nazo ngamanye amaxesha ibizwa imiphumo Hawthorne, ibinzana lusukela kwelinge entsimini, ngokukodwa yemifuniselo ukhanyiso odumileyo ezaqalisa ngo-1924 kwi-Hawthorne Yimisebenzi Western Electric Inkampani (Adair 1984; Levitt and List 2011) . Zombini iziphumo imfuno kunye neziphumo Hawthorn ezinento ingcamango zemilinganiselo esebenzayo kuchaziwe kwiSahluko 2 (makufundwe Webb et al. (1966) ).

Imbali eksperimenti endle sele kuchaziwe Economics (Levitt and List 2009) , kwinzululwazi yezopolitiko (Green and Gerber 2003; Druckman et al. 2006; Druckman and Lupia 2012) , ngezengqondo (Shadish 2002) , kwaye umgaqo-nkqubo karhulumente (Shadish and Cook 2009) . Enye indawo nangentlalo apho imifuniselo endle ngokukhawuleza baba adumileyo na uphuhliso ngamazwe. Kuba ngokutsha ezintle ukuba umsebenzi Economics ukubona Banerjee and Duflo (2009) , yaye uvavanyo ebalulekileyo ukubona Deaton (2010) . Kuba ngokutsha lo msebenzi kwinzululwazi yezopolitiko ukubona Humphreys and Weinstein (2009) . Okokugqibela, imingeni yokuziphatha inxaxheba experiment endle ziye luphononongwa kwinzululwazi yezopolitiko (Humphreys 2015; Desposato 2016b) kunye noqoqosho yophuhliso (Baele 2013) .

Kwisahluko, ndacebisa ukuba ulwazi pre-unyango ingasetyenziswa ukuphucula ngqo iziphumo zonyango kuqikelelwa, kodwa kukho ingxoxo ngale ndlela: Freedman (2008) , Lin (2013) , kunye Berk et al. (2013) ; bona Bloniarz et al. (2016) ukuze ufumane inkcazelo engakumbi.

• Ukuqhubeka ngaphaya imifuniselo ezilula (Section 4.4)

Kudala Ndikhethe bethelela ezintathu: ikuhlole, ukungafani ye iziphumo zonyango, kunye neendlela. La magama zinamagama ahlukeneyo kwiinkalo ezahlukeneyo. Umzekelo, iingcali bakholisa ukuba zizame ukwenza ngaphezu imifuniselo elula ngokujolisa ngabalamli neemodareyitha (Baron and Kenny 1986) . Ingcamango ngabalamli ifakwe koko ndiya kubiza iindlela, yaye ingcamango iimodareyitha ifakwe koko ndiya kubiza semthethweni yangaphandle (umzekelo, iziphumo zovavanyo iya kwahluka ukuba oko ndabaleka kwiimeko ezahlukeneyo) kunye ukungafani ye iziphumo zonyango ( umzekelo, ifuthe enkulu kwabanye abantu kunokuba abanye abantu).

Ukucikideka Schultz et al. (2007) ubonisa ukuba iingcamango zentlalo ingasetyenziswa ukuyila ungenelelo olusebenzayo. Kuba ingxoxo jikelele ngakumbi malunga nendima ngenkcazo ngokuyila ungenelelo olusebenzayo, bona Walton (2014) .

• Semthethweni (Section 4.4.1)

Yesigama semthethweni ngaphakathi nangaphandle baqala kwi Campbell (1957) . Bona Shadish, Cook, and Campbell (2001) ukuba imbali eneenkcukacha kunye nokunatyiswa ngenyameko semthethweni kwisigqibo manani, ukunyaniseka zangaphakathi, ukwakha ikuhlole, kunye nokunyaniseka yangaphandle.

Kuba ubenemboniselo imiba enxulumene manani isigqibo ngokusemthethweni kwi imifuniselo ukubona Gerber and Green (2012) (kuba ngokwembono nangentlalo) kwaye Imbens and Rubin (2015) (kuba ngokwembono zamanani). Eminye imiba manani isigqibo ngokusemthethweni ukuba kuvela ngokukhethekileyo kwi eksperimenti endle online ziquka imiba efana iindlela computationally kakuhle lokuqulunqela ngamathuba kukholosa data ixhomekeke (Bakshy and Eckles 2013) .

semthethweni ngaphakathi kunokuba nzima ukuqinisekisa kwi imifuniselo entsimini entsonkothileyo. Yabona, umzekelo, Gerber and Green (2000) , Imai (2005) , kunye Gerber and Green (2005) ukulungiselela ingxoxo malunga nokuphunyezwa kwe yekhompleksi yolingelo entsimini wovoto. Kohavi et al. (2012) kunye Kohavi et al. (2013) Ukubonelela kokukhutshelwa nemingeni semthethweni lokuphumla imifuniselo entsimini intanethi.

Enye inkxalabo enkulu semthethweni lwangaphakathi iingxaki engakhethiyo. Enye indlela, ukuba kunngeenzeka ukubona iingxaki engakhethiyo unokuthelekisa unyango kunye nolawulo amaqela ngeempawu eziqaphelekayo. Olu hlobo uthelekiso kuthiwa itshekhi eseleyo. Bona Hansen and Bowers (2008) ukulungiselela indlela zamanani ukulungelelanisa iinkangelo, ubone Mutz and Pemantle (2015) ukulungiselela iinkxalabo malunga iinkangelo eseleyo. Umzekelo, usebenzisa ulungelelwano khangela Allcott (2011) bafumanisa ukuba kukho ubungqina bokuba engakhethiyo zange iphunyezwe ngokuchanekileyo kwi ezintathu yemifuniselo kwezinye yemifuniselo OPower (bona iingongoma 2; amanxuwa 2, 6, 8). Kuba kwezinye iindlela, bona Imbens and Rubin (2015) , iSahluko 21.

Okunye inkxalabo enkulu enxulumene semthethweni lwangaphakathi: 1) omnye-emacaleni-ukungathotyelwa, apho wonke umntu kwiqela unyango okunene wafumana unyango, 2) ezimbini nabaxhasi-ukungathotyelwa, apho wonke umntu kwiqela lonyango lifumana unyango kunye nabanye abantu kwiqela lolawulo kufumana unyango, 3) Ukuyeka, apho iziphumo hayi zibalwa okuba abanye abathathi, no-4) uphazamiseko, apho unyango luwelela nakuwo ebantwini zikwimeko unyango abantu kwimeko yolawulo. Bona Gerber and Green (2012) Isahluko 5, 6, 7, 8 ngokungentla malunga nganye yale miba.

Vetshe kokuyilwayo ikuhlole, bona Westen and Rosenthal (2003) , kunye vetshe kokuyilwayo semthethweni kwiindawo imithombo yedatha enkulu, Lazer (2015) neSahluko 2 sale ncwadi.

Omnye umba semthethweni lwangaphandle sentlalo apho longenelelo kuvavanywa. Allcott (2015) inika unyango ngononophelo ithiyori loseni icala ukhetho site. Le mba sicaciswa Deaton (2010) . Ukongeza yaphindwa kwiindawo ezininzi, Ikhaya Energy Ingxelo ungenelelo sele ngokuzimeleyo zifundwa ngamaqela zophando ezininzi (umzekelo, Ayres, Raseman, and Shih (2013) ).

• Ukungafani ye iziphumo zonyango (Section 4.4.2)

Kuba isishwankathelo obalaseleyo ukungafani wemiphumela unyango amalinge endle, funda isahluko 12 sencwadi Gerber and Green (2012) . Kuba kwintshayelelo ukungafani wemiphumela zonyango kumalingo onyango, bona Kent and Hayward (2007) , Longford (1999) , kunye Kravitz, Duan, and Braslow (2004) . Ukungafani iziphumo zonyango ngokubanzi zigxininisa ngokwesicatshulwa iimpawu pre-unyango. Ukuba unomdla ukungafani esekelwe kwiziphumo post-unyango, ngoko approachs ezintsonkothileyo ezingaphaya ziyafuneka ezifana ngokwezinga inqununu (Frangakis and Rubin 2002) ; bona Page et al. (2015) ukulungiselela kwakhona.

Abaphandi abaninzi ukuqikelela ukungafani wemiphumela kunyango kusetyenziswa eguqulweyo yabaxwa, kodwa iindlela ezintsha sithembele ekufundeni kumatshini, umzekelo Green and Kern (2012) , Imai and Ratkovic (2013) , Taddy et al. (2016) , kunye Athey and Imbens (2016a) .

Kukho ukuthandatyuzwa iziphumo ukungafani nemiphumo ngenxa yeengxaki uthelekiso ezininzi kwaye "okuloba." Kukho iindlela ezahlukeneyo manani enokunceda idilesi iinkxalabo malunga nothelekiso ezininzi (Fink, McConnell, and Vollmer 2014; List, Shaikh, and Xu 2016) . Enye indlela iinkxalabo malunga "okuloba" owandulela-yobhaliso, kusiba eqhelekileyo bengqondo (Nosek and Lakens 2014) , kwinzululwazi yezopolitiko (Humphreys, Sierra, and Windt 2013; Monogan 2013; Anderson 2013; Gelman 2013; Laitin 2013) , kunye Economics (Olken 2015) .

Ekufundeni Costa and Kahn (2013) kuphela malunga nesiqingatha Kwimizi kwalingwa babekwazi esidityaniswe iinkcukacha ngendawo okuyo. Kubafundi abanomdla iinkcukacha kunye neengxaki ezinokubakho olu calu kufuneka ujonge iphepha lokuqala.

• Iindlela (Section 4.4.3)

zibaluleke kakhulu iindlela, kodwa kujika kuba nzima kakhulu ukufunda. Ngokusondeleyo Uphando malunga neendlela ezinxulumene isifundo yabalamli kwi bengqondo (kodwa bona VanderWeele (2009) uthelekiso ngqo phakathi iingcamango ezimbini). Iindlela zobalo ndlela yokufumana, ezifana indlela yaphuhliswa Baron and Kenny (1986) , khona eziqhelekileyo. Ngelishwa, kuvele elubala ukuba abo iinkqubo zixhomekeke ezinye kokucingelwayo ezomeleleyo (Bullock, Green, and Ha 2010) kwaye ubunzima xa kukho iindlela ezininzi, njengoko ubani ukuba kwiimeko ezininzi (Imai and Yamamoto 2013; VanderWeele and Vansteelandt 2014) . Imai et al. (2011) kunye Imai and Yamamoto (2013) zinika ezinye iindlela eziphuculweyo manani. Ngaphezu koko, VanderWeele (2015) inika unyango incwadi-ubude kunye nenani iziphumo ezibalulekileyo, kuquka indlela ebanzi kohlalutyo uvakalelo.

Indlela eyahlukileyo ijolise eksperimenti ezama ukulawula womatshini ngqo (umzekelo, ukunika oomatiloshe ivithamin C). Ngelishwa, kwiindawo ezininzi nangentlalo kukho amaninzi iindlela ezininzi yaye kunzima ukuba ukuyila unyango ukuba tshintsha enye ngaphandle kokutshintsha abanye. Ezinye iindlela ukuze ngokomfuniselo neendlela bujikwe zichazwe Imai, Tingley, and Yamamoto (2013) , Ludwig, Kling, and Mullainathan (2011) , kunye Pirlott and MacKinnon (2016) .

Okokugqibela, iindlela kwakhona zinembali ende ifilosofi kwinzululwazi njengoko kuchaziwe yi Hedström and Ylikoski (2010) .

• Ukusebenzisa bume ekhoyo (Section 4.5.1.1)

Kuba vetshe malunga nokusetyenziswa izifundo imbalelwano kunye nezifundo zophicotho ukulinganisa nocalulo ukubona Pager (2007) .

• Yakha ukucikideka zakho (Section 4.5.1.2)

Eyona ndlela iqhelekileyo ekugayeni nxaxheba ukuba imifuniselo ukuba endiyakhayo Amazon Mechanical Bonk (MTurk). Ngenxa MTurk iglu imiba experimenti ohlawula lab zemveli abantu ukuba bagqibezele imisebenzi babengayi kukwenzela abaphandi ivideo-ezininzi sele liqalisile ngokusebenzisa Turkers (abasebenzi kwi MTurk) njengabathathi kwizifundo zabantu eksperimenti leyo ngokukhawuleza kwaye zitshiphu nokuqokelelwa kwedatha ngaphezu zemveli kwi-kwikhampasi imifuniselo zelebhu (Paolacci, Chandler, and Ipeirotis 2010; Horton, Rand, and Zeckhauser 2011; Mason and Suri 2012; Rand 2012; Berinsky, Huber, and Lenz 2012) .

Amandla inkulu eCebisayo nxaxheba baqeshwa MTurk kukho amalungiselelo: bona kuvumela abaphandi ukuba ukugaya nxaxheba ngokukhawuleza njengoko kuyimfuneko. Kanti bekuhlolwa kunokuthatha iiveki ukubaleka aze afunisele endle inokuthatha iinyanga ukucwangcisa-up, umfuniselo nxaxheba baqeshwa MTurk esebenzisekayo ngeentsuku. Umzekelo, Berinsky, Huber, and Lenz (2012) sakwazi ukuqesha izifundo 400 ngosuku olunye inxaxheba ulingelo ngomzuzu 8. Ngaphezu koko, ezi -nxaxheba bafunwe ukuba nayiphi na injongo (kubandakanywa iisaveyi nobunzima intsebenziswano, njengoko kuchaziwe kwiSahluko 3 nesesi-5). Le ubulula yokugaya kuthetha ukuba abaphandi enokuqhutywa ulandelelwano eksperimenti ezinxulumene enye emva kwenye.

Phambi lufuna nxaxheba abasuka MTurk niselo yakho, kukho izinto ezine kubalulekile ukwazi. Okokuqala, abaphandi abaninzi kukulumka non-ezithile experimenti ezibandakanya Turkers. Ngenxa yokuba le kukulumka ayikho ngqo, kunzima ukuba sibali ubungqina. Noko ke, emva kweminyaka emininzi izifundo usebenzisa Turkers, sinako ngoku ukubona ukuba lo kukulumka akuyomfuneko ngakumbi. Kukho ezininzi izifundo ngokuthelekisa phethu Turkers kwezinye ngajongi izifundo ezininzi kuthelekiswa iziphumo umfuniselo Turkers ukuba iziphumo kwezinye babantu. Sijonge wonke lo msebenzi, ndiyacinga ukuba eyona ndlela wena ukucinga ngayo kukuba Turkers ezi kubeluncedo isampulu elifanelekileyo, kanye abafundi kodwa kancinane ezahlukeneyo ngaphezulu (Berinsky, Huber, and Lenz 2012) . Ngoko ke, kanye njengoko abafundi abemi elifanelekileyo kwabanye kodwa hayi lonke uphando zonyango, Turkers aba abemi elifanelekileyo kwabanye kodwa hayi lonke uphando. Ukuba uza kusebenza Turkers, ngoko iyavakala ukufunda ezininzi zezi zifundo athelekisekayo baqonde axoxe zabo.

Okwesibini, abaphandi baye ikhuliswa-zokusebenza ukulungiselela ukwandisa semthethweni ngaphakathi Bonk imifuniselo, kwaye kufuneka ufunde kwaye ulandele kwezi-izenzo (Horton, Rand, and Zeckhauser 2011; Mason and Suri 2012) . Umzekelo, abaphandi usebenzisa Turkers bayakhuthazwa ukuba basebenzise zelo ukususa nxaxheba ongakhathaliyo (Berinsky, Margolis, and Sances 2014; Berinsky, Margolis, and Sances 2016) (kodwa bona DJ Hauser and Schwarz (2015b) kwaye DJ Hauser and Schwarz (2015a) ). Ukuba awunalo ukususa nxaxheba ongakhathaliyo, ngoko ke nawuphi na isiphumo unyango zingakhukhuliseka yingxolo zivela nxaxheba ongakhathaliyo, kwaye ukwenza inani nxaxheba ongakhathaliyo kunokuba kakhulu. Xa ukucikideka Huber noogxa (2012) malunga ne-30% abathathi-nxaxheba abazange zelo ezisisiseko ingqalelo. Enye ingxaki eqhelekileyo kunye Turkers ke abathathi-abangazi (Chandler et al. 2015) .

Okwesithathu, xa kuthelekiswa nezinye iintlobo imifuniselo digital, MTurk imifuniselo ayikwazi kaNcinane; Stewart et al. (2015) iqikelela ukuba nangaliphi na ixesha kukho abantu-7,000 kuphela MTurk.

Okokugqibela, ufanele wazi ukuba MTurk Luluntu nemithetho yayo kunye nezithethe (Mason and Suri 2012) . Ngendlela efanayo ukuba uza kuzama ukufumanisa malunga inkcubeko yelizwe apho babesiya ukuba ukubaleka imifuniselo yakho, kufuneka uzame ukufumana ngaphandle okuninzi malunga nenkcubeko kunye nesithethe Turkers (Salehi et al. 2015) . Ke, ufanele wazi ukuba Turkers iza uthetha ilinge lakho ukuba wenze into engafanelekanga okanye ukungaziphathi (Gray et al. 2016) .

MTurk yindlela kakhulu iluncedo ukuqesha nxaxheba kwi imifuniselo bakho, enoba lab-ngathi, ezifana Huber, Hill, and Lenz (2012) , okanye intsimi-ezifana kakhulu, ezifana Mason and Watts (2009) , Goldstein, McAfee, and Suri (2013) , Goldstein et al. (2014) , Horton and Zeckhauser (2016) , kunye Mao et al. (2016) .

• Yakha imveliso yakho (Section 4.5.1.3)

Ukuba ucinga uzama ukwenza imveliso yakho, mna ndingakucebisa ukuba ufunda iingcebiso owenziwa liqela MovieLens in Harper and Konstan (2015) . A ingqiqo engundoqo kumava abo kukuba ngeprojekthi nganye yimpumelelo kukho ezininzi, iintsilelo ezininzi. Ngokomzekelo, iqela MovieLens ngokusesikweni nezinye iimveliso ezifana GopherAnswers ezibe iintsilelo epheleleyo (Harper and Konstan 2015) . Omnye umzekelo we umphandi yokusilela ngexesha uzama ukwakha imveliso lilinge Edward Castronova ukuba ndakhele umdlalo online ekuthiwa Arden. Nangona $ 250,000 kwi mali, iprojekthi leyo flop (Baker 2008) . Iiprojekthi ezifana GopherAnswers kwaye Arden Ngelishwa xha ngaphezu iiprojekthi ezifana MovieLens. Okokugqibela, xa wathi bendingazi naziphi ezinye abaphandi ukuba Ndikwakhele iimveliso ukulingwa ngokuphindaphindiweyo apha iikhrayitheriya yam: 1) nxaxheba ukusebenzisa imveliso ngenxa yoko inika nabo (umzekelo, be nga hlawulwa kwaye bengengabo amavolontiya ekuncedeni inzululwazi) kunye 2) imveliso leyo isetyenzisiwe ngaphezu kwesinye zovavanyo eyahlukeneyo (oko kukuthi, hayi kwalingwa omnye amaxesha amaninzi ngamaqula nxaxheba ezahlukeneyo). Ukuba uyazi neminye imizekelo, nceda undazise.

• Iqabane kunye enamandla (Section 4.5.2)

Ndakhe ndeva ingcamango ikota yesangqa Pasteur xa kuxoxwa rhoqo kwiinkampani tech, kwaye kuyanceda siququzelele iinzame zophando at Google (Spector, Norvig, and Petrov 2012) .

Bond kunye nesifundo noogxa ' (2012) naye uzama ukubona isiphumo ezi unyango kwi abahlobo abo bamkela kubo. Ngenxa noyilo kwalingwa, ezi nokube kunzima ukubona somntu; abafundi abanomdla kufuneka ubone Bond et al. (2012) ukulungiselela ingxoxo ngokucokisekileyo ngakumbi. Le ukucikideka yinxalenye isithethe elide kumalinge kwinzululwazi yezopolitiko neenzame ukukhuthaza ukuvotela (Green and Gerber 2015) . Ezi eksperimenti GET-out-the-ivoti aqhelekile ngokuyinxenye ngenxa yokuba in ikota yesangqa Pasteur yakhe. Oko kukuthi, kukho abantu abaninzi bayakhuthazeka ukwandisa okuvota ukuvota kungenza kuba ukuziphatha umdla ukuvavanya iingcamango jikelele ngakumbi ukutshintsha ukuziphatha kunye nempembelelo lwentlalo.

Abanye abaphandi ngokunikezela iingcebiso kwanangokubaleka imifuniselo endle kunye nemibutho engamaqabane efana amaqela ezopolitiko, NGOs, kunye namashishini (Loewen, Rubenson, and Wantchekon 2010; List 2011; Gueron 2002) . Abanye baye benza iingcebiso malunga nendlela ubambiswano kunye imibutho nefuthe uyilo ophando (Green, Calfano, and Aronow 2014; King et al. 2007) . Partnership kusenokukhokelela imibuzo yeenqobo (Humphreys 2015; Nickerson and Hyde 2016) .

• Iingcebiso Design (Section 4.6)

Ukuba uzakuthatha ukwenza isicwangciso uhlalutyo phambi kokusebenza umfuniselo wakho, Mna ndicebisa ukuba uqala ngokufunda izikhokelo ingxelo. Liqabane (Consolidated Standard Reporting Izilingo) zikhokelo zaphuhliswa amayeza (Schulz et al. 2010) lwaze uphando kwezentlalo (Mayo-Wilson et al. 2013) . Iseti ezinxulumene yezikhokelo iye yaphuhliswa abahleli Journal of Experimental Science Political (Gerber et al. 2014) (jonga kwakhona Mutz and Pemantle (2015) kunye Gerber et al. (2015) ). Okokugqibela, izikhokelo ingxelo ziye zaphuhliswa bengqondo (Group 2008) , yaye bona Simmons, Nelson, and Simonsohn (2011) .

Ukuba ukwenza isicwangciso uhlalutyo kufuneka ucinge pre-nokubhalisa kuba-yobhaliso pre uya kwandisa ukuzithemba ukuba abanye kwiziphumo zakho. Ngokubhekele phaya, ukuba usebenza kunye neqabane, uya umda ukukwazi iqabane lakho ukutshintsha uhlalutyo emva ukubona iziphumo. -Yobhaliso Pre kusiba eqhelekileyo bengqondo (Nosek and Lakens 2014) , kwinzululwazi yezopolitiko (Humphreys, Sierra, and Windt 2013; Monogan 2013; Anderson 2013; Gelman 2013; Laitin 2013) , kunye Economics (Olken 2015) .

Ngexesha kusenziwa isicwangciso sakho pre-uhlalutyo kufuneka ulumkele ukuba abanye abaphandi kwakhona ukusebenzisa parameter kunye neendlela ezinxulumene ukuphucula ngqo isiphumo unyango kuqikelelwa, kwaye kukho ingxoxo ngale ndlela: Freedman (2008) , Lin (2013) , kunye Berk et al. (2013) ; bona Bloniarz et al. (2016) ukuze ufumane inkcazelo engakumbi.

Iingcebiso Design ingakumbi eksperimenti endle Intanethi lukwaveziwe Konstan and Chen (2007) kunye Chen and Konstan (2015) .

• Yenza data zero iindleko eziguquguqukayo (Section 4.6.1)

Kuba vetshe yemifuniselo MusicLab, bona Salganik, Dodds, and Watts (2006) , Salganik and Watts (2008) , Salganik and Watts (2009b) , Salganik and Watts (2009a) , kwaye Salganik (2007) . Vetshe winner-thabatha-zonke iimarike, bona Frank and Cook (1996) . Vetshe luck untangling nobuchule kakhulu ngokubanzi, bona Mauboussin (2012) , Watts (2012) , kunye Frank (2016) .

Kukho nenye indlela ukuphelisa iintlawulo nxaxheba ukuba abaphandi kufuneka basebenzise ngononophelo: emkhosini. Xa ezininzi imifuniselo endle online nxaxheba ngokusisiseko awathunyelwa imifuniselo kwaye zange babuyekezwe. Imizekelo yalo ndlela ziquka Restivo and van de Rijt ngayo (2012) ulingelo ukuzanelisa in Wikipedia and Bond kunye nogxa wakhe (2012) ukucikideka nokukhuthaza abantu ukuba bavote. Ezi experiments musa ngokwenene kuba zero iindleko eziguquguqukayo, kufuneka zero iindleko eziguquguqukayo ukuba abaphandi. Nangona ixabiso ezininzi zezi experimenti ezincinane kakhulu nxaxheba ngamnye, iindleko ezincinane eziwiswe inani elikhulu -nxaxheba eneji zikhula ngokukhawuleza. Abaphandi ezisebenza imifuniselo omkhulu online badla kugwebela ukubaluleka iziphumo ezincinane unyango uqikelelo ngokuthi ukuba ezi ziphumo ezincinane unokuba ebalulekileyo xa lisetyenziswa kubantu abaninzi. Abacinga ngqo uyasebenza kwiindleko ukuba abaphandi banyanzelise nxaxheba. Ukuba eksperimenti bakho kubangela abantu nesigidi esinye kwenkunkuma ngomzuzu omnye, ulingo engenabungozi kakhulu ukuba nawuphi na umntu othile, kodwa xa zizonke sele aluphele phantse emibini iminyaka ixesha.

Enye indlela ekudaleni zero yokuhlawula iindleko eziguquguqukayo ukuba nxaxheba kukuba ukusebenzisa i lotto, indlela yokuba iye kusetyenziswa kuphando uphando (Halpern et al. 2011) . Okokugqibela, kuba banzi ngokuyila umsebenzisi-bumyoli ukubona Toomim et al. (2011) .

• Endaweni, Lungisa, kwaye Ukunciphisa (Section 4.6.2)

Nazi iinkcazelo zantlandlolo ezintathu R, ukusuka Russell and Burch (1959) :

"Ukutshintshwa ithetha endaweni izilwanyana abazi abahlala aphezulu eziphathekayo insentient. Ukunciphisa kuthetha ukuncipha kwamanani ezilwanyana ezisetyenziselwa ukufumana ingcaciso kwisixa elinikiweyo kwaye ngocwangco. Ukuphuculwa ithetha nayiphi ekuncipheni kwezenzo nobuzaza iinkqubo nabuntu isicelo kwabo izilwanyana nangoku ukuba kusetyenziswa. "

ezintathu R sokuba ndiba musa ngaphezulu imigaqo yokuziphatha kuchaziwe kwiSahluko 6 Kunoko, uguqulelo kunatywa ngakumbi omnye loo migaqo-beneficence-ngokukodwa ukusetwa imifuniselo zabantu.

Xa kuqwalaselwa njengesifo beemvakalelo, kukho imiba emithathu non-ngokweenqobo ukugcina engqondweni xa nokutolika olu vavanyo. Okokuqala, akucaci ukuba iinkcukacha kanye kwalingwa ukudibanisa kumabango theoretical; ngamanye amazwi, kukho imibuzo malunga kokuyilwayo ngokusemthethweni. Akucaci ukuba sobalo ilizwi ezintle nezimbi wayelumkisa bonisa kakuhle imeko ngokweemvakalelo-nxaxheba kuba 1) akucaci ukuba amagama ngeposi abantu ibonisa kakuhle iimvakalelo zabo 2) akucaci ukuba ethile uhlalutyo luvo ubuchule ukuba abaphandi esetyenziswayo uyakwazi ukuqonda ngokuthembekileyo neemvakalelo (Beasley and Mason 2015; Panger 2016) . Ngamanye amazwi, ukuze kubekho umlinganiselo kakubi umqondiso namkhethe. Okwesibini, uyilo kunye uhlalutyo kwalingwa isixelela nto lowo abaninzi nefuthe (oko kukuthi, akukho uhlalutyo ukungafani ye iziphumo zonyango) kwaye mhlawumbi oko. Kulo mzekelo, abaphandi kunjalonje ulwazi malunga nxaxheba, kodwa bona ncakasana iphathwa njenge widgets kwi uhlalutyo. Okwesithathu, ubungakanani nefuthe kule ukucikideka Ndandimncinci kakhulu; umahluko phakathi iimeko unyango kunye nolawulo imalunga 1 1,000. Xa iphepha wabo, Kramer kunye nabalingane ukwenza ityala ukuba isiphumo esingaka kubalulekile kuba kumakhulu ezigidi zabantu ukufikelela Iindaba wabo Yondla suku ngalunye. Ngamanye amazwi, bexabana ukuba iziphumo ukuba encinane umntu ngamnye ukuba enkulu aggregate. Nokuba ngaba ukwamkela le ngxoxo, ke akukacaci ukuba isiphumo esingaka kubalulekile malunga umbuzo lwezenzululwazi ngokubanzi ngakumbi njengesifo ngokweemvakalelo. Kuba ngakumbi kwiimeko apho iziphumo ezincinci ezibalulekileyo ukubona Prentice and Miller (1992) .

NgokoMthetho R yokuqala (Replacement), ngokuthelekisa ukosuIeleka ilinge ngokweemvakalelo (Kramer, Guillory, and Hancock 2014) kunye ukosuIeleka ngokweemvakalelo ukucikideka yendalo (Coviello et al. 2014) inika izifundo ngokubanzi malunga urhwebo-offs abachaphazelekayo kunye nokusuka uBoyle imifuniselo zendalo (kunye nezinye iindlela ezifana ezithelekisekayo iinzame beko kumalinge data non-zonyango, funda iSahluko 2). Ukongeza iinzuzo yokuziphatha, ifaka i evela kwizifundo ezingezizo-zovavanyo yenza abaphandi ukufunda unyango ukuba benkxaso akakwazi ukuba asebenze. Ezi nzuzo ezisesikweni kwaye amalungiselelo beze ngeendleko, kunjalo. Experiment zendalo abaphandi babe nolawulo ngaphantsi phezu izinto ezifana ngokuqesha nxaxheba, engakhethiyo, kwakunye nohlobo unyango. Ngokomzekelo, omnye zokunciphisa kwemvula njengonyango kukuba bobabini oku kwandisa owakhayo kwaye unciphisa kelwa. Noko ke, isifundo zonyango, Kramer kunye nabalingane kukwazi ukufunda owakhayo kunye kelwa ngokuzimeleyo.

Indlela ethile esetyenziswa Coviello et al. (2014) waye kunatywa ngakumbi Coviello, Fowler, and Franceschetti (2014) . Kuba intshayelelo variables sisixhobo ukubona Angrist and Pischke (2009) (ngaphantsi formal) okanye Angrist, Imbens, and Rubin (1996) (ezicwangcisiweyo). Kuba uvavanyo beluthandabuza variables sisixhobo ukubona Deaton (2010) , kunye intshayelelo variables negalelo ngeempahla buthathaka (imvula sisixhobo buthathaka), funda Murray (2006) .

Okuninzi ngokubanzi, intshayelelo okulungileyo imifuniselo zendalo na Dunning (2012) , kunye Rosenbaum (2002) , Rosenbaum (2009) , kunye Shadish, Cook, and Campbell (2001) zinika iingcamango ezilungileyo ukuqikelela iziphumo nakuyo ngaphandle imifuniselo.

NgokoMthetho R yesibini (uCikizo), kukho lwenzululwazi kunye amalungiselelo urhwebo-ebakho xa ucinga ngokutshintsha uyilo njengesifo Emotional ukusuka ethintela izithuba nokukhuthaza izithuba. Umzekelo, kusenokuba kwimeko yokuba ukuphunyezwa sobuchule News Feed kwenza kube lula kakhulu ukwenza umfuniselo evalela kunokuba izithuba ngaphezu ulingelo kunye nokunyusa izithuba (qaphela ukuba umfuniselo ethintela izithuba lingaphunyezwa njenge umaleko kwi iphezulu kwinkqubo News Feed ngaphandle imfuneko utshintsho kwinkqubo engabonakaliyo). Ngokwenzululwazi, Noko ke, le ngcamango wathetha ngako ukucikideka akazange athi ngokucacileyo kuyilo omnye phezu komnye.

Ngelishwa, mna ndiqonda uphando kakhulu phambi malunga nokufaneleka usisizalwane evalela kunye nokunyusa umxholo kwi News Feed. Kwakhona, Andikubonanga uphando okuninzi ukucokisa unyango ukubenza akuyongozi; khutshelwa ngaphandle Jones and Feamster (2015) , nto leyo ingqalelo kwimeko lomlinganiselo ukuhluza Internet (ngesihloko I kuxubusha kwiSahluko 6 kubudlelwane ekufundweni elithile (Burnett and Feamster 2015; Narayanan and Zevenbergen 2015) ).

NgokoMthetho R lesithathu (Ukuncitshiswa), intshayelelo kohlalutyo amandla yemveli Cohen (1988) . covariates Pre-unyango kufakwa lokuyilwa kwaye uhlalutyo isigaba eksperimenti; Isahluko 4 Gerber and Green (2012) inika intshayelelo okulungileyo zombini ezi nkqubo, yaye Casella (2008) inika unyango olunzulu. Techniques asebenzisa olu lwazi pre-unyango kwi engakhethiyo kuthiwa ngokokuqhelekileyo mhlawumbi egcweleyo uyilo wophando okanye uyilo lomfuniselo nehlelwe (isigama ayisetyenziselwanga rhoqo noluntu ngokubanzi); ezi ndlela zinxulumene ngokunzulu ubuchule isampuli nehlelwe kuchaziwe kwiSahluko 3. Bona Higgins, Sävje, and Sekhon (2016) ngaphezulu ukusebenzisa le misebenzi yoyilo in imifuniselo enkulu. Covariates Pre-unyango nazo zibandakanywe uhlalutyo eqongeni. McKenzie (2012) uphonononga umahluko-in-iiyantlukwano indlela kuhlalutywa imifuniselo endle ngokweenkcukacha enkulu. Bona Carneiro, Lee, and Wilhelm (2016) ngaphezulu urhwebo-offs phakathi iindlela ezahlukeneyo ukwandisa okuhle kwiingqikelelo iziphumo zonyango. Okokugqibela, xa kusenziwa isigqibo sokuba ngaba ukuzama ukubandakanya covariates pre-unyango kuyilo okanye uhlalutyo kwisigaba (okanye zombini), kukho izinto ezimbalwa ingqalelo. Kwi isicwangciso apho abaphandi ufuna ukubonisa ukuba bengengabo "okuloba" (Humphreys, Sierra, and Windt 2013) , esebenzisa covariates pre-unyango naba lokuyilwa kunokuba luncedo (Higgins, Sävje, and Sekhon 2016) . Kwiimeko apho abathathi bafika ngokulandelelana, amalinge intsimi ingakumbi intanethi, esebenzisa ulwazi pre-unyango naba lokuyilwa kusenokuba nzima ngamalungiselelo, bona umzekelo Xie and Aurisset (2016) .

Kubalulekile ngokudibanisa kancinci incebiso kutheni umahluko-in-iiyantlukwano kunokuba gqitha kakuhle ngaphezu umahluko-in-ndlela. Iziphumo ezininzi kwikhompyutha sokunxaxha eliphezulu kakhulu (jonga umzekelo, Lewis and Rao (2015) kunye Lamb et al. (2015) ) kwaye uzinzile kakhulu ekuhambeni kwexesha. Kulo mzekelo, inqaku tshintsho liya kuba sokunxaxha kakhulu ezincinci, ukwandisa amandla uvavanyo manani. Esinye isizathu oku wasondela alisetyenziswa kakhulu kukuba phambi kweminyaka yedijithali akuzange eqhelekileyo ukuba iziphumo pre-unyango. Indlela ezininzi eziphathekayo ukucinga ngayo nomfanekiso umfuniselo ukulinganisa ukuba umthambo rhoqo ethile ebangela ukulahlekelwa ubunzima. Ukuba wenza umahluko-in-iindlela ndlela, uqikelelo wakho uya kuba Ukwahluka olufumaneka ukuguquguquka kwimilinganiselo abakhoyo. Ukuba wenza umahluko-in-umahluko ndlela, nangona kunjalo, ukuba ongenabuyambo eyenzeka ngokwendalo kwimilinganiselo yasuswa kwaye lula ukubona umahluko okubangelwa unyango.

Enye indlela ebalulekileyo ukunciphisa inani nxaxheba umfuniselo wakho ukuba ziqhube uphando lokujongisisa ngamandla, nto leyo Kramer noogxa ubunokuyenza ezisekelwe ubukhulu isiphumo ndazigcina kwalingwa eyindalo Coviello et al. (2014) okanye ngaphambili uphando non-zovavanyo yi Kramer (2012) (enyanisweni ezi zinto ekupheleni kwesi sahluko). Qaphela ukuba oku ukusebenzisa uhlalutyo igunya isuntswana esahlukileyo kuneso soqobo. Kweli phakade analog, ngokubanzi abaphandi wenza uhlalutyo amandla ukuqinisekisa ukuba ukufunda lincinane kakhulu na (oko kukuthi, phantsi-Bing). Ngoku, ke, abaphandi ukwenza uhlalutyo amandla ukuqinisekisa ukuba ukufunda sabo inkulu kakhulu (oko kukuthi, phezu-Bing).

Okokugqibela, ndaqiqa ngokudibanisa R wesine: Repurpose. Oko kukuthi, ukuba abaphandi bazifumana data zovavanyo ngaphezu kufuneka ukujongana umbuzo zentsusa uphando, kufuneka repurpose idata ukubuza imibuzo emitsha. Ngokomzekelo, cinga ukuba Kramer noogxa wasebenzisa umahluko-in-iiyantlukwano Estimator bafumana zihlanjululwe data ngaphezu ukuze kulungiswe umbuzo yabo yophando. Kunokuba ungayisebenzisi data kwi ngokupheleleyo, ababa ezifunde ubungakanani ukusebenza umsebenzi yangaphambi-unyango ibinzana ngokweemvakalelo. Kanye njengokuba Schultz et al. (2007) bafumanisa ukuba isiphumo unyango ezahlukeneyo ukukhanya inzima abasebenzisi, mhlawumbi imiphumo News Feed eyahlukileyo kubantu abasele sathanda imiyalezo ukonwaba (okanye lusizi). UkuFumanela kungakhokelela ekubeni "okuloba" (Humphreys, Sierra, and Windt 2013) kunye "p-kusiko" (Simmons, Nelson, and Simonsohn 2011) , kodwa ke ezi becala iyaziseka ngokudibanisa ingxelo enyanisekileyo (Simmons, Nelson, and Simonsohn 2011) , pre-yobhaliso (Humphreys, Sierra, and Windt 2013) , kunye neendlela umatshini yokufunda ezizama ukuphepha phezu-gingci.