faallo dheeraad ah

Qaybtan waxa loo qorsheeyay in lagu isticmaalo sida tixraac, halkii in la akhriyo sida sheeko.

• Hordhac (Qaybta 4.1)

Su'aalaha ku saabsan bixinta sababaha cilmi bulsho inta badan waa adag iyo sir ah. Waayo, hab aasaas in bixinta sababaha ku salaysan garaafyada sababaha, arki Pearl (2009) , iyo hab ku saleeysan natiijooyinka ka iman kara, arki Imbens and Rubin (2015) (oo qabsinka farsamo ee cutubkan). Wixii la barbardhigo u dhexeeya labadan waji, arki Morgan and Winship (2014) . Waayo, hab rasmi ah u qeexaya confounder a, arki VanderWeele and Shpitser (2013) .

In cutubka, waxaan abuuray waxa u muuqatay sida line a dhalaalaya u dhexeeya awoodda aan u leenahay in la sameeyo qiyaas sababaha ka xogta tijaabo iyo non-tijaabo. Xaqiiqadii, waxaan qabaa in kala duwanaansho waa blurrier. Tusaale ahaan, qof kasta oo aqbala in sigaarka wuxuu sababaa kansarka inkastoo aan waligood sameeyo tijaabo nasiibin gacanta ku ah in ciidamada dadka si ay sigaar. Waayo, daaweynta dhererka buugga fiican on qaadashada qiyaasta sababaha ka xogta aan tijaabo arki Rosenbaum (2002) , Rosenbaum (2009) , Shadish, Cook, and Campbell (2001) , iyo Dunning (2012) .

Cutubyada 1aad iyo 2aad ee Freedman, Pisani, and Purves (2007) waxay bixiyaan hordhac ah oo si cad u galay farqiga u dhexeeya tijaabo, tijaabo gacanta ku, iyo nasiibin tijaabo gacanta.

Manzi (2012) waxay bixisaa horudhac xiiso iyo akhrin gelin underpinnings falsafada iyo tirakoobka ee tijaabo nasiibin gacanta. Waxa kale oo ay bixisaa tusaalooyin nolosha dhabta xiiso xoogga tijaabo ee ganacsiga.

• Waa maxay tijaabo? (Qaybta 4.2)

Casella (2008) , Box, Hunter, and Hunter (2005) , Athey and Imbens (2016b) siiyaan isbarasho wanaagsan in dhinacyada tirakoobka of design iyo falanqaynta tijaabo. Dheeraad ah, waxaa jira daaweyn heer sare ah isticmaalka tijaabo ee beeraha badan oo kala duwan, dhaqaalaha (Bardsley et al. 2009) , Cilmiga (Willer and Walker 2007; Jackson and Cox 2013) , cilmi nafsiga (Aronson et al. 1989) , Cilmiga siyaasadda (Morton and Williams 2010) , iyo siyaasadda arrimaha bulshada (Glennerster and Takavarasha 2013) .

Muhiimadda qaybgale qorista (tusaale ahaan, sampling) inta badan waa-hoos mahad tijaabada cilmibaarista. Si kastaba ha ahaatee, haddii saamaynta daaweynta waa Nuurre Aw in dadweynaha, ka dibna sampling waa muhiim. Longford (1999) si cad dhibic this markuu u dooda cilmi fekereyso tijaabo sida sahan dadka la sampling qasay.Shacabka ah.

• Laba dhinac oo tijaabo: shaybaarka-field iyo analog-digital (Qaybta 4.3)

Laba boglaynta aan soo bandhigay dhexeeya shaybaarka oo beerta tijaabo waa yara fududeeyey. Dhab ahaantii, cilmi kale ayaa soo jeediyay typologies faahfaahsan oo dheeraad ah, in kuwa gaar ah in la kala saaro noocyada kala duwan ee tijaabo beerta (Harrison and List 2004; Charness, Gneezy, and Kuhn 2013) . Dheeraad ah, waxaa jira laba nooc oo kale oo tijaabo lagu sameeyaa by saynisyahano bulshada aan u hir geli boglaynta shaybaarka oo beerta:. Tijaabo baadhitaan iyo tijaabo bulshada tijaabo Survey waxaa tijaabo isticmaalaya kaabayaasha sahan jira iyo is barbar jawaabaha versions kale oo ka mid ah su'aalo isku mid ah (tijaabo sahan qaar ka mid ah waxaa lagu soo bandhigay in Cutubka 3aad); waayo badan oo ku saabsan tijaabooyinka baadhitaanka arki Mutz (2011) . tijaabo Social waa tijaabo meesha daweynta waa qaar ka mid ah siyaasadda bulshada in kaliya la fulin karo by dowladda. tijaabo Social ay si dhow u la xiriira barnaamijka qiimaynta. Wixii dheeraad ah oo ku saabsan tijaabooyinka siyaasadda, arki Orr (1998) , Glennerster and Takavarasha (2013) , iyo Heckman and Smith (1995) .

Tiro ka mid ah wargeysyada ayaa marka la barbar dhigo shaybaarka oo beerta tijaabo ee aan la taaban karin ee (Falk and Heckman 2009; Cialdini 2009) iyo in la eego natiijada tijaabo gaar ah ee cilmiga siyaasadda (Coppock and Green 2015) , dhaqaalaha (Levitt and List 2007a; Levitt and List 2007b; Camerer 2011; Al-Ubaydli and List 2013) iyo cilmi nafsiga (Mitchell 2012) . Jerit, Barabas, and Clifford (2013) waxay bixisaa design a cilmi fiican for barbar natiijooyinka shaybaarka oo beerta tijaabo.

Su'aalaha ku saabsan ka qayb-galayaasha beddelo dhaqankooda, sababtoo ah waxay ogyihiin in ay si dhow lagu arkay waxaa mararka qaarkood loo yaqaan saameynta baahida, iyo waxay la soo bartay cilmi nafsiga (Orne 1962) iyo dhaqaalaha (Zizzo 2009) . Inkasta oo inta badan xiriir la leh tijaabo shaybaarka, kuwaas oo arrimo isku mid ah iyo sidoo kale keeni karaan dhibaatoyin tijaabo beerta. Dhab ahaantii, saamaynta dalabka waxaa sidoo kale mararka qaarkood loo yaqaan saameynta Hawthorne, muddo in waxuu ka yimid tijaabo beer, si gaar ah ugu tijaabo Nuur caan ah oo bilawday 1924 at Works Hawthorne ee Shirkadda Western ku Electric (Adair 1984; Levitt and List 2011) . Saamaynta dalabka Labada iyo saamaynta Hawthorn ay si dhow u la xidhiidha fikradda ah cabbirka Waxyeelo hadlay Cutubka 2 (sidoo kale eeg Webb et al. (1966) ).

Taariikhda tijaabo berrinka ayaa lagu tilmaamay in dhaqaalaha (Levitt and List 2009) , cilmiga siyaasadda (Green and Gerber 2003; Druckman et al. 2006; Druckman and Lupia 2012) , cilmi nafsiga (Shadish 2002) , iyo siyaasadda guud (Shadish and Cook 2009) . Mid ka mid ah degaanka ee cilmiga bulshada halkaas oo tijaabo beerta si deg deg ah u noqday caan ah waa horumarinta caalamiga ah. Wixii dib u eegista wanaagsan oo shaqo ah in dhaqaalaha gudahood arki Banerjee and Duflo (2009) , iyo qiimayn muhiim arki Deaton (2010) . Wixii dib u eegista ah ee shaqada this ee cilmiga siyaasadda arki Humphreys and Weinstein (2009) . Ugu dambeyntii, caqabadaha anshaxa ku lug leh tijaabo berrinka ayaa la sahamiyey cilmiga siyaasadda (Humphreys 2015; Desposato 2016b) iyo dhaqaalaha horumarinta (Baele 2013) .

In cutubka, waxaan soo jeediyay in warbixin-daaweyn pre waxaa loo isticmaali karaa si loo hagaajiyo sax ah ee saamaynta daaweynta lagu qiyaasay, laakiin waxaa jira qaar ka mid ah dood ku saabsan habkan: Freedman (2008) , Lin (2013) , iyo Berk et al. (2013) ; arki Bloniarz et al. (2016) wixii macluumaad dheeraad ah.

• Ka gudubka tijaabo fudud (Qaybta 4.4)

Waxaan doortay in ay diiradda saaraan fikradaha saddex: ansax, heterogeneity of saamaynta daaweynta, iyo farsamooyin. fikradaha waxay leeyihiin magacyo kala duwan ee beeraha oo kala duwan. Tusaale ahaan, cilmi nafsiga u muuqdaan in ay ka gudbaan tijaabo sahlan oo diiradda saaraya dhex dhexaadinta iyo Moderators (Baron and Kenny 1986) . Fikradda ah ee dhex dhexaadinta waxaa lagu qabtay by waxa aan wici habab, xaqiiqduna waxay of Moderators waxaa lagu qabtay by waxa aan wici ansax dibadda ah (sida, natiijada tijaabo u noqon doono duwan haddii lagu ordo xaalado kala duwan) iyo heterogeneity of saamaynta daaweynta ( tusaale ahaan, waa saamaynta ugu weyn dadka qaarkood ka badan dadka kale).

Tijaabada ah ee Schultz et al. (2007) wuxuu muujinayaa sida aragtiyaha bulshada loo isticmaali karaa in la qorsheeyo waxqabadka wax ku ool ah. Wixii muran ah ka badan guud oo ku saabsan doorka aragtida in la dhigaayo waxqabadyada wax ku ool ah, arki Walton (2014) .

• Waqtiga (Qaybta 4.4.1)

Fikradaha of ansax gudaha iyo dibadda ah ayaa marka hore soo bandhigay in Campbell (1957) . Eeg Shadish, Cook, and Campbell (2001) , waayo, taariikh faahfaahsan oo dheeraad ah iyo caddaynta ah taxaddar of ansax gabagabo tirakoobka, ansaxnimada gudaha, dhisaan ansax, iyo saxsanaanta dibadda.

Waayo, muuqaalka guud ee arrimaha la xiriira ansax gabagabo tirakoobka in tijaabo arki Gerber and Green (2012) (waayo, aragti ah cilmiga bulshada) iyo Imbens and Rubin (2015) (waayo, aragti statistics). Qaar ka mid ah arrimaha ansax gabagabo tirakoobka si gaar ah uga kacayaan tijaabo beerta online waxaa ka mid ah arrimaha sida hababka computationally hufan ee abuuraya dura kalsoonida xogta ku tiirsan (Bakshy and Eckles 2013) .

ansaxnimada Gudaha noqon kartaa wax adag si loo hubiyo in tijaabo adag beerta. Arag, tusaale ahaan, Gerber and Green (2000) , Imai (2005) , iyo Gerber and Green (2005) doodo ku saabsan hirgelinta tijaabo adag beerta ku saabsan codbixinta. Kohavi et al. (2012) iyo Kohavi et al. (2013) waxay bixiyaan hordhac ah galay caqabadaha ansax bareeg in tijaabo beerta online.

Mid ka mid walaac weyn ee kuligeed gudaha waa dhibaato randomization. Hal si oo laga yaabo inay la ogaado dhibaatooyinka la randomization waa inay is barbar dhigaan kooxaha daaweynta iyo gacan ku haynta on sifooyin arki karo. Noocan ah waxaa loo yaqaan marka la barbardhigo jeeg dheelitirka ah. Eeg Hansen and Bowers (2008) waayo hab xisaabeed si la isugu dheellitiro jeegaga, oo bal arag Mutz and Pemantle (2015) , waayo walaac ku saabsan jeegagga dheelitirka. Tusaale ahaan, iyadoo la isticmaalayo miisaan hubi Allcott (2011) ayaa lagu ogaaday in qaar ka mid ah waxaa jira cadaymo muujinaya in randomization lama fulin sax ah in saddex ka mid ah tijaabo ah in qaar ka mid ah tijaabo OPower (fiiri Shaxda 2, goobaha 2, 6, iyo 8). Waayo, hababka kale, arki Imbens and Rubin (2015) , Cutubka 21.

walaac kale oo waaweyn oo la xiriira xaqiiqo ahaanta gudaha ku jira: 1) hal-madaxyo simanna hoggaansanaan la'aan, halkaas oo qof kasta oo aan in kooxda daaweynta si dhab ah heshay daawaynta, 2) laba madaxyo simanna hoggaansanaan la'aan, halkaas oo qof kasta oo aan in kooxda daaweynta helo daaweynta iyo qaar ka mid ah dadka in kooxda ay gacanta ku hesho daaweynta, 3) laastay, halkaas oo natiijo aan la qiyaasi ka qaybgalayaasha qaar ka mid ah, iyo 4) faragelin, meesha daaweynta daadinta ka qof oo ku nool xaalad daaweynta dadka xaaladoodu ay gacanta. Eeg Gerber and Green (2012) Cutubyada 5, 6, 7, 8 iyo waayo badan oo ku saabsan mid kasta oo arrimahan.

Wixii dheeraad ah oo ku ansax dhisida, arki Westen and Rosenthal (2003) , iyo ka badan oo ku saabsan ansax dhisida ee ilaha xogta weyn, Lazer (2015) iyo Cutubka 2aad ee kitaabkan lagu qoray.

Mid ka mida ansax dibadda waa goob halkaas oo faragelinta lagu tijaabiyo. Allcott (2015) waxay bixisaa daaweynta taxadir leh af iyo qoraal taaban karo ee eexda xulashada goobta. Arrintan ayaa sidoo kale ka hadlay in Deaton (2010) . Waxa intaa dheer in la hirgaliyay goobo badan, faragelinta Report Energy Home ayaa sidoo kale si madax banaan bartay kooxaha cilmi badan (tusaale ahaan, Ayres, Raseman, and Shih (2013) ).

• Heterogeneity of saamaynta daaweynta (Qaybta 4.4.2)

Wixii aragti guud fiican heterogeneity of saamaynta daaweynta in tijaabo beerta, ka eeg Cutubka 12 of Gerber and Green (2012) . Waayo, isbarasho in heterogeneity of saamaynta daaweynta in tijaabooyin caafimaad, arki Kent and Hayward (2007) , Longford (1999) , iyo Kravitz, Duan, and Braslow (2004) . Heterogeneity of saamaynta daaweynta guud ahaan diiradda on kala duwan oo ku salaysan sifooyinka-daaweyn pre. Haddii aad xiisaynayso in heterogeneity ku salaysan natiijooyinka-daweynta post, markaas approachs dheeraad ah oo adag loo baahan yahay sida stratification maamulaha (Frangakis and Rubin 2002) ; arki Page et al. (2015) in dib loo eego.

Cilmi badan oo la qiyaaso heterogeneity ee saamaynta daaweyn la isticmaalayo dhaca toosan, laakiin habab cusub ku tiirsan yihiin barashada mashiinka, tusaale ahaan Green and Kern (2012) , Imai and Ratkovic (2013) , Taddy et al. (2016) , iyo Athey and Imbens (2016a) .

Waxaa jira qaar ka mid shaki ku saabsan natiijooyinka of heterogeneity of saamaynta sababtoo ah dhibaatooyin la barbardhigo badan iyo "kalluumaysiga." Waxaa jira noocyo kala duwan oo habab tirakoobka oo kaa caawin kara walaaca cinwaanka ku saabsan isbarbardhigga badan (Fink, McConnell, and Vollmer 2014; List, Shaikh, and Xu 2016) . Mid ka mid ah hab si walaac ku saabsan "kalluumaysiga" waa pre-diiwaangelinta, kaas oo la sii caadi ah in cilmi nafsiga (Nosek and Lakens 2014) , cilmiga siyaasadda (Humphreys, Sierra, and Windt 2013; Monogan 2013; Anderson 2013; Gelman 2013; Laitin 2013) , iyo dhaqaalaha (Olken 2015) .

In daraasadda ee Costa and Kahn (2013) oo kaliya kala bar qoysaska ku jira tijaabo ka ahaayeen in la lala macluumaadka dadka ay awoodaan. Readers xiiseynaya faahfaahinta iyo dhibaatooyinka ay suurtagal tahay la falanqaynta this waa in tixraac warqada asalka ah.

• Farsamo (Qaybta 4.4.3)

Farsamo waa cajiib muhiim ah, laakiin waxa ay soo baxayso in aad u adag inaad wax ka barato. Research oo ku saabsan hab si dhow ula xidhiidha daraasadda ee dhex dhexaadinta ee cilmi nafsiga (laakiin sidoo kale arki VanderWeele (2009) is barbar dhig ah oo sax ah u dhexeeya labada fikradaha). Habab tirokoobka in hababka loo helo, sida hab horumartay ee Baron and Kenny (1986) , waa wax caadi ah. Nasiib darro, waxaa soo baxday in habka kuwa ku xiran qaar ka mid ah fikradaha xoog (Bullock, Green, and Ha 2010) iyo xanuunsado marka ay jiraan habab kala duwan, sida mid ka fili karto xaalado badan (Imai and Yamamoto 2013; VanderWeele and Vansteelandt 2014) . Imai et al. (2011) iyo Imai and Yamamoto (2013) waxay bixiyaan qaar ka mid ah hababka tirakoobka wanaagsanaaday. Dheeraad ah, VanderWeele (2015) waxay bixisaa daaweynta buug-dherer la tiro ka mid ah natiijooyinka muhiim ah, oo ay ku jiraan in hab ballaaran in falanqaynta dareen.

hab A gaar ah diiradda on tijaabo in isku dayaan in ay isku dubaridi hab ay si toos ah (tusaale ahaan, la siiyo ragga baxriyiinta ah vitamin C). Nasiib darro, in goobaha badan oo cilmiga bulshada waxaa inta badan waa hab badan oo ay adag tahay in la qorsheeyo daaweynta in la beddelo mid ka mid ah iyada oo aan la beddelo kuwa kale. Qaar ka mid ah hababka in experimentally habab ku beddelaan waxa lagu sharaxay Imai, Tingley, and Yamamoto (2013) , Ludwig, Kling, and Mullainathan (2011) , iyo Pirlott and MacKinnon (2016) .

Ugu dambeyntii, hab kale oo ay leeyihiin taariikh dheer in falsafadda ah sayniska sida uu tilmaamay Hedström and Ylikoski (2010) .

• Isticmaalka deegaanno jira (Section 4.5.1.1)

Wixii dheeraad ah oo ku saabsan isticmaalka cilmiga xiriirka iyo waxbarashada hubinta si loo cabbiro takoorka arki Pager (2007) .

• Dhis aad tijaabo gaar ah (Qaybta 4.5.1.2)

Habka ugu badan oo ay u qortaan ka qaybgalayaasha in ay tijaabo ah in aad ku dhistaan ​​waa Amazon Farsamada Turk (MTurk). Maxaa yeelay, MTurk ayadoo dhinacyada tijaabo-bixinta shaybaarka dhaqanka dadka si ay u dhamaystiraan hawlaha in aanay samayn lahaa cilmi free-badan ayaa durba bilaabay isticmaalaya Turkers (shaqaalaha on MTurk) sida ka qaybgalayaashu in tijaabo maadooyinka aadanaha taasoo keentay in si degdeg ah oo xog ururinta ka jaban tahay dhaqanka tijaabooyin shaybaadh ku-campus (Paolacci, Chandler, and Ipeirotis 2010; Horton, Rand, and Zeckhauser 2011; Mason and Suri 2012; Rand 2012; Berinsky, Huber, and Lenz 2012) .

xoog ugu weyn ee tijaabo la qayb-galayaasha ka MTurk qortaan waa saadka, ay u oggolaan inay qortaan cilmi ka qaybgalayaashu si deg deg ah iyo sida loogu baahan yahay. Halka tijaabo shaybaarka qaadan kartaa toddobaad si ay u ordaan oo tijaabo beerta qaadan kartaa bilood ilaa dhigay-up, tijaabo la qayb-galayaasha ka MTurk qorteen in la ordi karaan in maalmood. Tusaale ahaan, Berinsky, Huber, and Lenz (2012) ay awoodaan inay qortaan 400 maadooyinka hal maalin inay ka qayb qaataan tijaabo 8 daqiiqo ah. Dheeraad ah, ka qaybgalayaasha waxaa laga shaqaalaysiiyay kartaa shiidaa ujeeddo kasta (oo ay ku jiraan sahan iyo iskaashi ballaaran, sida hadlay Cutubyada 3aad iyo 5). Tani fududaato qorista macnaheedu waa in cilmi-ordi karaan isku xigxigta ee tijaabooyinka la xiriira oo isku xigta oo degdeg ah.

Ka hor inta qoranayaan ka qaybgalayaashu ka MTurk waayo tijaabo adiga kuu gaar ah, waxaa jira afar waxyaalood oo muhiim ah in la ogaado. First, cilmi badan oo ay leeyihiin a shaki non-gaarka ah ee ku lug leh tijaabo Turkers. Maxaa yeelay, shaki tani ma aha gaar ah, way adag tahay in looga hortago la caddayn. Si kastaba ha ahaatee, dhowr sano oo waxbarashada la isticmaalayo Turkers ka dib, waxaan hadda gabagabayn kartaa in shaki tani ma aha gaar ahaan loo baahan yahay. Waxaa jiray baaritaano badan oo is barbar ee tirakoobka of Turkers dadka kale iyo cilmiga badan barbar natiijada tijaabo la Turkers in natiijada ka dadka kale. Marka la eego in shuqulka oo dhan this, waxaan u malaynayaa in sida ugu fiican ee aad u malaynayso in ku saabsan waa in Turkers waa muunad dhaw raaxadeeda waxa macquul ah, inta badan sida ardayda, laakiin wax yar ka badan kala duwan (Berinsky, Huber, and Lenz 2012) . Sidaas darteed, sida ardayda yihiin dadka macquul ah oo qaar ka mid ah laakiin dhammaantiin ma aha cilmi tijaabo, Turkers yihiin dadka macquul ah oo qaar ka mid ah laakiin aan cilmi oo dhan. Haddii aad doonaysid in aad la Turkers shaqeeyaan, ka dibna waxa macno u akhri badan oo ka mid ah waxbarashada la barbar-dhigo iyo fahmo Gudigga ay.

Second, cilmi-yeelatay-ugu wanaagsan ee dhaqanka sii kordhaya for ansax gudaha ee tijaabo Turk, oo waa in aad wax ka bartaan oo raac fiican-dhaqanka (Horton, Rand, and Zeckhauser 2011; Mason and Suri 2012) . Tusaale ahaan, cilmi isticmaalaya Turkers waxaa lagu dhiirigelinayaa in ay isticmaalaan dadka baaraa si loo soo saaro ka-qaybgalayaasha hurdaan (Berinsky, Margolis, and Sances 2014; Berinsky, Margolis, and Sances 2016) (laakiin sidoo kale arki DJ Hauser and Schwarz (2015b) iyo DJ Hauser and Schwarz (2015a) ). Haddii aadan ka saar ka qaybgalayaasha hurdaan, markaas wax saameyn ah daaweynta lagu dhaqi karo by qaylada soo bandhigay ka qayb-galayaasha hurdaan, iyo ku dhaqanka tirada ka qayb galayaasha hurdaan noqon kartaa la taaban karo. In tijaabo ee Huber iyo asxaabtii (2012) oo ku saabsan 30% ka mid ah ka qaybgalayaasha ku guuldareystay baaraa dareenka aasaasiga ah. Dhibaatada kale ee caadiga ah la Turkers waa ka qaybgalayaasha non-gaab (Chandler et al. 2015) .

Saddexaad, qaraabo in ay qaabab kale oo tijaabo digital qaar ka mid ah, tijaabo MTurk ma qaaddo karo, Stewart et al. (2015) ku qiyaastay in mar kasta la siiyey waxaa jira oo kaliya oo ku saabsan 7,000 oo qof on MTurk.

Ugu dambayntii, waa in aad ogtahay in MTurk waa bulsho leh sharciyada iyo xeerarka u gaarka ah ee ay (Mason and Suri 2012) . In si la mid ah in aad isku dayi lahaa inaan si aad u ogaato oo ku saabsan dhaqanka ah wadan aad socdeen si ay u ordaan tijaabo aad, waa in aad isku daydo inaad ogaato wax badan oo ku saabsan dhaqanka iyo caadooyinka of Turkers (Salehi et al. 2015) . Oo, waa in aad ogtahay in Turkers la hadlaya doonaa saabsan tijaabo haddii aad wax ka qaban habboonayn ama akhlaaq daro (Gray et al. 2016) .

MTurk waa hab cajiib u sahlan inay qortaan ka qaybgalayaasha in ay tijaabo aad, in ay yihiin shaybaar-sida, sida Huber, Hill, and Lenz (2012) , ama beerta oo u eg in ka badan, sida Mason and Watts (2009) , Goldstein, McAfee, and Suri (2013) , Goldstein et al. (2014) , Horton and Zeckhauser (2016) , iyo Mao et al. (2016) .

• Dhis aad wax soo saarka gaar ah (Qaybta 4.5.1.3)

Haddii aad ka fekereyso inaad isku dayaya si ay u abuuraan aad wax soo saarka gaar ah, waxaan ku talinaynaa in aad ka akhrisan talo ka bixiyaan group MovieLens ee Harper and Konstan (2015) . aragti A muhiimka ah ee ay waayo-aragnimo waa in mashruuc kasta oo guul waxaa jira dad badan, Guuldarrooyin badan. Tusaale ahaan, koox MovieLens u bilaabay waxyaabaha kale sida GopherAnswers joogay oo dhamaystiran fashilaad (Harper and Konstan 2015) . Tusaale kale oo cilmibaadhis ku guuldareystay iyadoo isku dayaysa in ay dhisto wax soo saarka waa Edward Castronova ee isku day in la dhiso kulan online yeedhay Arden. Inkastoo $ 250,000 ee maalgelinta, mashruuca ahaa flop ah (Baker 2008) . Mashaariicda sida GopherAnswers iyo Arden nasiib daro waa wax badan ka badan mashaariicda sida MovieLens. Ugu dambeyntii, markii aan u sheegay in aan ma u ogtahay Eebe wax cilmi kale in uu si guul ah waxyaabaha dhisay tijaabo noqnoqda halkan yihiin shuruudaha aan: 1) ka qayb-galayaasha u isticmaalaan wax soo saarka sababtoo ah waxa ay iyaga siisaa (tusaale ahaan, ay ma bixiyo oo aan ahayn iskaa wax caawinaya sayniska) iyo 2) sheyga ayaa loo isticmaali jiray muddo ka badan hal tijaabo oo kala duwan (tusaale ahaan, ma la mid tijaabo marar badan la barkadaha qaybgale kala duwan). Haddii aad ogtahay of tusaalayaal kale, fadlan ii soo sheeg.

• Partner la awood ah (Qaybta 4.5.2)

Waxaan maqley fikradda ah Rubuca Pasteur ee inta badan ka wada hadleen at shirkadaha casriga ah, iyo waxa ay ka caawisaa abaabulo dadaallada cilmi-baarista ee Google (Spector, Norvig, and Petrov 2012) .

Bond iyo waxbarasho asxaabteena " (2012) ayaa sidoo kale isku dayaysa in la ogaado saamaynta daawaynta waxaa ku saaxiibbo kuwa iyaga soo dhoweeyey. Sababtoo ah design ee tijaabo ah, spillovers kuwaas oo ay adag tahay in si nadiif ah loo ogaado, akhristayaasha xiiso waa in ay arkaan Bond et al. (2012) si looga doodo badan oo dhammaystiran. Tijaabada Tani waa qayb ka mid ah dhaqanka dheer oo tijaabo ee cilmiga siyaasadda on dadaallada lagu dhiiri codeynta (Green and Gerber 2015) . Kuwani tijaabo karaa baxay-the-cod waa wax caadi ah in qayb maxaa yeelay waa in Pasteur ee Rubuca. Taasi waa, waxaa jira dad badan oo dano in la kordhiyo codbixinta oo cod noqon kartaa dabeecadda xiiso leh si ay u tijaabiso aragtiyaha guud oo ku saabsan isbeddelka dabeecadda iyo saamaynta bulshada.

Cilmi kale ayaa la siiyaa talooyin ku saabsan socda tijaabo beerta ururada lammaanahaaga sida xisbiyada siyaasadeed, NGOs, iyo ganacsiyada (Loewen, Rubenson, and Wantchekon 2010; List 2011; Gueron 2002) . Qaar kale ayaa u soo bandhigtay talo ku saabsan sida iskaashiga ururada saamayn karaa naqshado cilmi (Green, Calfano, and Aronow 2014; King et al. 2007) . Iskaashiga ayaa sidoo kale keeni kara in ay su'aalo anshaxa (Humphreys 2015; Nickerson and Hyde 2016) .

• Talo Design (Qaybta 4.6)

Haddii aad la abuuro qorshe falanqayn ka hor ordaya aad tijaabo, Waxaan u soo jeedin in aad bilowdo by reading xeerarka warbixinta. Consort The (Iskujira Standard Warbixinta of Maxkamadeyn) tilmaamaha lagu soosaaray in daawo (Schulz et al. 2010) iyo modified cilmi bulsho (Mayo-Wilson et al. 2013) . Set A la xiriira ee tilmaamaha la soo saaro by ah tifaftirayaasha ah Journal of tijaabooyin Cilmiga Siyaasadda (Gerber et al. 2014) (sidoo kale eeg Mutz and Pemantle (2015) iyo Gerber et al. (2015) ). Ugu dambeyntii, xeerarka warbixinta la soo saaray ee cilmi nafsiga (Group 2008) , iyo sidoo kale arki Simmons, Nelson, and Simonsohn (2011) .

Haddii aad la abuuro qorshe falanqayn in aad ka fiirsatid waxa pre-diiwaan sababtoo ah pre-diiwaan-kordhin doonaa kalsooni ah in dadka kale leeyihiin in natiijada. Dheeraad ah, haddii aad la Wadaage ka shaqeeya, waxay xadidi doonaa awoodda wehelkaaga in la beddelo falanqaynta kadib markii uu arkay natiijada. Pre-diiwaangelinta sii caadi ah in cilmi nafsiga (Nosek and Lakens 2014) , cilmiga siyaasadda (Humphreys, Sierra, and Windt 2013; Monogan 2013; Anderson 2013; Gelman 2013; Laitin 2013) , iyo dhaqaalaha (Olken 2015) .

Iyada oo la abuuro qorshe pre-falanqayn aad in aad la socoto in cilmi qaar kaloo isticmaali dhaca iyo hababka la xiriira si loo hagaajiyo sax ah ee saamaynta daaweynta lagu qiyaasay, oo waxaa jira qaar ka mid ah dood ku saabsan habkan: Freedman (2008) , Lin (2013) , iyo Berk et al. (2013) ; arki Bloniarz et al. (2016) wixii macluumaad dheeraad ah.

Talo Design si gaar ah loogu talagalay tijaabo beerta online waxaa sidoo kale lagu soo bandhigay in Konstan and Chen (2007) iyo Chen and Konstan (2015) .

• Abuur eber xogta qiimaha variable (Qaybta 4.6.1)

Wixii dheeraad ah oo ku saabsan tijaabooyinka MusicLab ah, arki Salganik, Dodds, and Watts (2006) , Salganik and Watts (2008) , Salganik and Watts (2009b) , Salganik and Watts (2009a) , iyo Salganik (2007) . Wixii dheeraad ah oo ku guuleystay-dhan qaataan suuqyada, arki Frank and Cook (1996) . Wixii dheeraad ah oo ku nasiib iyo xirfad untangling more guud ahaan, arki Mauboussin (2012) , Watts (2012) , iyo Frank (2016) .

Waxaa jira hab kale si ay baabi'inta lacagaha qaybgale in cilmi-waa in si taxadir leh u isticmaali: qasabka ah. In tijaabo badan beerta online ka qaybgalayaasha waxa asal ahaan qoray galay tijaabo oo marnaba magdhow. Tusaalooyinka habkani waxaa ka mid ah Restivo iyo van de Rijt ee (2012) tijaabo on abaal in Wikipedia oo Bond iyo saaxiibkiisii ​​ee (2012) tijaabo on dhiiri dadka in ay u codeeyaan. tijaabo Kuwani runtii ma leeyihiin eber kharashka variable, waxay leeyihiin eber kharashka variable in cilmi. Inkasta oo qiimaha badan oo tijaabo waa mid aad u yar in ay ka qaybgale kasta, kharash yar oo lagu soo rogay tirada aadka u badan ee ka qaybgalayaashu si deg deg ah ku dari kartaa ilaa. Cilmi socda tijaabo online weyn inta badan xaq kaga dhigi muhiimadda ay leedahay saamaynta daaweynta lagu qiyaasay yar leh in saamaynta yar noqon kartaa muhiim ah marka codsatay in ay dad badan oo. Isla fikirka saxda ah khuseysaa kharashka in cilmi-rogi on qayb-galayaasha. Haddii tijaabo aad keentaa hal milyan oo qof in ay baabba 'hal daqiiqo, tijaabo ah ma aha wax aad u dhib in qof gaar ah, laakiin in wadar ahaan labada lug waxa uu khasaariyey ku dhowaad laba sano mar.

Hab kale oo la abuurayo eber bixinta kharashka variable ka qaybgalayaasha waa in ay isticmaalaan bakhtiyaa, hab in ayaa sidoo kale loo isticmaalaa in cilmi baadhitaan (Halpern et al. 2011) . Ugu dambeyntii, waayo badan oo ku saabsan naqshadaynta xiiso leh user-waayo aragnimada arki Toomim et al. (2011) .

• Beddel, sifee, oo la yareeyo (Qaybta 4.6.2)

Halkan waxa ku jira qeexidda asalka ah ee saddex R ah, ka Russell and Burch (1959) :

"Bedelida ka dhigan bedelay ee ka dhawrsada nool xayawaanka sare ee wax insentient. Dhimista ka dhigan tahay hoos u tirada xoolaha loo isticmaalo si aad u hesho macluumaad ah ee qadar siin iyo sax. Hadba waxay ka dhigan tahay hoos u dhac kasta oo dhacdooyinka ama adkaanta nidaamka ka baxsan codsatay in kuwa xoolaha oo weli in la isticmaalo. "

saddex R ee aan soo jeedin ma shardi ah mabaadi'da anshaxa ku xusan cutubka 6. Saas ma aha, iyagu waa version ah oo dheeraad ah, nafaca mid ka mid ah kuwa mabaadi'da-Raxmaan-gaar ah, waayo, goob ka mid ah tijaabo aadanaha.

Marka la fiirinayo sadhaysan Qiirada, waxaa jira saddex arrimood oo aan anshax in maskaxda lagu hayo xilliga turjumidda tijaabiyaan this. First, ma cadda sida faahfaahinta dhabta ah ee tijaabo ah ku xidhmaan sheegashada teori; in si kale loo dhigo, waxa jira su'aalo ku saabsan ansax dhisida. Ma cadda in togan iyo kuwa taban erayga dacwadood dhab yihiin tilmaame fiican ee gobolka dareen ka qaybgalayaasha sababtoo ah 1) Ma cadda in erayada ay dadku boostada waa tilmaame wanaagsan ee dareenka iyo 2) Ma cadda in farsamada falanqaynta caadifo gaar ah in loo isticmaalo cilmi waa inay awoodaan inay kalsoonaan karo xawilaan dareenka (Beasley and Mason 2015; Panger 2016) . In si kale loo dhigo, waxaa laga yaabaa in qiyaas xun ee calan u janjeeraa dhinac. Second, ka design iyo falanqaynta tijaabo ah inoo sheegayaa waxba ku saabsan oo ahaa inta badan u saamaysay (ie, waxaa jira falanqaynta ma heterogeneity of saamaynta daaweynta) iyo waxa hab ay laga yaabaa. Xaaladdan oo kale, cilmi lahaa macluumaad badan oo ku saabsan ka qayb-galayaasha, laakiin ay muhiimad ahaan dhaqmo sida widgets Falanqaynta. Saddexaad, size saamayn ku tijaabiyaan this ahaa aad u yar, farqiga u dhexeeya xaaladaha daaweynta iyo gacanta qiyaastii 1 ee 1,000 erayo yahay. In ay warqad, Kramer iyo asxaabtii dhigi kiiska in saamayn of size this waa muhiim sababtoo ah boqolaal malaayiin dad ah helaan ay News Feed maalin kasta. In si kale loo dhigo, waxay ku doodayaan in xitaa saamaynta in yar yar yihiin, waayo, qof kasta oo ay yihiin weyn ee wadar ahaan labada lug. Xitaa haddii aad ahaayeen inay aqbalaan dooddan, weli ma cadda haddii saamayn of size this waa muhiim ah oo ku saabsan su'aasha guud cilmiyaysan oo ku saabsan cudurada faafa dareen. Wixii dheeraad ah oo ku saabsan xaaladaha halkaas oo saameyn yar waa muhiim arki Prentice and Miller (1992) .

In la eego R ugu horeysay (Bedelida), is barbar ee tijaabo Dareenka ku sadhaysan (Kramer, Guillory, and Hancock 2014) iyo tijaabo dabiiciga sadhaysan dareen ah (Coviello et al. 2014) Waxay bixisaa qaar ka mid ah casharada guud oo ku saabsan ganacsiga-offs ku lug leh ka dhaqaaqin tijaabo si tijaabo dabiiciga ah (iyo habab kale sida tallaabadaas isku day in qiyaasi tijaabo in xogta aan tijaabo, ka eeg Cutubka 2). Waxa intaa dheer in faa'iidooyinka anshaxa, bedesheen ka tijaabo ah si ay waxbarashada non-tijaabo sidoo kale awood cilmi inaad wax ka barato daaweynta inay Ma'aha awoodin in la geeyo. Kuwani faa'iidooyinka anshaxa iyo saadka yimaado kharash ah, si kastaba ha ahaatee. Iyada oo tijaabo dabiiciga ah cilmi hayaan yar waxyaabaha sida qorista ee ka qaybgalayaasha, randomization badan, iyo nooca daaweynta. Tusaale ahaan, mid ka mid xaddidaadda of roobab sida daawaynta a waa in ay labada kordhiyaa positivity iyo hoos u niyadjabka. In daraasadda tijaabo, si kastaba ha ahaatee, Kramer iyo asxaabtii ay awoodaan si ay ula qabsadaan positivity iyo dhaleeceynta madax bannaan.

Habka gaar ah loo isticmaalo by Coviello et al. (2014) waxaa la sii faah Coviello, Fowler, and Franceschetti (2014) . Waayo hordhac ah si ay doorsoomayaasha qalabka arki Angrist and Pischke (2009) (in ka yar rasmi ah) ama Angrist, Imbens, and Rubin (1996) (more rasmi ah). Waayo, qiimaynta a walaacsan doorsoomayaasha qalabka arki Deaton (2010) , iyo hordhac ah si ay doorsoomayaasha qalabka alaabtii daciif ah (roob waa aalad daciif ah), arki Murray (2006) .

More guud ahaan, sal-dhigida wanaagsan in tijaabo dabiiciga ah waa Dunning (2012) , iyo Rosenbaum (2002) , Rosenbaum (2009) , iyo Shadish, Cook, and Campbell (2001) waxay bixiyaan fikrado wanaagsan oo ku saabsan qiyaasidda saamaynta sababaha aan tijaabo.

In la eego R labaad (uhabaynta), waxaa jira sayniska iyo saadka ganacsiga-offs marka la beddelo design ee ku sadhaysan Dareenka ka xanibeen posts in hagaajinaysa posts. Tusaale ahaan, waxaa laga yaabaa in ay dhacdo in hirgelinta farsamo ee Feed News ee ka dhigayaa si xooga sahlan tahay in la sameeyo tijaabo ah la qaxo posts halkii tijaabo ah la kor posts (la soco in tijaabo ah oo la qaxo posts la fulin karin sida lakabka ah on top of nidaamka Feed News ah oo aan wax u baahan tahay isbedel nidaamka dahsoon). Cilmiga, si kastaba ha ahaatee, aragtida ah qabto by tijaabo ma si cad u soo jeedinaynaa in hal design ka badan tii kale waxay.

Nasiib darro, waxaan ka war cilmi ka hor la taaban karo oo ku saabsan mudnaanta qaraabo ah ka xanibeen oo hagaajinaysa content in Feed News ma ihi. Sidoo kale, Anigu ma arag cilmi badan oo ku saabsan hagaajinaayo daawaynta si aad u yar waxyeelada leh, mid ka mid ah marka laga reebo waa Jones and Feamster (2015) , kaas oo uu arko kiiska cabbirka of faafreebka Internet (mawduuc waxaan ka hadli Cutubka 6 ee xiriirka baadhitaan Encore ah (Burnett and Feamster 2015; Narayanan and Zevenbergen 2015) ).

In la eego R saddexaad (Dhimista), sal-dhigida wanaagsan in falanqaynta xoog dhaqanka waa Cohen (1988) . covariates daaweyn-Pre lagu dari karaa in marxalad design iyo falanqaynta tijaabo ah; Cutubka 4aad ee Gerber and Green (2012) waxay bixisaa horudhac wanaagsan in labada habab, iyo Casella (2008) waxay bixisaa daaweynta ka sii qoto-dheer. Tabaha in ay isticmaalaan macluumaadka ku-daweynta pre this ee randomization waxaa caadi ahaan loo yaqaan midkood xidhmey naqshado tijaabo ama naqshado tijaabo heerarka (Eray aan la isticmaalo si joogto ah guud ahaan bulshada); farsamooyinka kuwan waxaa si qoto dheer oo la xidhiidha farsamooyinka sampling heerarka kala hadlay Cutubka 3. Eeg Higgins, Sävje, and Sekhon (2016) muddo ka on isticmaalaya naqshado kuwaas oo tijaabo weyn. Covariates daaweyn-Pre sidoo kale lagu dari karaa in stage falanqaynta. McKenzie (2012) baadhayaa hab ay u falanqeeyo tijaabo beerta si faahfaahsan weyn farqiga-in-duwanaanshaha. Eeg Carneiro, Lee, and Wilhelm (2016) waayo badan oo ku saabsan ka ganacsiga-offs dhexeeya habab kala duwan si loo kordhiyo sax in qiyaasta of saamaynta daaweynta. Ugu dambeyntii, marka la go'aaminayo in ay isku dayaan in ay ka mid covariates-daaweyn pre at design ama falanqaynta marxaladda (ama labadaba), waxaa jira arrimo dhowr ah in la tixgeliyo. In meel halkaas oo cilmi doonayaa in aan tuso in aanay "kalluumaysiga" (Humphreys, Sierra, and Windt 2013) , iyadoo la isticmaalayo covariates-daaweyn pre-yada design waxay noqon kartaa mid waxtar leh (Higgins, Sävje, and Sekhon 2016) . In xaaladaha ay ka qaybgalayaashu sequentially yimaadaan, tijaabo beerta gaar ahaan online, iyadoo la isticmaalayo macluumaadka-daaweyn pre-yada design waxaa laga yaabaa in ay adag tahay Ma'aha, eeg tusaale ahaan Xie and Aurisset (2016) .

Waxaa xusid mudan isagoo intaa ku daray qayb ah dareen ku saabsan sababta farqiga-in-duwanaanshaha waxay noqon kartaa wax badan ka waxtar badan farqiga-in-hab. Natiijooyinka online badan waxay leeyihiin cadow aad u sarreeya (eeg tusaale ahaan, Lewis and Rao (2015) iyo Lamb et al. (2015) ) iyo waa yara deggan muddo. Xaaladdan oo kale, score isbedelka yeelan doontaa cadow xooga yar, sii kordhaya xoogga baaritaanka tirakoobka. Hal sabab this soo dhawaaday aan marar badan loo isticmaalo waa in ka hor da'da digital waxa aan ahaa caadi ah in ay leeyihiin natiijooyinka-daaweyn pre. Hab more la taaban karo in aad ka fikirto waxa ku saabsan yahay in la qiyaaso tijaabo ah si loo qiyaaso in a sameynta jir dhiska gaar keena miisaanka oo yaraada. Haddii aad hab farqiga-in-hab, aad qiyaasta yeelan doonaan tirayn ka timaada tirayn ee miisaanka ee dadweynaha. Haddii aad hab farqiga-in-farqiga, si kastaba ha ahaatee, in kala duwanaansho si dabiici ah ka dhaca miisaanka uu saaro oo aad si fudud lagu ogaan karaa farqiga uu keeno daweynta.

Mid ka mid ah habka muhiim ah in la yareeyo tirada ka-qaybgalayaasha in aad tijaabo waa si ay u qabtaan falanqaynta xoog ah, oo Kramer iyo asxaabtii samayn karay oo ku salaysan tirada saamayn dhawray tijaabo dabiiciga ah oo ay Coviello et al. (2014) ama cilmi aan tijaabo horaantii by Kramer (2012) (dhab ahaantii waa kuwan hawlaha dhamaadka cutubkan). Fiiro u yeelo in la isticmaalo this falanqaynta xoog ka yara duwan tahay ka caadiga ah. In da'da analog ah, cilmi-guud sameeyey falanqaynta xoog si loo hubiyo in ay daraasadda ma ahaa mid aad u yar (ie, hoos-shaqeeya). Hadda, si kastaba ha ahaatee, waa in cilmi-samayn falanqaynta xoog si loo hubiyo in ay daraasadda ma aha mid aad u weyn (ie, in ka badan-powered).

Ugu dambeyntii, Waxaan ka fiirsaday ku daray R afaraad: repurpose. Taasi waa, haddii cilmi-isku arkaan macluumaadka dheeraad ah tijaabo badan ay u baahan yihiin in wax laga qabto ay su'aal cilmi asalka ah, waa in ay repurpose xogta si aad u weydiiso su'aalo cusub. Tusaale ahaan, qiyaasi in Kramer iyo asxaabtii uu isticmaalo Xisaabiye ah farqiga-in-duwanaanshaha iyo isu helay xogta ka badan ee loo baahan yahay si wax looga qabto su'aal cilmi ay. Halkii aan la isticmaalayo xogta ilaa xad buuxda, waxay ku bartay yaabaa baaxadda saamaynta sida shaqo ah si ay u hor-daweynta iyo qiiro. Sida Schultz et al. (2007) ayaa lagu ogaaday in saamaynta daaweynta ahaa kala duwan ee isticmaala iftiin weyn oo culus, laga yaabee saamaynta Feed News waxay ahaayeen kala duwan oo loogu talo galay dadka hore u muuqdaan in boostada fariimaha ku faraxsan yahay (ama murugo). Repurposing ay keeni karaan "kalluumaysiga" (Humphreys, Sierra, and Windt 2013) iyo "p-jabsiga" (Simmons, Nelson, and Simonsohn 2011) , laakiin inta badan waxaa addressable la isku-dar ah warbixinta daacad ah (Simmons, Nelson, and Simonsohn 2011) , pre-diiwaangelinta (Humphreys, Sierra, and Windt 2013) , iyo habab waxbarasho mashiinka in isku dayaan in ay ka fogaadaan in ka badan-habboona.