hlalosang ka ho eketsehileng

Karolong ena e reretsoe ho sebelisoa e le ya referense eo e neng e e-na le ho bala e le phetoang.

• Selelekela (Karolo 4.1)

Na le lipotso mabapi causality a etsa lipatlisiso tsa phedisano hangata rarahaneng le rarahaneng. Bakeng sa katamelo moboko ho causality thehiloeng dikerafo e bakang, sheba Pearl (2009) , le bakeng sa katamelo moboko thehiloeng diphetho le bokhoni bo bona Imbens and Rubin (2015) (le sehlomathiso botekgeniki khaolong ena). Etsoe papiso e pakeng tsa tsena tse peli tse le atamela, sheba Morgan and Winship (2014) . Bakeng sa katamelo e hlophisitsweng ya ho ho hlalosang e confounder, sheba VanderWeele and Shpitser (2013) .

Khaolong eo, ke bōpile seo bonahala joaloka mola khanyang pakeng tsa bokhoni ba rōna ba ho etsa likhakanyo tsa e bakang ho tloha ya data etsa liteko le bao e seng teko. Ha e le hantle, ke nahana hore phapang ke blurrier. Ka mohlala, mong le e mong o amohela hore ho tsuba ho etsa hore tsoaroa ke kankere le hoja re e-s'o entse ea lotho laoloa teko eo qobelle batho ho tsuba. Etsoe o babatsehang liphekolo buka bolelele ho etsa likhakanyo tsa e bakang ho tloha ya data bao e seng liteko bona Rosenbaum (2002) , Rosenbaum (2009) , Shadish, Cook, and Campbell (2001) , le Dunning (2012) .

Khaolo ea 1 le ea 2 ea Freedman, Pisani, and Purves (2007) fana ka kenyelletso e hlakileng ho kena phapang pakeng tsa liteko, liteko laoloa,' me ea lotho liteko laoloa.

Manzi (2012) e fana ka tsela e hlollang le le balwang qala ho sebelisoa underpinnings tsa filosofi le dipalopalo tsa ea lotho liteko laoloa. E boetse e fana ka mehlala ea sebele e thahasellisang-lefatše oa matla a tekeletso likhoebong tsa bona.

• Tse liteko? (Karolo 4.2)

Casella (2008) , Box, Hunter, and Hunter (2005) , Athey and Imbens (2016b) fana tumeliso molemo likarolong dipalopalo boqapi etsa liteko le ho hlahloba. Ho ekelletsa moo, ho na le liphekolo tse babatsehang oa tšebeliso ea liteko masimong a mangata a fapaneng: moruo (Bardsley et al. 2009) , Kahisano (Willer and Walker 2007; Jackson and Cox 2013) , kelello (Aronson et al. 1989) , Saense litabeng tsa lipolotiki. (Morton and Williams 2010) , 'me leano la' muso tsoang sechabeng (Glennerster and Takavarasha 2013) .

Bohlokoa ba morupeluoa ho thaotha (mohlala, mehlala) Hangata tlas'a-ananela a etsa lipatlisiso liteko. Leha ho le joalo, haeba phello ea phekolo eo ke heterogeneous le baahi ba bangata, ka nako eo mohlala o mahlonoko. Longford (1999) le etsa ntlha ena ka ho hlaka ha a Babuelli bakeng bafuputsi nahana ho liteko tsa joalokaha phuputso e entsoeng le baahi ba nang le mohlala haphazard.

• 'Meli ho kgephoha ha liteko: laboratori-ea tšimo' me analoge-dijethale (Karolo 4.3)

The dichotomy hore ke hlahisa pakeng tsa laboratori le tšimo liteko ke a batla tsela e bonolo. Ha e le hantle, bafuputsi ba bang ba sisintsweng typologies qaqileng haholoanyane, e ba itseng hore a arole ba mefuta e sa tšoaneng ea liteko tšimo (Harrison and List 2004; Charness, Gneezy, and Kuhn 2013) . Ho ekelletsa moo, ho na le mefuta e meng e 'meli ea liteko e etsoang ke bo-rasaense ba tsoang sechabeng hore u se ke ua lumellane ka makhethe tsa limakasine ha nehelanoa ka eena laboratori le tšimo dichotomy:. Liteko phuputso e entsoeng le ho liteko tsa phedisano Survey liteko tse liteko tsa ho sebelisa tsa motheo tsa liphuputso jwale le ba bapisa likarabo ho liphetolelo mefuta e meng ea lipotso tse tšoanang (ba bang ba liteko phuputso e entsoeng tse hlahisoang Khaolong ea 3); etsoe eketsehileng mabapi le liteko tsa phuputso e entsoeng bona Mutz (2011) . liteko Social ke liteko moo kalafo e ba bang ba leano la tsa phedisano hore ka feela ho sebediswa ke puso ea. Liteko Social ba amanang haufi le Lenaneo la lekolane. Etsoe eketsehileng mabapi le liteko tsa leano la, sheba Orr (1998) , Glennerster and Takavarasha (2013) , le Heckman and Smith (1995) .

Maloa ba lipampiri tse di bapisoa laboratori le tšimo liteko bo inahaneloang (Falk and Heckman 2009; Cialdini 2009) le ho latela diphetho tsa liteko tse tobileng tseo tsa saense litabeng tsa lipolotiki. (Coppock and Green 2015) , moruo (Levitt and List 2007a; Levitt and List 2007b; Camerer 2011; Al-Ubaydli and List 2013) le kelello (Mitchell 2012) . Jerit, Barabas, and Clifford (2013) e fana ka e ntle etsa lipatlisiso moetso bapisa tlisoang ke ho laboratori le tšimo liteko.

Dingongoreho mabapi barupeluoa ho fetola boitšoaro ba bona hobane ba tseba hore ba ntseng ba haufi-ufi hlokomela ka linako tse ling bitsoa tlhokeho liphello, 'me ba' nile ba ithuta a kelello (Orne 1962) le moruo (Zizzo 2009) . Le hoja boholo ba amahanngoa le liteko laboratoring ba, litaba tsena tšoanang ka baka mathata bakeng sa liteko tšimo hammoho. Ha e le hantle, ya tlhokeho liphello ba boetse ba ka linako tse ling bitsoa litla-Hawthorne, nako e derives ho tswa teko tšimo, ka ho khetheha e tummeng liteko khanya eo o ile a qala ka 1924 nakong ea Hawthorne Mesebetsi ea Bophirimela Electric Company (Adair 1984; Levitt and List 2011) . Ka bobeli liphello tlhokeho le liphello Hawthorn ba haufi-ufi amanang le khopolo ea ho lekanya arabelang tšohloa Khaolong ea 2 (bona le Webb et al. (1966) ).

Historing ea liteko tšimo e se e hlalosoa ka moruo (Levitt and List 2009) , saense litabeng tsa lipolotiki. (Green and Gerber 2003; Druckman et al. 2006; Druckman and Lupia 2012) , kelello (Shadish 2002) , 'me leano la' muso setjhaba (Shadish and Cook 2009) . sebakeng sa e mong oa saense ea bitsoang tsoang sechabeng moo liteko tšimo ka potlako ea e-ba motho ea hlahelletseng ke tsoelo-pele ea machaba. Bakeng sa ho lekolwa botjha matle ka hore mosebetsi matsatsing a moruo bona Banerjee and Duflo (2009) , le bakeng sa thuto tekolo tse mahlonoko tseo ho bona Deaton (2010) . Bakeng sa tlhahlobo ea mosebetsi ona haholo saenseng theha mokhatlo oa lipolotiki bona Humphreys and Weinstein (2009) . Qetellong, diphepetso boitshwaro amehang le liteko tšimo 'nile tsa utolola tsa saense litabeng tsa lipolotiki. (Humphreys 2015; Desposato 2016b) le moruo ntshetsopele (Baele 2013) .

Khaolong eo, ke etsa tlhahiso ea hore pele ho kalafo u fumane boitsebiso bo ka sebelisoa ho ntlafatsa sebetsa ka ho nepahetseng ea liphello hakanngoa kalafo, empa ho na le ba bang ba khang ka mokhoa ona: Freedman (2008) , Lin (2013) , le Berk et al. (2013) ; sheba Bloniarz et al. (2016) bakeng sa tlhahisoleseding e eketsehileng.

• Fallela ka nģ'ane ho liteko tsa le bonolo hakana (Karolo 4.4)

Ke khethile ho tsepamisa kelello bopa dikgopolo tse tharo: tumello, heterogeneity oa liphello kalafo, le likeletso. bopa dikgopolo bana ba mabitso a fapaneng masimong a fapaneng. Ka mohlala, litsebi tsa kelello le tšekamelo ea ho fallela ka nģ'ane ho liteko tsa le bonolo hakana ke ho lebisa tlhokomelo mediators le baokameli (Baron and Kenny 1986) . Khopolo ea mediators e haptjoa ke seo ke bitsa mekgwa, 'me khopolo ea baokameli e haptjoa ke seo ke bitsa Link tumello (mohlala, na liphello tsa teko ea ho fapaneng haeba e ne e mathela maemo a fapaneng) le heterogeneity oa liphello kalafo ( mohlala, ke liphello tse khōloanyane bakeng sa batho ba bang ba ho feta batho ba bang).

The teko ea Schultz et al. (2007) e bontša ka moo likhopolo tse tsoang sechabeng e ka sebediswa ho meralo tsereletsi atlehang. Bakeng sa ho pheha khang eketsehileng kakaretso ka seabe sa khopolo a qapa tsereletsi atlehang, bona Walton (2014) .

• Tumello (Karolo 4.4.1)

Dikgopolo tsa tumello ya kahare le kantle li ne li ea pele hlahiswa Campbell (1957) . Sheba Shadish, Cook, and Campbell (2001) bakeng sa histori ea qaqileng haholoanyane' me e le elaboration hloko dipalopalo buang sephethong tumello, bonnete hare, hahe tumello, 'me tumello kantle.

Etsoe kakaretso ea litaba tse amanang le dipalopalo buang sephethong tumello a Liteko bona Gerber and Green (2012) (etsoe ka tsela e saense ea bitsoang tsoang sechabeng) le Imbens and Rubin (2015) (etsoe ka tsela e dipalopalo). Limakasine tsa dipalopalo buang sephethong tumello tse hlahang ka ho toba a Inthaneteng liteko tšimo kenyeletsa litaba tse kang mekhoa e computationally sebetsang hantle bōpa dikgao kholiseho ea ka tlhahisoleseding e itšetlehileng ka (Bakshy and Eckles 2013) .

Hare tumello ho ka ba thata ho etsa bonnete ba a liteko tse rarahaneng tšimo. Sheba mohlala, Gerber and Green (2000) , Imai (2005) , le Gerber and Green (2005) bakeng sa khang ka ho kenngwa tshebetsong ha e rarahaneng ea tšimo teko mabapi vouta. Kohavi et al. (2012) le ho Kohavi et al. (2013) fana ka kenyelletso ho kena mathata a karohano tumello a Inthaneteng liteko tšimo.

kameho ea seng se seholo le tumello hare ke mathata le randomization. Tsela e 'ngoe ho nang le monyetla lemoha mathata a amanang le randomization ke ho bapisa lihlopha tse kalafo le taolo mabapi le mekhoa e bonoang. Mofuta ona oa ho bapisa e bitsoa ho leka-lekana kokobetse. Sheba Hansen and Bowers (2008) bakeng sa katamelo dipalopalo ho leka-lekanya batlana, 'me le bone Mutz and Pemantle (2015) bakeng sa dingongoreho mabapi batlana ho leka-lekana. Ka mohlala, ho sebelisa teka-tekano e hlahloba Allcott (2011) fumane hore ho na le ba bang ba ka bopaki ba hore randomization o ne a sa kenngwa tshebetsong ka nepo a tharo tsa liteko tse ling tsa liteko OPower (sheba Lethathamo 2; liwebsaete 2, 6 'me tse 8). Etsoe le atamela tse ling, sheba Imbens and Rubin (2015) , Khaolong ea 21.

dingongoreho tse ling tse kholo tse amanang le bonnete hare ke: 1) mong-ka lehlakoreng bao e seng ho latelwa, moo hase batho bohle ba leng sehlopheng kalafo ha e le hantle a fumana phekolo eo, 2) tse pedi lehlakoreng bao e seng ho latelwa, moo hase batho bohle ba leng sehlopheng kalafo fumaneng phekolo ea mofuta 'me ba bang batho ba sehlopha se laola liketso fumana kalafo, 3) attrition, moo diphetho tse sa lekanyetsoa ho ba bang barupeluoa, 'me 4) kgokahano, moo kalafo ea spills holim' ho batho ba boemong bo sa ho phekoloa le batho ka boemo ba laoloa ke eena. Sheba Gerber and Green (2012) Khaolo ea 5, 6, 7 'me tse 8 ho nts'etsa pele ka' ngoe ea litaba tsena.

Etsoe eketsehileng mabapi le ho betla tsela tumello, sheba Westen and Rosenthal (2003) , le bakeng sa eketsehileng mabapi le ho betla tsela tumello a mehloling khōlō ya data, Lazer (2015) le Khaolo ea 2 ea buka ena.

Tšobotsi e 'ngoe ea amoheleha kapa Link ke bophirima moo ho kenella e e lekoa. Allcott (2015) e fana ka hlokolosi mogopolofela le bonahetseng kalafo ea setsheng sa kgetho leeme. Makasineng ena e boetse e tšohloa Deaton (2010) . Ho phaella ho ntse replicated a liwebsaete tse ngata, ho kenella ha Home Energy Report e boetse e le nosi ithuta ka lihlopha tsa multiple etsa lipatlisiso (mohlala, Ayres, Raseman, and Shih (2013) ).

• Heterogeneity oa liphello kalafo (Karolo 4.4.2)

Etsoe thutohohle babatsehang oa heterogeneity oa liphello phekolong ea liteko tšimo, sheba Khaolo ea 12 of Gerber and Green (2012) . Etsoe tumeliso ho heterogeneity oa liphello kalafo litekong tsa bongaka, sheba Kent and Hayward (2007) , Longford (1999) , le Kravitz, Duan, and Braslow (2004) . Heterogeneity oa liphello kalafo ka kakaretso ho ameha ka ho se tšoane thehiloeng litsobotsi pele ho kalafo. Haeba u thahasella ho heterogeneity thehiloeng diphetho poso e mpe, joale approachs rarahaneng haholoanyane ho hlokahala kang sehloohong stratification (Frangakis and Rubin 2002) ; sheba Page et al. (2015) bakeng sa tekolo botjha e.

Bafuputsi ba bangata ba hakanya ea heterogeneity oa liphello kalafo sebelisa guttate bofokoli, empa mekhoa e ncha itšetleha ka ithuta mochini, ho etsa mohlala Green and Kern (2012) , Imai and Ratkovic (2013) , Taddy et al. (2016) , le Athey and Imbens (2016a) .

Ho na le ke ba bang ba belaella mabapi fumanoeng ke tsa heterogeneity oa liphello lebaka la mathata a multiple papiso le "ho tšoasa litlhapi." Ho na le mefuta e fapaneng ya le atamela dipalopalo e ka re thusang aterese dingongoreho mabapi papiso multiple (Fink, McConnell, and Vollmer 2014; List, Shaikh, and Xu 2016) . Katamelo e mong ho dingongoreho mabapi "ho tšoasa litlhapi" ke pele ho re ngolisa ka molao, e leng se tšoanelang ka ho eketsehileng tloaelehileng le sebelisoang malapeng a kelello (Nosek and Lakens 2014) , saense litabeng tsa lipolotiki. (Humphreys, Sierra, and Windt 2013; Monogan 2013; Anderson 2013; Gelman 2013; Laitin 2013) , 'me moruo (Olken 2015) .

Ho ithuta Costa and Kahn (2013) feela ka bang halofo ea malapa ka teko ba ne ba ka khona ho amahanngoa le ho fumana boitsebiso bo sa babapatsi. Babali ba thahasella ka makolopetso le mathata khoneha ho Analysis sena ho lokela ho bua ka pampiri pele.

• Mekgwa (Karolo 4.4.3)

Mekgwa ke Hoa makatsa hore ebe ea bohlokoa, empa ba retelehela ho ba thata haholo ho ithuta. Research mabapi mekgwa amana haufi le ho ithuta mediators a kelello (empa sheba le VanderWeele (2009) bakeng sa papiso e tobileng pakeng tsa mehopolo eo e 'meli). Le atamela dipalopalo ho hlahisa mekgwa e fumanang, e kang atamela ntshetswa pele a Baron and Kenny (1986) , ke lintho tse tloaelehileng haholo. Ka bomalimabe, ho fellang kateng le hore mekgwatshebetso ya tseo itšetleha ka ba bang likhopolong matla (Bullock, Green, and Ha 2010) le utloa bohloko ha ho na mekgwa multiple, joalokaha motho a ka lebella maemo a mangata a (Imai and Yamamoto 2013; VanderWeele and Vansteelandt 2014) . Imai et al. (2011) le ho Imai and Yamamoto (2013) fana ba bang ba e ntlafalitsoeng mekgwa dipalopalo. Ho ekelletsa moo, VanderWeele (2015) e fana ka buka-bolelele kalafi ka maloa ea liphetho tsa bohlokoa haholo, ho akarelletsa le katamelo e akaretsang ho Analysis nahanela.

A katamelo arohaneng e bua haholo ka liteko tseo leka ho laola mochine ea ho toba (mohlala, ho fana basesisi vithamine C). Ka bomalimabe, a di-setting tse ngata tsa saense ea bitsoang tsoang sechabeng ho na le hangata multiple mekgwa 'me e ke ho le thata ho meralo ea phekolo e le hore fetoha ea se nang fetola ba bang. Ba bang ba mekgwa ya go experimentally mekgwa fetolang li hlalositsoe ka Imai, Tingley, and Yamamoto (2013) , Ludwig, Kling, and Mullainathan (2011) , le Pirlott and MacKinnon (2016) .

Qetellong, mekgwa le bona ba na le histori e telele ka ho akha filosofi oa saense joalokaha ho hlalositsoe ke Hedström and Ylikoski (2010) .

• U sebelisa dibaka teng (Karolo 4.5.1.1)

Bakeng sa tabeng ea ho sebelisa lithuto tsa lefapheng la ngollano le lithuto tsa boruni ho lekanya kgethollo bona Pager (2007) .

• Haha teko ya hao (Karolo 4.5.1.2)

Tsela e tloaelehileng ho e tsoma badiri barupeluoa ho liteko tseo u hahe ke Amazon Mechanical Turk (MTurk). Hobane MTurk mimics likarolo tse ling tsa liteko lefang tsa moetlo laboratori batho ho tlatsa mesebetsi eo ba ne ba sa etsetsa bafuputsi mahala-ba bangata ba se a ntse a qalile ho sebelisa Turkers (basebetsi mabapi le MTurk) joalokaha barupeluoa a bafo botho liteko fella ka lebelo le pokello theko e tlaase ya data ho feta tsa moetlo on-khamphaseng liteko tsa laboratori (Paolacci, Chandler, and Ipeirotis 2010; Horton, Rand, and Zeckhauser 2011; Mason and Suri 2012; Rand 2012; Berinsky, Huber, and Lenz 2012) .

Le matla kholo ka ho fetisisa ea liteko tsa le barupeluoa le thaotha tsoang MTurk ke logistical: ba lumella bafuputsi ho e tsoma badiri barupeluoa ka potlako le ka ha ho hlokahala. Athe liteko laboratoring ba ka nka dibeke tse ho matha le liteko tšimo ka nka likhoeli hore beha-up liteko le barupeluoa le thaotha tsoang MTurk ka matha ka matsatsi. Ka mohlala, Berinsky, Huber, and Lenz (2012) ba ile ba khona ho e tsoma badiri bafo 400 ka letsatsi le le leng lesoha o re ho ba le seabo sa teko 8 motsotso. Ho ekelletsa moo, barupeluoa tsena ka le thaotha ka morero leha e le efe (ho akarelletsa le liphuputso le tshebedisano boima, joalokaha re ithutile Khaolong ea 3 le ea 5). khatholoha, ena ea ho thaotha bolela hore bafuputsi ka matha tatellanong ea liteko tse amanang ka ho hlahlamana ka potlako.

Pele thaothang barupeluoa ho tloha MTurk bakeng liteko tsa hao, ho na le lintho tse 'nè tse bohlokoa ho tseba. Taba ea pele, bafuputsi ba bangata ba na le lipelaelo tse bao e seng itseng tsa liteko tse amang Turkers. Hobane lipelaelo sena hase tobileng, ho thata ho loantša le bopaki. Leha ho le joalo, ka mor'a lilemo tse 'maloa tsa lithuto tsa sebelisa Turkers, re ka fihlela qeto ea hore hona joale lipelaelo ena ha ho hlokahale haholo-holo. Ho 'nile ha lithuto tse ngata bapisa dipalopalo tsa Turkers ho baahi ba bang le lithuto tse ngata bapisa liphello tsa liteko le Turkers ho liphello e tsoang baahi ba bang. Fuoa mosebetsi ona tsohle, ke nahana hore tsela e molemohali bakeng sa uena ho nahana ka eona ke hore Turkers ke sebelisa kahlolo e molemo bebofalletsa sampole, ho tšoana liithuti tse empa hanyenyane ho feta tse sa tšoaneng (Berinsky, Huber, and Lenz 2012) . Kahoo, feela joalokaha liithuti tse ke baahi le kahlolo e molemo ho ba bang empa eseng kaofela etsa lipatlisiso liteko, Turkers ke baahi le kahlolo e molemo ho ba bang empa eseng kaofela lipatlisiso. Haeba u eang ho sebetsa le Turkers, joale e utloahala ho bala tse ngata tsa lithuto tsa tsena bapisang le ho utloisisa makolopetso a bona.

Ea bobeli, bafuputsi ba ntshetswa pele ho fetisisa litloaelo bakeng eketseha tumello hare tsa Turk liteko, 'me u lokela ho ithuta ka le ho latela tsena fetisisa litloaelo (Horton, Rand, and Zeckhauser 2011; Mason and Suri 2012) . Ka mohlala, bafuputsi ba sebelisa Turkers ba khothalletsoa hore ba sebelise screeners ho tlosa barupeluoa inattentive (Berinsky, Margolis, and Sances 2014; Berinsky, Margolis, and Sances 2016) (empa sheba le DJ Hauser and Schwarz (2015b) le DJ Hauser and Schwarz (2015a) ). Haeba u sa tlosa barupeluoa inattentive, joale le phello leha e le efe ea ho phekoloa ka ho hlatsuoa ka ho lerata hlahiswa ho tloha barupeluoa inattentive, 'me ka mokhoa ona o palo ea barupeluoa inattentive ka bonahalang. Ka teko ea Huber le basebetsi-'moho (2012) ka bang 30% ea barupeluoa hlōleha fuoe tlhokomelo screeners motheo. Bothata bo bong tšoana le Turkers ke bao e seng khongoana holimo barupeluoa (Chandler et al. 2015) .

Ea boraro, amanang le ho tse ling mefuta e meng ea liteko tsa digital, MTurk liteko ke keng hloa; Stewart et al. (2015) o hakanya hore nakong leha e le efe ho na le feela ka 7,000 batho MTurk.

Qetellong, u lokela ho tseba hore MTurk ke kopano le melawana a eona 'me Litloaelo (Mason and Suri 2012) . Ka tsela e tšoanang le eo o tla leka ho fumana hore ka setso sa naheng eo u ba il'o matha liteko hao, u lokela ho leka ho fumana hore lintlha tse eketsehileng ka setso sa eona le ditlwaelo tsa Turkers (Salehi et al. 2015) . 'Me, u lokela ho tseba hore Turkers tla bua ka teko ya hao haeba u ka etsa ntho e' ngoe e sa lokelang kapa e fosahetseng (Gray et al. 2016) .

MTurk ke tsela e Hoa makatsa hore ebe e bonolo ea ho e tsoma badiri barupeluoa ho liteko tsa hao, hore na ba laboratori-joaloka, tse kang Huber, Hill, and Lenz (2012) , kapa ho feta tšimo-joaloka, tse kang Mason and Watts (2009) , Goldstein, McAfee, and Suri (2013) , Goldstein et al. (2014) , Horton and Zeckhauser (2016) , le Mao et al. (2016) .

• Haha sehlahiswa sa hao hae (Karolo 4.5.1.3)

Haeba u nahana ho leka ho bōpa sehlahiswa sa hao, ke ho kgothaletsa e le hore u bala keletso e filwe ke MovieLens sehlopha ho Harper and Konstan (2015) . A temohisiso senotlolo ho tsoa phihlelong ea bona ke hore kaha projeke e mong le e atlehileng na le ba bangata, ho hloleha ha ba bangata. Ka mohlala, MovieLens sehlopha se qala lihlahisoa tse ling tse kang GopherAnswers ba neng ba e feletseng ho hloleha ha (Harper and Konstan 2015) . Mohlala o mong oa mofuputsi e hlōleha ha a ntse a leka ho haha ​​sehlahiswa ke wa leka Edward Castronova ho haha ​​papali e Inthaneteng e bitsoang Arden. Ho sa tsotellehe $ 250.000 a dithuso tsa ditjhelete, morero e ne e le flop (Baker 2008) . Projects joaloka GopherAnswers le Arden ke bomalimabe ho hongata ho feta tloaelehileng le sebelisoang malapeng diporojeke joaloka MovieLens. Qetellong, ha ke re hore ke ne ke sa tsebe sa bafuputsi leha e le efe e neng e ka katleho hahiloeng dihlahiswa bakeng tekeletso pheta mona dihlahlobo tse ka: 1) barupeluoa sebelisa sehlahiswa ka lebaka la seo e fana tsona (mohlala, ha ba lefuoang 'me ha ba baithaopi ba thusa saense ea bitsoang) le 2) sehlahiswa 'nile la sebelisoa bakeng sa teko ho feta e' ngoe ikhethang (khr, eseng teko e tšoanang ka makhetlo a multiple le matamo a fapaneng morupeluoa). Haeba u tseba mehlala e meng, ka kōpo ntsebisa.

• Bao re Sebetsang le ea matla (Karolo 4.5.2)

Ke utloa khopolo ea Quadrant Pasteur oa tšohloa khafetsa nakong lik'hamphani tse theknoloji, 'me e re thusa hore hlophisa boiteko ba ho etsa lipatlisiso Google (Spector, Norvig, and Petrov 2012) .

Bond le ho ithuta basebetsi-'moho le ' (2012) boetse a leka ho lemoha phello ea mekhoa ea phekolo tsena metsoalle ea ba ileng ba amohela tsona. Ka lebaka la a entseng teko eo, spillovers tsena ke ho le thata ho lemoha cleanly; babali nang le thahasello ba lokela ho bona Bond et al. (2012) bakeng sa puisano eketsehileng ka ho phethahetseng. Teko ena ke karolo ea neano e telele ea liteko tsa saense litabeng tsa lipolotiki. boitekong ho khothatsa kgetha (Green and Gerber 2015) . Tsena liteko kolo-tsoa-the-voute tloaelehileng karolo hobane ba a Pasteur oa Quadrant. Ke hore, ho na le batho ba bangata ba susumelletseha hore ba re eketsa vouta le ho vouta ka ba laola boitshwaro thahasellisang hore a leke likhopolo tse eketsehileng kakaretso phetoho boitshwaro le tšusumetso sechabeng.

Bafuputsi ba bang ba fane ka keletso ka matha liteko tšimo le mekhatlo molekane kang mekga e lipolotiki, mekhatlo e ikemetseng, le likhoebo (Loewen, Rubenson, and Wantchekon 2010; List 2011; Gueron 2002) . Ba bang ba e filwe ho kopa keletso mabapi le kamoo boikopanyo le mekhatlo e ka ama boqapi boo etsa lipatlisiso (Green, Calfano, and Aronow 2014; King et al. 2007) . Partnership boetse ho ka etsa hore lipotso tse boitshwaro (Humphreys 2015; Nickerson and Hyde 2016) .

• Design kopa keletso (Karolo 4,6)

Haeba u eang ho theha rera leano la Analysis pele ho matha teko ya hao, ke fana ka maikutlo a hore u qale ka ho bala melao e tataisang tlalehang. The mofumahali (matlafaditswe Standard tlaleha oa Liteko) melao e tataisang ba ne ba ntshetswa pele lekaleng la bongaka (Schulz et al. 2010) le fetotsoeng bakeng sa phuputso tsa phedisano (Mayo-Wilson et al. 2013) . A sete tse amanang ea melao e tataisang e se e ntshetswa pele ke bahlophisi ba Journal of liteko Science Lipolotiki (Gerber et al. 2014) (Bona le Mutz and Pemantle (2015) le ho Gerber et al. (2015) ). Qetellong, ho tlaleha melao e tataisang ba 'nile ba ntshetswa pele a kelello (Group 2008) ,' me le bone e boetse e Simmons, Nelson, and Simonsohn (2011) .

Haeba u theha rera leano la Analysis u lokela ho nahana ka pele ho ngodisa hobane pele ho re ngolisa ka molao tla eketsa kholiseho eo ba bang ba na a diphetho tsa hao. Ho ekelletsa moo, haeba u ba sebetsa le mekhatlo ea selekane e, e tla fokotsa bokhoni ba molekane oa hao hore a fetole Analysis ka mor'a bona liphello. Pele ho re ngolisa ka molao se tšoanelang ka ho eketsehileng tloaelehileng le sebelisoang malapeng a kelello (Nosek and Lakens 2014) , saense litabeng tsa lipolotiki. (Humphreys, Sierra, and Windt 2013; Monogan 2013; Anderson 2013; Gelman 2013; Laitin 2013) ,' me moruo (Olken 2015) .

Ha a ntse a bōpa pele ho Analysis moralo oa hao o lokela ho hlokomela hore bafuputsi ba bang ba boetse ba sebelisa bofokoli le atamela mathata a amanang le ho ntlafatsa sebetsa ka ho nepahetseng ba hakanyetsoang ho le phello kalafo, 'me ho ke ba bang ba khang ka mokhoa ona: Freedman (2008) , Lin (2013) , le Berk et al. (2013) ; sheba Bloniarz et al. (2016) bakeng sa tlhahisoleseding e eketsehileng.

Design kopa keletso ho toba bakeng sa Inthaneteng liteko tšimo e boetse e fanoa ka Konstan and Chen (2007) le ho Chen and Konstan (2015) .

• Create zero ya data polygonal litšenyehelo (Karolo 4.6.1)

Etsoe eketsehileng mabapi le liteko tsa ea MusicLab, sheba Salganik, Dodds, and Watts (2006) , Salganik and Watts (2008) , Salganik and Watts (2009b) , Salganik and Watts (2009a) ,' me Salganik (2007) . Etsoe eketsehileng mabapi le mohloli-nka-bao kaofela limmaraka, sheba Frank and Cook (1996) . Etsoe eketsehileng mabapi le mahlohonolo untangling le tsebo ka kakaretso bo eketsehileng, sheba Mauboussin (2012) , Watts (2012) , le Frank (2016) .

Ho na le katamelo e mong hore a felisa ditefello tse morupeluoa eo bafuputsi ba lokela ho sebelisa ka masene: conscription. Liphetolelong tse ngata tsa Inthaneteng liteko tšimo barupeluoa ba ha e le hantle bitsetsoa ho kena liteko le ka mohla lefisa. Mehlala ea mokhoa ona kenyeletsa Restivo le van de Rijt a (2012) teko mabapi le meputso ea Wikipedia 'me Bond le mosebetsi-'moho oa (2012) teko pele ho kgothaletsa batho ho vouta. Liteko tsena ha ba hlile ha ba na zero litšenyehelo polygonal, ba na le zero polygonal litšenyehelo moholo ho bafuputsi. Le hoja litsenyehelo tsa ho tse ngata tsa liteko tsena ke tse nyenyane haholo e mong ho e morupeluoa, ditjeho tse nyenyane behilweng palo e khōlōhali ea barupeluoa ka eketsang fihlang ka potlako. Bafuputsi matha setonanahali Inthaneteng liteko hangata lokafatsa bohlokoa ba tse nyenyane liphello hakanngoa kalafo ka ho bolela hore liphello tsena tse nyenyane e ka ba ea bohlokoa ha sebelisoa ho batho ba bangata. E tobileng monahano oa tšoanang e sebetsa ho ditjeo eo bafuputsi ba tlama barupeluoa. Haeba liteko tsa ea hao e etsa hore batho ba milione ho senya motsotso 'ngoe, teko e seng kotsi haholo ho motho leha e le ofe o itseng, empa ha e aggregate e sentse hoo e ka bang lilemo tse peli ea nako.

Katamelo e mong a bōpa zero polygonal litšenyehelo tefo ho barupeluoa ke ho sebelisa lothari, e leng mokhoa oo o boetse o 'nile oa sebelisoa ka etsa lipatlisiso phuputso e entsoeng (Halpern et al. 2011) . Qetellong, bakeng sa boitsebiso bo eketsehileng mabapi qapa e be e thabisang ke mosebedisi liphihlelo bona Toomim et al. (2011) .

• Nka sebaka, Hloekisa, 'me Fokotsa (Karolo 4.6.2)

Mona ke litlhaloso tsa pele tsa lintho tse tharo tseo R, ho tloha Russell and Burch (1959) :

"Phetolo bolela tloswa bakeng elang hloko phelang liphoofolo phahameng oa lintho tse bonahalang insentient. Phokotso bolela phokotso e palo ea liphoofolo sebelisoa ho fumana tlhahisoleseding ya tjhelete e fuoa le sebetsa ka ho nepahetseng. Ntlafatso ho bolela fokotseha leha e le lefe incidence kapa teba ha mekgwatshebetso ya hlokang botho sebelisoa ho liphoofolo tseo e leng ntse ba lokela ho sebelisoa. "

Eo ke sisinya u se ke ua hatelle melao-motheo boitshwaro hlalosoa Khaolong ea 6. R tharo li na le hoo, ba ile ba ba ka haholoanyane qaqisa phetolelo e 'ngoe ea melao-motheo eo-beneficence-ho toba bakeng sa kemiso ea liteko tsa batho.

Ha le tšohla Emotional boloetse bo tsoaetsanoang, ho na le tse tharo tse litokollo non-boitshwaro ho boloka ka kelellong ha o toloka teko ena. Taba ea pele, ho ke ke ha hlakileng kamoo makolopetso a sebele oa teko ea amahanya ho na lipolelo ya kgopolo; ka mantsoe a mang, ho na le lipotso mabapi le ho betla tsela bonnete. Ho ke ke ha hlakileng ba hore ntho e molemo 'me a fosahetseng lentsoe lokisa mathata ao ha e le hantle e pontshi molemo tsa puso maikutlong barupeluoa hobane 1) ho ke ke ha hlakileng ba hore mantsoe ao batho ngolang ke pontshi molemo tsa maikutlo a bona le 2) ho ke ke ha hlakileng ba hore itseng ea maikutlo Analysis mokhoa hore bafuputsi sebelisoa ke khona ho tshepahala ka fihlela qeto ea maikutlo (Beasley and Mason 2015; Panger 2016) . Ka mantsoe a mang, ho ka 'na e mpe tekanyong ea pontšo leeme. Ea bobeli, rala le ho tshekatsheko ya teko ea re bolella letho ka ea neng a mbano ya mabe na fetisisa (khr, ha ho na tshekatsheko ya heterogeneity oa liphello kalafo) 'me seo mochine a ka ba teng. Tabeng ena, bafuputsi ba ile ba tlameha lotho ea tlhahisoleseding e mabapi le barupeluoa, empa ba ne ba hantle tšoaroa joaloka diwidget a Analysis eo. Ea boraro, phello saese teko ena e ne e le nyenyane haholo; phapang pakeng tsa ho phekola le ho laola maemo a ke about 1 mantsoeng 1,000. Ka pampiri ea bona, Kramer le basebetsi-'moho etsa tabeng eo le phello ea boholo ena ke ea bohlokoa hobane ba limilione tse makholo batho fihlella News bona Fepa letsatsi le letsatsi. Ka mantsoe a mang, ba ile ba pheha khang ea hore esita le liphello tse nyenyane motho ka mong li khōlō aggregate. Esita le haeba u ne u ka amohela ho pheha khang ena, e ntse e hlakile haeba phello ea boholo ena ke ea bohlokoa mabapi le potso e akaretsang libuka tsa saense mabapi le boloetse bo tsoaetsanoang maikutlong. Etsoe eketsehileng mabapi le maemo moo ho liphello tse nyenyane ke tsa bohlokoa ho bona Prentice and Miller (1992) .

Ka ho ya ka R pele (Phetolo), bapisa Emotional boloetse bo tsoaetsanoang teko (Kramer, Guillory, and Hancock 2014) le maikutlong boloetse bo tsoaetsanoang tlhaho teko (Coviello et al. 2014) e fana ka ba bang ba lithuto life kakaretso mabapi le lintho tse khoebo-offs amehang le ho fallela ho tloha liteko ho liteko tsa tlhaho (le le atamela tse ling joaloka matching le lekang ho batlang li atametse ho liteko tsa ka ya data bao e seng liteko, sheba Khaolo ea 2). Ho phaella ho melemo boitshwaro, ho switjha ho tloha etsa liteko ho lithuto tsa bao e seng liteko boetse nolofalletsa bafuputsi ho ithuta mekhoa ea phekolo le hore ba ba logistically khone ho romela. Melemo ena boitshwaro le thuso tle le liphello tse bohloko, leha. Le liteko tsa tlhaho bafuputsi ba le ka tlaase ho moo laola lintho tse joaloka ho thaotha oa barupeluoa, randomization, le mofuta wa kalafo eo. Ka mohlala, ho meedi e mong oa pula li ka phekoloa ke hore e ka bobeli eketsa positivity le decreases negativity. Thutong etsa liteko le joalo, Kramer le basebetsi-'moho ba ile ba khona ho tloaela positivity le negativity nosi.

Atamela. itseng sebelisoa ke Coviello et al. (2014) o ne a ka ho eketsehileng ile a qaqisa a Coviello, Fowler, and Franceschetti (2014) . Bakeng sa selelekela divariabole karaoke bona Angrist and Pischke (2009) (ka tlase bo hlomamisitsoeng) kapa Angrist, Imbens, and Rubin (1996) (eketsehileng bo hlomamisitsoeng). Bakeng sa appraisal lekhonono ya dipharologano karaoke bona Deaton (2010) , le bakeng sa selelekela divariabole karaoke ka liletsa fokolang (pula e seletsa fokola), sheba Murray (2006) .

More ka kakaretso, e leng selelekela molemo ho liteko tsa tlhaho Dunning (2012) , le Rosenbaum (2002) , Rosenbaum (2009) , le Shadish, Cook, and Campbell (2001) fana mehopolo molemo tse mabapi le ka go fopholetsa liphello e bakang ntle liteko.

Ka ho ya ka R bobedi (ntlafatso), ho na le libuka tsa saense le thuso khoebo-offs ha re nahana ho fetola moralo oa boloetse bo tsoaetsanoang Emotional tsoang thibela posts by ho boosting posts by. Ka mohlala, ho ka 'na e-ba joalo hore ho kenngwa tshebetsong botekgeniki tsa News Feed etsa hore ho ke haholo bonolo ho etsa teko e le thibela lintho tse ngotsoeng moo ho e-na teko e le boosting posts by (ela hloko hore e teko le thibela lintho tse ngotsoeng moo ka ho sebediswa e le lera ka la tlhōrōng ea tsamaiso News Feed ntle ho hlokahale leha e le efe bakeng sa alterations ea tsamaiso bothata bo itseng '). Tsa saense tsa Leha ho le joalo, khopolo lebisitsoe ke teko eo ha aa ka ho hlaka a bontša boqapi ba bona le 'ngoe holim' a mang.

Ka bomalimabe, ha ke tseba ka taba ho etsa lipatlisiso mohlomong pele mabapi le lintho tse lokelang mong ka thibela le boosting a khotsofetse ha News Feed. Hape, ke e-s'o bone ho etsa lipatlisiso ho hongata ka hloekisa liphekolo ho etsa hore ba ka tlaasenyana ho moo kotsi; mokhelo 'ngoe e Jones and Feamster (2015) , e leng se nahana ka taba ea tekanyo ea Internet li thibela (a sehlooho ke buisana Khaolong ea 6 kamanong ho ithuta Encore (Burnett and Feamster 2015; Narayanan and Zevenbergen 2015) ).

Ka ho ya ka R boraro (Phokotso), e ngotsoeng selelekeleng molemo ho Analysis tsa moetlo matla ke Cohen (1988) . covariates pele ho kalafo ka akarelletsoang sethaleng sa ho qaptjoa le ho ba sethaleng tshekatsheko ya liteko; Khaolo ea 4 ea Gerber and Green (2012) e fana ka selelekela molemo ho e atamela mathata bobeli, 'me Casella (2008) e fana ka ho feta ka botebo kalafo. Mekhoa eo sebedisa tlhahisoleseding ena pele ho kalafo ea randomization ba ka tloaelo ho bitsoa mohlomong koetsoe boqapi boo etsa liteko kapa stratified boqapi boo liteko (lenanentswe e sa sebelisoa ka mokhoa o tsitsitseng ka mose baahi ba metse); mekhoa ena li ka ho teba amanang le mekhoa stratified mohlala tšohloa Khaolong ea 3. See Higgins, Sävje, and Sekhon (2016) bakeng sa tse eketsehileng mabapi le ho sebelisa boqapi boo tsena liteko maholohali. Covariates pele ho kalafo ka boela akarelletsa ho ba sethaleng hlahlojoe. McKenzie (2012) hlahloba atamela. phapang-in-ho se tšoane ho hlahloba liteko tšimo ka ho qaqileng e khōloanyane. Sheba Carneiro, Lee, and Wilhelm (2016) bakeng sa ho eketsehileng ka khoebo-offs pakeng tsa le atamela fapaneng ho eketsa nepahetseng a likhakanyo tsa liphello kalafo. Qetellong, ha a etsa qeto ea hore na ba leke ho kenyeletsa covariates pele ho kalafo nakong ea sethala tsa boqapi kapa Analysis (kapa ka bobeli), ho na le mabaka a 'maloa hore ba nahane ka. Ka bophirima moo bafuputsi batla ho bontša hore ha ba "tšoasa litlhapi" (Humphreys, Sierra, and Windt 2013) , a sebelisa pele ho kalafo covariates a sethala moralo ka thusa (Higgins, Sävje, and Sekhon 2016) . Maemong moo barupeluoa fihla sequentially, haholo-holo Inthaneteng liteko ea tšimo, ho sebelisa boitsebiso bo pele ho kalafo ea sethaleng moralo ka ba thata logistically, sheba Ka mohlala Xie and Aurisset (2016) .

Ke habohlokoa ho phaella a batla a ea intuition na ke hobane'ng ha phapang-in-ho se tšoane ka etsa ho hongata hakana molemo ho feta ho phapang-in-bolela'ng. Inthaneteng diphetho ba bangata ba na phahameng haholo lumellane ha (sheba mohlala, Lewis and Rao (2015) le ho Lamb et al. (2015) ) mme ba batlang e tsitsitse ha nako e ntse. Tabeng ena, ho etsoa phetoho lintlha tla ba le haholo nyenyane tse lumellane ha eketsa matla a mamella teko dipalopalo. Lebaka le leng sena atamela e sa sebelisoa khafetsa ke hore pele ho a le lilemo dijethale e ne e se ho tloaelehile hore ho be le diphetho pele ho kalafo. Tsela e eketsehileng e hahiloeng ka konkreite ho nahana ka eona ke ho nahana teko ea ho lekanya hore na e ikoetlisa kamehla e be tse tobileng bakang boima ba 'mele. Haeba u etsa phapang e-in-le bolela atamela, khakanyo ea hao e tla ba dipharologano fa e tlisoang ke ho dipharologano fa ka litekanyo tse ka baahi. Haeba u etsa phapang e-in-phapang katamelo Leha ho le joalo, e le hore ka tlhaho etsahalang feto-litekanyo gets bukkake tlosoa 'me u ka habonolo lemoha phapang bakoang ke phekolo eo.

Tsela e 'ngoe ea bohlokoa ho fokotsa palo ea barupeluoa a teko ya hao ke ho khanna Analysis matla, e leng Kramer le basebetsi-'moho ba ka be ba entse thehiloeng boholo bo phello hlokomela ho tloha teko tlhaho ke Coviello et al. (2014) kapa pejana ho etsa lipatlisiso bao e seng etsa liteko ka Kramer (2012) (ha e le hantle tsena ke lintho qetellong ea khaolo ena). Hlokomela hore u sebelise sena wa manollo matla ke a batla fapaneng ho feta tloaelehileng. A le lilemo li analoge, bafuputsi ka kakaretso ile a etsa Analysis matla a ho etsa bonnete ba hore thuto ea bona e ne e se e le nyenyane haholo (khr, tlas'a-haheletsoe). Hona joale, ha ho le jwalo, bafuputsi lokelang ho se etsa Analysis matla a ho etsa bonnete ba hore thuto ea bona e ke ke seholo haholo (khr, holimo-haheletsoe).

Qetellong, ke nka hore phaella ka ho re bone R: Repurpose. Ke hore, haeba bafuputsi ba iphumana ba le ya data eketsehileng etsa liteko ho feta eo ba e hlokang ho rarolla etsa lipatlisiso potso ea bona ea pele, ba lokela ho repurpose ya data ho botsa lipotso tse ncha. Ka mohlala, nahana hore Kramer le basebetsi-'moho o ne a sebelisitse phapang-in-ho se tšoane estimator 'me ba fumana ba le ka ya data tse fetang tse hlokahalang ho rarolla potso ea bona lipatlisiso. Ho e-na hase ho sebelisa ya data kahohle kamoo nka, ba ne ba ithutile boholo ba le phello e le mosebetsi ho pele ho kalafo poleloana e reng maikutlong. Feela joalokaha Schultz et al. (2007) fumane hore phello ea phekolo eo e ne e fapane sebakeng sa khanya le boima basebedisi, mohlomong liphello tsa News Feed ne li fapane bakeng sa batho ba seng ba ntse sekamela ho ngolang melaetsa thabileng (kapa ka utloa bohloko.) Repurposing ka lebisang "ho tšoasa litlhapi" (Humphreys, Sierra, and Windt 2013) le se reng "E leq-Hacking" (Simmons, Nelson, and Simonsohn 2011) , empa tsena li haholo-holo addressable le motswako wa tlalehelang tšepahala (Simmons, Nelson, and Simonsohn 2011) , pele ho re ngolisa ka molao (Humphreys, Sierra, and Windt 2013) , le mekhoa mochine ithuta hore leka ho qoba holimo-loketseng.