Zvaunofanira kuverenga zvinotevera

• Nhanganyaya (chikamu 4.1)

Mibvunzo pamusoro pezvakaitika mukutsvakurudza kwemagariro evanhu zvinowanzooma uye zvakaoma. Kuti uwane nzira yekutanga yezvakakonzerwa nemafungiro emamwe mabhuku, ona Pearl (2009) , uye nenzira yakatangira maererano nemigumisiro inogona kuitika, ona Imbens and Rubin (2015) . Kuti uenzanise pakati penzira idzi mbiri, ona Morgan and Winship (2014) . Kuti uwane nzira yakagadziriswa yekutsanangurira kupesana, ona VanderWeele and Shpitser (2013) .

Muchitsauko chino, ndakasika izvo zvaiita sechiedza chakajeka pakati pekwanisi yedu yekuita maonero ekuenzanisa kubva kune dhigirii uye kwete-dheedhi data. Zvisinei, ini ndinofunga kuti, muchokwadi, kusiyana kwakanyanya kujeka. Semuenzaniso, munhu wose anobvuma kuti kusvuta kunokonzera kenza, kunyange zvazvo kusina kuongororwa kwakaitwa nemaitiro kunokonzera vanhu kusvuta kunenge kwaitwa. Nokuda kwekubwinya kwebhuku-urefu hwemishonga pakuita zvinokonzera kushandiswa kubva pane zvisiri nyaya yekuona kuona Rosenbaum (2002) , ( ??? ) , Shadish, Cook, and Campbell (2001) , uye Dunning (2012) .

Zvitsauko 1 ne2 zve Freedman, Pisani, and Purves (2007) zvinopa kujekesa zvakajeka kune kusiyana pakati pekuedza, kuongororwa kwakaedzwa, uye kuongororwa kwakaitwa randomized.

Manzi (2012) inopa sumo inofadza uye inooneka inoenderana nefilosofi uye zvinyorwa zvinotarisirwa zvidzidzo zvinongorongedzwa. Inopawo inonakidza mienzaniso chaiyo-yenyika ye simba rekuedza mune zvebhizimisi. Issenberg (2012) inopa sumo inonakidza yekushandiswa kwekuedza mune zvematongerwe enyika.

• Chii chinoedza? (chikamu 4.2)

Box, Hunter, and Hunter (2005) , @ casella_statistical_2008, uye Athey and Imbens (2016b) vanopa mazano akanaka kumatanho ekuenzanisa nekuongorora. Uyezve, kune hutano hwakanaka hwekushandiswa kwezviongorori mundima dzakawanda dzakasiyana-siyana: mari (Bardsley et al. 2009) , zvehupfumi (Willer and Walker 2007; Jackson and Cox 2013) , psychology (Aronson et al. 1989) , sayenzi yezvematongerwe enyika (Morton and Williams 2010) , uye mutemo wevanhu (Glennerster and Takavarasha 2013) .

Kukosha kwekutora chikamu chevatori vechikamu (semuenzaniso, shanduro) kazhinji kunonzwisiswa mukutsvakurudza kwekuedza. Zvisinei, kana kukanganisa kwehutano kune hutano muhuwandu, ipapo sampuli inokosha. Longford (1999) inonyatsojekesa pfungwa iyi apo inotsigira vatsvakurudzi vachifunga nezvekuedza sekuongororwa kwevanhu pasina sampling.

• Zviyero zviviri zvekuedza: lab-munda uye analog-digital (chikamu 4.3)

Ini ndakaratidza kuti pane kuenderera mberi pakati pebasa nemashamba ekuongorora, uye vamwe vatsvakurudzi vakagadzira zvinyorwa zvakanyatsorongeka, kunyanya izvo zvinoparadzanisa mhando dzakasiyana dzekuongorora masimi (Harrison and List 2004; Charness, Gneezy, and Kuhn 2013) .

Mamwe mapepa akafananidza mabhidhi uye masayeri emunyanzvi mune zvinyorwa (Falk and Heckman 2009; Cialdini 2009) uye maererano nemigumisiro yezviongorori zvakajeka mune zvematongerwo enyika (Coppock and Green 2015) , mari yezvehupfumi (Levitt and List 2007a, 2007b; Camerer 2011; Al-Ubaydli and List 2013) , uye psychology (Mitchell 2012) . Jerit, Barabas, and Clifford (2013) vanopa mazano akanaka ekutsvakurudza kuenzanisa migumisiro yebasa nemashamba ekuedza. Parigi, Santana, and Cook (2017) inotsanangura kuti mhepo inoshandiswa sei muIndaneti inogona kuenzanisa mamwe maitiro ekuraira uye masimi ekuedza.

Kunetseka pamusoro pevatori vechikamu kuchinja mufambiro wavo nokuti vanoziva kuti vari kunyatsocherechedza dzimwe nguva kunonzi kudzvirwa kwezvinhu , uye vakadzidza mune zvepfungwa (Orne 1962) uye mune zvehupfumi (Zizzo 2010) . Kunyange zvazvo inowanzobatanidzwa neongororo yekuongorora, nyaya idzi dzinogona kukonzera zvinetso zvekuongorora masimi zvakare. Muzvokwadi, zvinoda mitsara inowanzonziwo Hawthorne tsvina, izwi rinowana maitiro anozivikanwa ekujekesa anotanga muna 1924 paWawthorne Works yeWestern Electric Company (Adair 1984; Levitt and List 2011) . Zvose zvinoda mitsara uye maHotthorne zvinowanikwa zvakanyatsoenderana nemafungiro ekuita basa rekuita basa rinokurukurwa muchitsauko 2 (onawo Webb et al. (1966) ).

Mune mienzaniso yakareba kare mune zvehupfumi (Levitt and List 2009) , sayenzi yezvematongerwe enyika (Green and Gerber 2003; Druckman et al. 2006; Druckman and Lupia 2012) , psychology (Shadish 2002) , uye mutemo wevanhu (Shadish and Cook 2009) . Imwe nzvimbo yezvesayenzi iyo mashandisirwo emunda akakurumidza kuve akakurumbira ndeyekusimudzira nyika. Kuongorororwa kwakanaka kwebasa iri mukati mehupfumi kuona Banerjee and Duflo (2009) , uye nokuda kwekuongorora kwakakosha ona Deaton (2010) . Kuongorora kwebasa iri mune zvematongerwe enyika kuona Humphreys and Weinstein (2009) . Pakupedzisira, maitiro ehutano anokonzerwa nekuongorora masimba ave akaongororwa munyaya yezvematongerwe enyika (Humphreys 2015; Desposato 2016b) uye hupfumi hwehupfumi (Baele 2013) .

Muchikamu chino, ndakakurudzira kuti utano hwekusatirapa hunogona kushandiswa kugadzirisa kukwana kwekufungidzirwa kwekurapa, asi pane imwe nharo pamusoro peiyi nzira; ona Freedman (2008) , W. Lin (2013) , Berk et al. (2013) , uye Bloniarz et al. (2016) kuti uwane mamwe mashoko.

Pakupedzisira, kune mamwe maviri maitiro eongororo yakagadzirwa nesayendisiti yehupfumi iyo isingakodzeri zvakakodzera pamwe ne-lab-field dimension: kuongorora kuongororwa uye kuongororwa kwevanhu. Kuongororwa kuongororwa ndiko kuedza kushandisa shanduko yezvinoongororwa uye kuenzanisa mhinduro kune dzimwe shanduro dzemibvunzo imwechete (dzimwe ongororo dzinoongororwa dzinowanikwa muChitsauko 3); kune zvakawanda pamusoro peongororo yekuongorora ona Mutz (2011) . Kuongororwa kwevanhu munzanga ndiko kuongororwa apo kurapwa kune imwe nzira yehupfumi iyo ingangoshandiswa nehurumende. Kuongorora kwehutano kwakabatana zvikuru nehutano hwepurogiramu. Kuti uwane zvakawanda pamusoro pekuongorora mazano, ona Heckman and Smith (1995) , Orr (1998) , uye @ glennerster_running_2013.

• Kufambira mberi kwezvinhu zvisiri nyore (chikamu 4.4)

Ndakasarudza kufunga nezvemashoko matatu: kuvimbika, huterogeneity yemigumisiro yemishonga, uye nzira. Aya mazano ane mazita akasiyana mune dzimwe nyika. Semuenzaniso, vanodzidza nezvepfungwa vanowanzofambisa mberi nekuongorora zvisiri nyore nekuisa pfungwa kune vamiririri nevatongi (Baron and Kenny 1986) . Pfungwa yevamiririri inotorwa nezvo zvandinodanidza nzira, uye pfungwa yevatungamiri inotorwa nezvose zvandinonzi kunze kwepamutemo (semuenzaniso, zvingaguma zvekuedza zvingave zvakasiyana kana zvichishanda mumamiriro ezvinhu akasiyana) uye heterogeneity yemigumisiro yechirwere ( somuenzaniso, iyo migumisiro yakakura kune vamwe vanhu kupfuura vamwe).

Kuedzwa Schultz et al. (2007) inoratidza kuti masangano evanhu anokwanisa sei kushandiswa pakugadzirisa mazano anobatsira. Kuti uwane nharo huru pamusoro pebasa renyanzvi mukugadzira matanho anobatsira, ona Walton (2014) .

• Kuzadzika (chikamu 4.4.1)

Idzo pfungwa dzekunze nekunze dzakagadziriswa dzakatanga kuziviswa Campbell (1957) . Ona Shadish, Cook, and Campbell (2001) kuti uwane mamwe mashoko akawandisa uye kunyatsotsanangurwa kwemashoko ekugumisa kugutsikana, mukati mekuvimbika, kuvaka zvakavimbika, uye kunze kwekuita.

Nokuda kwehuwandu hwemashoko akabatanidzwa nehuwandu Imbens and Rubin (2015) ona Gerber and Green (2012) (kubva mune zvemaonero ehutano hwevanhu) uye Imbens and Rubin (2015) (kubva pane zvinotaridzirwa). Dzimwe nyaya dzekuverenga (Bakshy and Eckles 2013) zvinosanganisira nyaya dzakadai sedhidziro dzakagadzirisa nzira dzekugadzirisa kuvimba panguva dzakasiyana nedzimwe tsanangudzo (Bakshy and Eckles 2013) .

Kushandiswa kwemukati kunogona kuva kwakaoma kuratidza mumamiriro ezvinhu akaoma. Ona, semuenzaniso, Gerber and Green (2000) , Imai (2005) , Gerber and Green (2005) nokuda kwekukakavadzana pamusoro pekushandiswa kwenzvimbo yakaoma yekuedza nezvekuvhota. Kohavi et al. (2012) uye Kohavi et al. (2013) inopa chiziviso mumatambudziko ekubvumirana kwemazuva ekutsvaga mundima.

Chimwe chekutyisidzira kukuru mukati mechiratidzo ndechokuti inogona kukanganisa randomization. Imwe nzira inogona kuve nayo yekuona zvinetso nemaitiro ekuenzanisira ndeyekuenzanisa kurapwa uye mapoka ekudzivirira pane zviitiko zvinoonekwa. Iko rudzi rwekuenzanidza kunonzi kutarisa check . Ona Hansen and Bowers (2008) nenzira yekuverengera kuenzanisa checks uye Mutz and Pemantle (2015) pamusoro pekutsvaga pamusoro peyeri chekuongorora. Semuenzaniso, kushandisa check balance, Allcott (2011) akawana humwe humwe uchapupu hwokuti randomization haina kuitwa nenzira yakarurama muzvikamu zvitatu zveMabasa ekuvhima (ona tafura 2; nzvimbo 2, 6, uye 8). Kune dzimwe nzira, ona chitsauko 21 Imbens and Rubin (2015) .

Zvimwe zvinonyanya kukosha zvine chokuita nekugadzirisa mukati ndezvekuti: (1) rimwe chete risina kubvumirana, uko vanhu vese vari muboka rekurapa vakagamuchira kurapwa, (2) maviri asingateereri, apo vanhu vose vari muboka rekurapa vanogamuchira kurapwa nevamwe vanhu boka rekutonga rinogamuchira kurapwa, (3) maitiro, apo zvibereko zvisinganzwisisi kune vamwe vatori vechikamu, uye (4) kupindira, apo kurapwa kunopararira kubva kune vanhu vari mumatambudziko ekurapa kune vanhu vari mumamiriro ekudzivirira. Ona zvitsauko 5, 6, 7, uye 8 Gerber and Green (2012) kuitira zvakawanda pane imwe neimwe yemibvunzo iyi.

Nokuda kwekuvaka kuvaka, ona Westen and Rosenthal (2003) , uye nezvimwe pakuvaka kuvimbika mune zvinyorwa zvemashoko makuru, Lazer (2015) uye chitsauko 2 chebhuku rino.

Chimwe chezvinhu zvekunze kwechokwadi ndezvekugadziriswa kwekupindira kunoedzwa. Allcott (2015) inopa zvakanyatsofungisisa uye zvinyorwa zvekutsvaga kwekutsvaga nzvimbo yekusarudza. Iyi nyaya inokurukurwawo Deaton (2010) . Chimwe chezvinhu zvekunze kwechokwadi ndeyekuti kushandiswa kwemashoko mamwechete ekupindira kwakadaro kuchava nemigumisiro yakafanana. Munyaya iyi, kuenzanisa pakati Schultz et al. (2007) uye Allcott (2011) inoratidza kuti Majekesheni ekugonesa aiva nemaitiro mashoma anokonzerwa nehutano kupfuura hutsva hwepakutanga hwaSchultz pamwe nevamwe (1.7% kusvika 5%). Allcott (2011) akafungidzira kuti kuongorora kwekutevera kwakave nemigumisiro shoma nekuda kwenzira iyo kurapwa kwakasiyana: chikwangwani chinyorwa chinyorwa sechikamu chekudzidza kunotsigirwa neyunivhesiti, kana ichienzaniswa nekodhikisi yakadhindwa sechikamu chemahombe-akabudiswa mushumo wekambani yemasimba.

• Heterogeneity yemishonga zvinokonzerwa (chikamu 4.4.2)

Nokuda kwehuwandu hwemashoko ehutano hwehutano hwehutano hwekurapa mumunda unoedza, ona chitsauko 12 Gerber and Green (2012) . Nokuda kwekutaura kwehutano hwehutano hwekurapa muhutano muhutano, ona Kent and Hayward (2007) , Longford (1999) , Kravitz, Duan, and Braslow (2004) . Kufunga nezvehutano hwehutano hwehutano hunowanzoisa pfungwa pazvisiyanano zvinoenderana nehutano husati hwavapo. Kana iwe unofarira huterogeneity zvichienderana nemigumisiro yekugadziriswa kwemashure, kune dzimwe nzira dzakaoma dzinoshandiswa dzinoshandiswa, dzakadai sedzidziso huru (Frangakis and Rubin 2002) ; ona Page et al. (2015) kwekuongorora.

Vatsvakurudzi vakawanda vanofungidzira kuti kupera kwemaitiro ekurapa kunoshandiswa kushandiswa kwemaitiro, asi nzira itsva dzichivimba nemichina yekudzidza; ona, semuenzaniso, Green and Kern (2012) , Imai and Ratkovic (2013) , Taddy et al. (2016) , uye Athey and Imbens (2016a) .

Pane kumwe kusava nechokwadi pamusoro pekuwana huterogeneity yemigumisiro pamusana pezvipingamupinyi zvinowanzofananidza uye "kubata." Kune nzira dzakasiyana-siyana dzezvikwangwani zvinogona kubatsira kugadzirisa zvinetso pamusoro pekuenzaniswa kwakawanda (Fink, McConnell, and Vollmer 2014; List, Shaikh, and Xu 2016) . Imwe nzira yekunyanya kunetseka pamusoro pe "hove" inotanga kunyorwa, iyo inowedzera kuwanzana mune zvepfungwa (Nosek and Lakens 2014) , sayenzi yezvematongerwo enyika (Humphreys, Sierra, and Windt 2013; Monogan 2013; Anderson 2013; Gelman 2013; Laitin 2013) , uye economics (Olken 2015) .

Muchidzidzo Costa and Kahn (2013) chete chikamu chehafu yemhuri iri kuedza chingabatanidzwa nehuwandu hwevanhu. Vaverengi vanoda kuziva izvi vanofanira kureva pepa rekutanga.

• Nzira (chikamu 4.4.3)

Nzira dzakakosha zvikuru, asi dzinoita kuti dzive dzakaoma kudzidza. Kutsvakurudza pamusoro pezvirongwa zvakabatana zvikuru nekudzidza kwevanopindirana mune zvepfungwa (asi onawo VanderWeele (2009) kuitira kuenzanisira pakati pemashoko maviri). Nzira dzakagadziriswa dzekuwana nzira, dzakadai senzira yakagadzirwa Baron and Kenny (1986) , yakajairika. Zvinosuruvarisa, zvinowedzera kuti iyo nzira inotarisana nedzimwe pfungwa dzakasimba (Bullock, Green, and Ha 2010) uye vanotambudzika kana pane nzira dzakawanda, sezvinogona kutarisirwa mumamiriro ezvinhu akawanda (Imai and Yamamoto 2013; VanderWeele and Vansteelandt 2014) . Imai et al. (2011) uye Imai and Yamamoto (2013) vanopa mamwe maitiro ekuvandudza maitiro. Uyezve, VanderWeele (2015) inopa kurapa kwenguva refu kwebhuku nemigumisiro yakawanda yezvakakosha, kusanganisira nzira yakakwana yekuongorora senzwi.

Imwe nzira yakasiyana inotsvaga kuongorori dzinoedza kushandisa nzira yacho zvakananga (semuenzaniso, kupa vafambisi vemavhitamini C). Zvinosuruvarisa, mune zvakawanda zvekugadzirisa sayenzi, kune dzimwe nguva nzira dzakawanda uye zvakaoma kuronga mishonga inoshandura imwe isina kushandura vamwe. Vamwe vanosvika pakuedza kuchinja maitiro zvinotsanangurwa Imai, Tingley, and Yamamoto (2013) , Ludwig, Kling, and Mullainathan (2011) , uye Pirlott and MacKinnon (2016) .

Vatsvakurudzi vanoshandisa zvizere zviitiko zvekutsvaga vachada kufunganya nezvekuongororwa kwakawanda kwekuongorora; ona Fink, McConnell, and Vollmer (2014) uye List, Shaikh, and Xu (2016) kuti uwane mamwe mashoko.

Pakupedzisira, maitiro anewo nguva refu mufilosofi yesayenzi yakatsanangurwa Hedström and Ylikoski (2010) .

• Kushandisa nzvimbo dziripo (chikamu 4.5.1)

Nokuda kwekushandisa kwekutsvakurudza mabhuku uye zvidzidzo zvekuongorora kuongorora kusarura, ona Pager (2007) .

• Zvivakire iwe (chidzidzo 4.5.2)

Nzira yakakurumbira yekuunganidza vatori vechikamu kuongorori yaunovaka ndiyo Amazon Mechanical Turk (MTurk). Nemhaka yokuti MTurk mimikirwo yezviitiko zvekare zvebazi-vanhu vanobhadhara kuita mabasa avo zvavasingazoiti nokuda kwemasununguko-vakawanda vatsvakurudzi vatotanga kushandisa Turkers (vashandi veTTurk) sevadzidzi vanoongorora, zvichiita kuti kukurumidza kudhura uye kudhura kudarika kudarika kunogona kuwanikwa mumasikirwo e-campus laboratory experiments (Paolacci, Chandler, and Ipeirotis 2010; Horton, Rand, and Zeckhauser 2011; Mason and Suri 2012; Rand 2012; Berinsky, Huber, and Lenz 2012) .

Kazhinji, zvakanakisa zvakakosha zvekushandisa vatori vechikamu vakanyorerwa kubva kuMaturk vane logistics. Kunyange zvazvo mazano ekuraira angatora masvondo kuti aende uye mashandisirwo emunda anogona kutora mwedzi kuti agadzire, kuongorora nevatori vechikoro vakanyorerwa kubva kuMaturk kunogona kushanda mumazuva. Semuenzaniso, Berinsky, Huber, and Lenz (2012) vakakwanisa kutora zvidzidzo mazana mana muzuva rimwe chete kuti vatore chikamu mumutambo wemaminiti 8. Uyezve, vatori vechikamu ava vanogona kunyoreswa kuitira chero chinangwa chero chipi zvacho (kusanganisira kuongororwa uye kubatana kukuru, sezvakakurukurwa muzvitsauko 3 ne5). Iko kusununguka kwekutora kunoreva kuti vatsvakurudzi vanogona kutevedzera zvikamu zvekuenzanisa kwakabatana mukutsvaga kwakakurumidza.

Usati waunganidza vatori vechikamu kubva kuMaturk nokuda kwekuongorora kwenyu, kune zvinhu zvina zvinokosha zvaunoda kuziva. Chokutanga, vatsvakurudzi vazhinji vane hanya nekunetseka kwezviitiko zvinosanganisira Turkers. Nokuti izvi kusava nechokwadi hazvisi pachena, zvakaoma kupikisana nouchapupu. Zvisinei, mushure memazana emakore ekudzidza achishandura Turkers, isu zvino tinogona kugumisa kuti kusava nechokwadi uku hakusi kukodzera. Pakave nekuongororwa kwakawanda kuenzanisa nhamba yevanhu veTurkers neyevamwe vanhu uye zvidzidzo zvakawanda kuenzanisa migumisiro yekuedza neTurkers kune avo vanobva kune vamwe vanhu. Ndichipa basa iri rose, ndinofunga kuti nzira yakanakisisa yekuti iwe ufunge pamusoro payo ndeyekuti Turkers inyanzvi yakanaka, yakafanana nevadzidzi asi zvishoma zvakasiyana (Berinsky, Huber, and Lenz 2012) . Nokudaro, sekudzidzira kwevadzidzi vane chikonzero chakanaka kune vamwe, asi kwete yose, tsvakurudzo, Turkers inzvimbo yakakodzera kune vamwe, asi kwete yose, tsvakurudzo. Kana iwe uri kuenda kunoshanda naTurkers, zvinonzwisisika kuverenga dzakawanda zvezviongororwa izvi nekuenzanisa uye kunzwisisa mavara avo.

Chechipiri, vatsvakurudzi vakaita zvirongwa zvakanakisisa zvekuwedzera kukwanisa kwemukati maTurk kuedza, uye iwe unofanira kudzidza pamusoro pekutevera izvi zvakanakisisa-maitiro (Horton, Rand, and Zeckhauser 2011; Mason and Suri 2012) . Semuenzaniso, vatsvakurudzi vanoshandisa Turkers vanokurudzirwa kushandisa zvipfeko kuti vabvise vatori (Berinsky, Margolis, and Sances 2014, 2016) (asi onawo DJ Hauser and Schwarz (2015b) DJ Hauser and Schwarz (2015a) ). Kana iwe usingaiti vatori vechikamu vasina kutarisira, ipapo chero chiitiko chekurapa chinogona kushambidzwa kunze neruzha rwavanotanga, uye mukuita kuti nhamba yevasingafungi vatori vechikamu inogona kuva yakakura. Mukuedza kwakaitwa naHuber nevamwe vashandi (2012) , vanenge 30% vevatori vechikamu vakakundikana kunyatsotarisa. Zvimwe zvinetso zvinowanzomuka apo Turkers inoshandiswa nevatori vasiri vasiri (Chandler et al. 2015) uye attrition (Zhou and Fishbach 2016) .

Chechitatu, maererano nedzimwe mhando dzekuedza kwedhijitori, maturk anoedza haagoni kukura; Stewart et al. (2015) inofungidzira kuti panguva ipi zvayo kune vanhu vanenge 7 000 vari muMaturk.

Pakupedzisira, iwe unofanirwa kuziva kuti MTurk inharaunda ine mitemo yayo nemitemo (Mason and Suri 2012) . Nenzira imwe chete yaunogona kuedza kuziva nezvetsika yenyika yawakanga uri kuenda kuzotanga kuedza kwako, iwe unofanira kuedza kuwana zvakawanda pamusoro pemaitiro nemitemo yeTurkers (Salehi et al. 2015) . Uye iwe unofanirwa kuziva kuti vaTurkers vari kutaura pamusoro pekuedza kwenyu kana muchiita chimwe chinhu chisina kukodzera kana chisina maturo (Gray et al. 2016) .

MTurk inzira inonakidza yakanakisisa yekuunganidza vatori vechikamu kuongororo dzenyu, ingave vari-lab-yakafanana, yeiyo Huber, Hill, and Lenz (2012) , kana yakawanda-yakafanana, yakadai Mason and Watts (2009) , Goldstein, McAfee, and Suri (2013) , Goldstein et al. (2014) , Horton and Zeckhauser (2016) , uye Mao et al. (2016) .

• Zvivakire chigadzirwa chako (chikamu 4.5.3)

Kana iwe uchifunga nezvekuedza kuumba iwe pachako chigadzirwa, ndinokurudzira kuti uverenge mazano anopiwa neSvondoLenski Harper and Konstan (2015) . Nzwisiso inokosha kubva pane zvakaitika kwavari ndeyekuti kune imwe yega purogiramu yakabudirira kune dzakawanda, kukundikana kukuru. Semuenzaniso, boka reVideoLens rakatanga mamwe mimwe michina, yakadai seGopherAnswers, iyo yakakundikana zvachose (Harper and Konstan 2015) . Chimwe chiitiko chemutsvakurudzi anokundikana paanenge achiedza kuvaka chigadzirwa ndiEdry Castronova muedzo wekuvaka mutsara wekuInternet unonzi Arden. Pasinei ne $ 250,000 mumari yekubhadhara, purojekiti yacho yaifamba (Baker 2008) . Zvirongwa zvakadai seGopherAnswers naArden zvinosuruvarisa zvakanyanya kufanana nemapurojekiti akafanana neLinemaLens.

• Kugoverana nevane simba (chikamu 4.5.4)

Ndanzwa pfungwa yePasteur's Quadrant inokurukurwa nguva dzose kumakambani ekunyanya, uye inobatsira kuronga mano ekutsvakurudza paGoogle (Spector, Norvig, and Petrov 2012) .

Bond and colleagues 'study (2012) inoedzawo kuona kuitika kwezvirapa izvi kune shamwari dzeavo vakagamuchira. Nemhaka yekugadzirwa kwekuedza, izvi zvinokonzera kuona zvakajeka; Vaverengi vanofarira vanofanira kuona Bond et al. (2012) kuitira hurukuro yakazara. Jones neshamwari dzake (2017) vakaitawo chiitiko chakafanana panguva yekusarudzwa kwe 2012. Izvo zviongorori zvikamu zvetsika yakareba yekuedza mune zvematongerwe enyika nezvekuedza kusimbisa kuvhota (Green and Gerber 2015) . Izvi zvinowanikwa-in-out-the-kura zviongorori zvakajairika, muchidimbu nokuti zviri muPasteur's Quadrant. Izvi zvinoreva kuti pane vanhu vakawanda vanokurudzirwa kuwedzera kuvhota uye kuvhota kunogona kuva mufambiro unofadza kuti uedze dzimwe pfungwa dzakawanda pamusoro pekuchinja kwehutano uye hukama hwevanhu.

Kuti uwane mazano pamusoro pekushanda zvidzidzo nemasangano ehurumende akadai semasangano ezvematongerwe enyika, NGOs, uye mabhizinesi, ona Loewen, Rubenson, and Wantchekon (2010) , JA List (2011) , uye Gueron (2002) . Nokufunga pamusoro pekuti kushamwaridzana nemasangano kunobatsira sei kutsvakurudza maitiro, ona King et al. (2007) uye Green, Calfano, and Aronow (2014) . Kubatana kunogonawo kutungamirira kumibvunzo yakanaka, sezvakakurukurwa Humphreys (2015) Nickerson and Hyde (2016) .

• Kugadzira mazano (chikamu 4.6)

Kana uri kuenda kugadzira chirongwa chekuongorora usati watanga kuedza kwako, ndinokurudzira kuti utange nokuverenga mirayiridzo yekupa. ICHINYORWA (Consolidated Standard Reporting of Trials) mirayiridzo yakagadzirwa mumishonga (Schulz et al. 2010) uye yakashandurwa pakuongorora kwevanhu (Mayo-Wilson et al. 2013) . Nheyo yakafanana yehutungamiri yakagadzirwa nevatori ve Journal of Experimental Political Science (Gerber et al. 2014) (onawo Mutz and Pemantle (2015) Gerber et al. (2015) ). Pakupedzisira, kupa nhungamiro yakagadzirwa mune zvepfungwa (APA Working Group 2008) , uye ona Simmons, Nelson, and Simonsohn (2011) .

Kana iwe ukagadzira chirongwa chekuongorora, iwe unofanira kufungidzira kusati wanyora pasi nekuti pre-kubhadhara ichawedzera chivimbo chekuti vamwe vane mhinduro dzako. Uyezve, kana uri kushanda nemumwe wako, zvinoderedza kukwanisa komumwe wako kuchinja shanduko mushure mokuona zvawanikwa. Pre-kubhadhara iri kuwedzera kuwanzana mune zvepfungwa (Nosek and Lakens 2014) , svesayenzi yezvematongerwo enyika (Humphreys, Sierra, and Windt 2013; Monogan 2013; Anderson 2013; Gelman 2013; Laitin 2013) , uye economics (Olken 2015) .

Kugadziriswa mazano zvakananga kumusangano wekuInternet munyaya Konstan and Chen (2007) uye Chen and Konstan (2015) .

Izvo zvandakadana hurongwa hwe armada dzimwe nguva kunonzi tsvakurudzo yepurogiramu ; ona Wilson, Aronson, and Carlsmith (2010) .

• Gadzira zero data yakasiyana-siyana (chikamu 4.6.1)

Kune zvakawanda pamusoro peMusicLab kuongororwa, ona Salganik, Dodds, and Watts (2006) , Salganik and Watts (2008) , Salganik and Watts (2009b) , Salganik and Watts (2009a) , uye Salganik (2007) . Kana uchida zvakawanda pamusoro pemiti yekukunda-kutora-yose, ona Frank and Cook (1996) . Nokuda kwekuwedzera kusunungura ruzivo uye unyanzvi kakawanda kazhinji, ona Mauboussin (2012) , Watts (2012) , Frank (2016) .

Pane imwe nzira yekugumisa zvikwereti zvevatori vechikamu izvo vatsvakurudzi vanofanira kushandisa nekuchenjerera: kubvumirana. Munzvimbo dzakawanda dzekutsvaga mutsvaga mutsvaga vatori vechikamu vanonyorwa kushandiswa mukuedza uye havatombogadziri. Mienzaniso yemaitiro aya inosanganisira Restivo uye van de Rijt's (2012) kuedza kwezvikomborero mu Wikipedia uye Bond pamwe neshamwari (2012) kuedza kukurudzira vanhu kuvhota. Izvi zvidzidzo hazvina zero mari yakasiyana-asi, dzine zero yakasiyana-siyana kune vatsvakurudzi . Mukuedza kwakadaro, kunyange kana mari iyo mumwe nomumwe ari mudiki ari shoma zvikuru, mari yakawanda inogona kuva yakakura kwazvo. Vatsvakurudzi vari kuita hurukuro yakawanda yekuIndaneti vanowanzoratidzira kukosha kwechidiki chinokonzerwa nehutano hwekurapa nokutaura kuti zviduku izvi zvingave zvakakosha kana zvashandiswa kuvanhu vakawanda. Kunyatsofunga kwakafanana kunoshandiswa kune zvinotengeswa nevatsvakurudzi kune vanotora. Kana kuedza kwako kukonzera mamiriyoni evanhu kuparadza miniti imwe chete, kuedza hakukuvadzi kune chero munhu upi zvake, asi pamwe chete yakarasika anenge makore maviri emakore.

Imwe nzira yekugadzira zero kubhadhara mari yakawanda kune vatori vechikamu ndeye kushandisa lottery, nzira yakashandiswa zvakare mukutsvakurudza kwakaitwa (Halpern et al. 2011) . Nokuda kwekugadzira zviitiko zvinonakidza zvemushandisi, ona Toomim et al. (2011) . Nokuda kwekushandisa bhokuti kuumba zero kushandiswa kwezvinhu zvinodhura kuona ( ??? ) .

• Rongedza, Ratidza, uye Deredza (chikamu 4.6.2)

Iro mitatu yaR sokutanga yakasarudzwa Russell and Burch (1959) ndeyotevera:

"Replacement zvinoreva achigamuchira nokuda vanoziva mhuka insentient zvinhu zvipenyu yepamusorosoro. Rutapudzo zvinoreva zvinoderedza nhamba zvemhuka zvinoshandiswa vawane mashoko ane mari yakapiwa uye zvakarurama. Chakagadziridzwa zvinoreva chero vasanyanye kuitika kana kwokuomerwa zvoutsinye maitirwo kushandisirwa mhuka idzo vachiri kushandiswa. "

Izvo zvitatu zveR R zvandinokurudzira hazvirevi tsika dzinotsanangurwa muchitsauko 6. Asi, dzakanyatsotsanangurwa shanduro imwe yemitemo iyo-inobatsira-kunyanya pakugadzirisa kwevanhu.

Nekutanga kwekutanga R ("kushandurwa"), kuenzanisa kupikisa kwepfungwa (Kramer, Guillory, and Hancock 2014) uye kuongororwa kwepfungwa yekuedza (Lorenzo Coviello et al. 2014) inopa dzidzo dzakawanda pamusoro pezvokutengeserana zvakabatanidzwa pakufamba kubva kuongororo kusvika kuongorori rwezvisikwa (nezvimwe nzira dzinofanana nedzimwe dzinoedza kuongorora zvakaongororwa mune zvisiri nyaya yekuedza; ona chitsauko 2). Mukuwedzera kune zvakanakira maitiro, kushanduka kubva kuongororo kuenda kune zvidzidzo zvisiri izvo zvinoitawo kuti vatsvakurudzi vadzidze kurapwa kwavanenge vasingagoni kutora. Aya maitiro ekutsika uye maitiro anobatsira anouya pamutengo, zvisinei. Nekuongororwa kwechisikigo vatsvakurudzi havasi kudzora kuderera pamusoro pezvinhu zvakadai sokutsvaga vatori vechikamu, kurongedza, uye maitiro ekurapa. Semuenzaniso, imwe miganhu yemvura sekurapa ndeyokuti zvose zvinowedzera positivity uye inoderedza kusagutsikana. Muchidzidzo chekuedza, zvisinei, Kramer nevamwe vaaishanda navo vakakwanisa kugadzirisa positivity nekusava nehanya nekuzvimirira. Iyo nzira yakashandiswa yakashandiswa Lorenzo Coviello et al. (2014) yakanyatsotsanangurwa L. Coviello, Fowler, and Franceschetti (2014) . Nokuda kwekutsvaga kwezvinhu zvakasiyana-siyana, iyo ndiyo nzira inoshandiswa Lorenzo Coviello et al. (2014) , ona Angrist and Pischke (2009) (vasina kutongeka) kana Angrist, Imbens, and Rubin (1996) (akareruka). Nokuda kwekusava nechokwadi kwezvinhu zvakasiyana-siyana, ona Deaton (2010) , uye nokuda kwekutsvaga kwezvinhu zvakasiyana-siyana zvinoshandiswa nezvinhu zvisina simba (mvura isimbi isina simba), ona Murray (2006) . Zvimwe zvinowanzoitika, sumo yakanaka yezviitiko zvepanyama inopiwa Dunning (2012) , Rosenbaum (2002) , ( ??? ) , uye Shadish, Cook, and Campbell (2001) vanopa mazano akanaka pamusoro pekufungidzira zvinokonzerwa nezvinokonzera pasina kuedza.

Mukutaura kwechipiri R ("kugadziridzwa"), pane masayenzi uye maitiro ekutengeserana-offs paunenge uchitarisa kushandura kugadzirwa kweKupikisa kweEmotional kubva kudzivirira zvinyorwa pakuwedzera zvikamu. Semuenzaniso, zvingadaro ndezvokuti inyanzvi yekushandiswa kweNews Feed inoita kuti zvive nyore kwazvo kuita chiyero pane zvivako zvakadzivirirwa panzvimbo pane imwe yavanokurudzirwa (cherechedza kuti kuedza kwekuvhara matanda kunogona kushandiswa sechepamusoro pamusoro peNews Feed system pasina chero chidimbu chekuchinja kwegadziriro yehurongwa). Scientific, zvisinei, dzidziso yakataurirwa neongororo haina kuratidza zvakajeka chimwe chinhu pamusoro peimwe. Zvinosuruvarisa, ini handizivi nezvekutsvakurudza kwakawanda pamusoro pezvakanaka zvekuvhara uye kukurisa zvinyorwa mu News Feed. Uyezve, handina kumboona tsvakurudzo yakawanda pamusoro pokuchenesa maitiro kuitira kuti vasanyanya kukuvadza; chimwe chikamu ndechokuti B. Jones and Feamster (2015) , iyo inofunga nyaya yekuyera kwekutsvaga paIndaneti (chinangwa ini ndinokurukura muchitsauko 6 muhukama nekudzidzazve (Burnett and Feamster 2015; Narayanan and Zevenbergen 2015) ).

Mukutaura kwechitatu yechitatu ("kuderedza"), mazano matsvene ekugadzirisa masimba emagariro anopiwa Cohen (1988) (bhuku) uye Cohen (1992) (chinyorwa), asi Gelman and Carlin (2014) vanopa maonero akasiyana. Pre-treatment covariates inogona kubatanidzwa mukugadzirisa nekuongorora shanho rekuedza; chitsauko 4 Gerber and Green (2012) chinopa chiziviso chakanaka kune nzira mbiri, uye Casella (2008) inopa zvakanyanya kurapa. Nzira dzinoshandisa mashoko aya ehutachiona hwekugadzirisa hutachiona hunowanzozivikanwa kana hunovharwa kuongororwa kwekugadzira maitiro kana kushandiswa kwekuenzanisa maitiro (izwi racho harishandiswi nguva dzose kumativi ose); maitiro aya anowirirana zvakananga neakanyatsogadziriswa sampling maitiro anokurukurwa muchitsauko 3. Ona Higgins, Sävje, and Sekhon (2016) pane zvakawanda pakushandisa zvirongwa izvi mukuedza kukuru. Pre-treatment covariates inogonawo kubatanidzwa mukutsvakurudza. McKenzie (2012) inotsvaga musiyano-mukusiyana-siyana kunoongorora kuongorora kwenyika zvakajeka. Ona Carneiro, Lee, and Wilhelm (2016) kune zvakawanda pamusoro pezvokutengeserana pakati pezvimwe nzira dzekuwedzera zvakakwana mukufungidzira kwemigumisiro yechirwere. Pakupedzisira, paunosarudza kana uchiedza kuisa mapurovariate asati arapa pakusarudza kana kuongorora shanho (kana kuti zvose), kune zvimwe zvishoma zvingafanira kufunga. Munzvimbo iyo vatsvakurudzi vanoda kuratidza kuti havasi "hove" (Humphreys, Sierra, and Windt 2013) , vachishandisa mavariro ekurapa mukati mekugadzirisa nzvimbo inogona kubatsira (Higgins, Sävje, and Sekhon 2016) . Mumamiriro ezvinhu apo vatori vechikamu vanosvika sequentially, kunyanya inowanikwa mundima yekuongorora, kushandiswa kwemashoko epamberi kwehutano mumugwagwa wekugadzira kungave kwakaoma logistically; ona, semuenzaniso, Xie and Aurisset (2016) .

Zvakakosha kuwedzera zvinyorwa zvekuti nei kusiyana-kwe-kusiyana-siyana kungave kwakanyanyisa kubudirira kupfuura kupesana-mu-zvinoreva imwe. Zvakawanda zvekutsvaga paIndaneti zvine kusiyana kwakasiyana-siyana (ona, R RA Lewis and Rao (2015) uye Lamb et al. (2015) ) uye havana kusimba kwenguva. Muchiitiko ichi, kuchinja kwechikwata kuchava nekusiyana kwakasiyana-siyana, kuwedzerwa simba rekuyera kwenhamba. Chimwe chikonzero nei nzira iyi isingashandiswi kakawanda ndeyokuti isati yasvika panguva yemadhora, zvakanga zvisina kujairika kuva nemigumisiro yekutangira. Imwe nzira yakajeka yekufunga pamusoro peizvi ndeyokufungidzira kuedza kuongorora kana chiitiko chekuita muviri chinokonzera kuora muviri. Kana iwe ukatora mutsauko-mu-zvinoreva nzira, iwe unofungidzira uchange uine kuchinja kunokonzerwa nekusiyana kwezviremba muhuwandu hwevanhu. Kana iwe ukaita mutsauko-mukusiyana-siyana, zvisinei, izvo zviitiko zvakasiyana-siyana zvinowanikwa mumasero zvinobviswa, uye iwe unogona kuona nyore nyore kusiyana kunokonzerwa nekurapa.

Pakupedzisira, ndakafunga kuwedzera yechina R: "repurpose". Iko ndiko kuti, kana vatsvakurudzi vanozviwana vaine dhiari dzakawanda dzekuongorora pane zvavanoda kugadzirisa mubvunzo wavo wekutanga wekutsvakurudza, vanofanirwa kudzorerazve data kuti vabvunze mibvunzo mitsva. Somuenzaniso, fungidzira kuti Kramer nevamwe vaishanda navo vakashandisa muongorori wekusiyana-uye-vakazviwana ivo vane deta yakawanda kudarika ivo vaida kugadzirisa mubvunzo wavo wekutsvakurudza. Panzvimbo pokusashandisa iyo data zvakakwana, vangadai vakadzidza ukuru hwemagariro sekushanda kwekutangira kurapwa kwekutaura pfungwa. Sezvakaita Schultz et al. (2007) yakaona kuti migumisiro yekurapa yakanga yakasiyana nevashandi vashoma uye vane simba, zvichida migumisiro yeNews Feed yakanga yakasiyana kune vanhu vakange vagara vachida kutumira mashoko anofadza (kana kusuruvara). Kudzokorora kunogona kutungamirira ku "hove" (Humphreys, Sierra, and Windt 2013) uye "p-hacking" (Simmons, Nelson, and Simonsohn 2011) , asi izvi zvinonyanya kugadziriswa nekubatanidzwa kwekutaura kwechokwadi (Simmons, Nelson, and Simonsohn 2011) , pre-registration (Humphreys, Sierra, and Windt 2013) , uye nzira dzokudzidza dzemichina dzinoedza kudzivisa kudarika.