hou commentary

Keia pauku, ua papahana e, e hoʻohana 'e like me ka olua, i ole e heluhelu ia ka moolelo.

• Introduction (Pauku 4.1)

Ninau e pili ana i causality ma ka lawelawe noiʻi he pinepine eiiieaena a me ka pohihihi. No ka mea, he foundational kokoke i causality muli o causal pakuhi, ike Pearl (2009) , a no ka foundational kokoke loa i kahi muli oʻia nā haumāna,ʻike Imbens and Rubin (2015) (a me ka oaoieei-i paiiaʻi i loko o kēia mokuna). No ka mea, o ka hoʻohālike ma waena o kēia mau hoʻokokoke mai, ike Morgan and Winship (2014) . No ka mea, ke olelo kokoke i ka wā i confounder, ike VanderWeele and Shpitser (2013) .

Ma ka mokuna, aole au i hana i ka mea ua like me ka laina e ao alohilohi i ko makou hiki e hana causal koho mai mana a me ka 'ole-hoʻokolohua aeaiiua ma waena o. I ike maopopo, ke manao nei au o ka hoʻokaʻawale ua blurrier. No ka mea, laʻana, mea lawe maikai nei ia oa ke kumu maʻi 'aʻai nae mea, aole loa makou i hana i kekahi lāʻau' randomized hoʻokolohua i poe koa poe kanaka, e puhi. No ka mea, maikaʻi puke loa kūlana kanaka ma ka hana ana i causal koho ka politika-mana ike ike Rosenbaum (2002) , Rosenbaum (2009) , Shadish, Cook, and Campbell (2001) , a me Dunning (2012) .

Mokuna 1 a me 2 o Freedman, Pisani, and Purves (2007) e kaumaha aku i ka maopopo pōkole i loko o nā oko ao ma waena o ia mea, i kekahi lāʻau 'hoʻokolohua, a randomized lāʻau hoʻokolohua.

Manzi (2012) i kekahi manawa, a me readable pōkole i loko o ka philosophical a ana helu underpinnings o randomized lāʻau hoʻokolohua. It no hoi i nā hana hoihoi maoli-ao examples o ka mana o ke experimentation ma ka hana.

• He aha ia mea? (Pauku 4.2)

Casella (2008) , Box, Hunter, and Hunter (2005) , Athey and Imbens (2016b) hoʻolako maikai hoʻolauna i ke ana helu, ao ke ano o ka hana ho'āʻo a me Ka Ikepili. Eia, he maikai loa nohoʻana o ka hoʻohana 'ana o ia mea i loko o nāʻokoʻa mahinaʻai:? Aneia iainiiaaiea (Bardsley et al. 2009) , Sociology (Willer and Walker 2007; Jackson and Cox 2013) , psychology (Aronson et al. 1989) , Ka manaʻo politika' epekema (Morton and Williams 2010) , a me ka nohona pili (Glennerster and Takavarasha 2013) .

Ka nui o ka i komo Recruitment (e like me, mea hōʻikeʻikeʻuʻuku) he pinepine malalo o-mahalo i ka papa hana noiʻi. Akā naʻe, inā i ke kanawai o ka iapaau ana, ua heterogeneous ma ka heluna kanaka, alaila mea hōʻikeʻikeʻuʻuku, ua noho pilikia. Longford (1999) i keia wahi maopopo ia ia e Uwao no ka noiʻi me ka manao o ka ho okolohua e like me ka heluna kanaka ana me ka haphazard mea hōʻikeʻikeʻuʻuku.

• Elua Ana o na ho okolohua: Keʻena-mahinaʻai, a me Analog-kikohoʻe (Pauku 4.3)

Ka dichotomy ka mea aʻu i hōʻike mai iā ia ma waena o Keʻena a me kula hoʻokolohua mea iki Nohie. I ka mea, ua manaoia na noiʻi hou piha typologies, i ka nui o kēia mau kamaliʻi a hoʻokaʻawale ai i nāʻano likeʻole o ke kula hoʻokolohua (Harrison and List 2004; Charness, Gneezy, and Kuhn 2013) . Eia, aia he elua na ano o na mea hana e lawelaweʻepekema i ole ponoʻia i loko o ke Keʻena a me kula dichotomy:. Anamanaʻo hoʻokolohua, a me ka nohona ho okolohua Survey ho okolohua ua ho okolohua hoʻohana i ka aeuiiai ieaia? O nei ana, ae hoohalike pane i kekahi papa hana wale nō o ka ia nīnau (ua hōʻike mai iā ia kekahi anamanaʻo hoʻokolohua ma ka Mokuna 3); no ka mea, nui ma anamanaʻo ho okolohua ike Mutz (2011) . Social ho okolohua he ho okolohua kahi o ka hana, oia kekahi nohona pili ana wale no e hoʻokō hiki ma ke aupuni. Social ho okolohua ua pili loa aku i ka papa loiloi. No ka mea, hou i pili ia mea, e ike Orr (1998) , Glennerster and Takavarasha (2013) , a me Heckman and Smith (1995) .

A helu o na pepa, ua hoʻohālike Keʻena a me ka mahinaʻai ia mea i loko o ka Hōʻuluʻulu Manaʻo (Falk and Heckman 2009; Cialdini 2009) , a ma na olelo o nā haumāna o nā ho'ākāka 'ia ia mea i loko o pili aupuni science (Coppock and Green 2015) ,? Aneia iainiiaaiea (Levitt and List 2007a; Levitt and List 2007b; Camerer 2011; Al-Ubaydli and List 2013) , a me psychology (Mitchell 2012) . Jerit, Barabas, and Clifford (2013) e kaumaha ana i kiʻi noiʻi manao no ka hoʻohālikelikeʻia nā hualoaʻa mai Keʻena a me ka mahinaʻai ia mea.

Pili e pili ana i komo hoʻololi i ko lakou hana, no ka mea, ike lakou, ua pili loa ke malama i kekahi manawa kapa koi waiwai, a ua aʻo ma ka psychology (Orne 1962) a me na 'aoʻao hoʻokele waiwai (Zizzo 2009) . 'Oiai, ka hapanui pili i Keʻena hoʻokolohua, i keia mau mea nāna e hoʻopuka i ke kumu pilikia no ka mahinaʻai ia mea e like me ka pono. In mea, koi 'ole ua hoi i kekahi manawa kapaia Hawthorne loa, he manawa e loaa mai ka mahinaʻai hoʻokolohua, ua hōʻike hewa ka mea kaulana illumination ho okolohua i hoomaka i ka makahiki 1924 ma ka Hawthorne hana o ka' Āpana uila Company (Adair 1984; Levitt and List 2011) . Na koi waiwai, a me Hawthorn ole ua pili loa aku i ka manaʻo o ka reactive mahele kūkākūkā ma ka Mokuna 2 (e nānā hoʻi Webb et al. (1966) ).

I ka moolelo o ke kula hoʻokolohua, ua hōʻike ma? Aneia iainiiaaiea (Levitt and List 2009) , ka manaʻo politika science (Green and Gerber 2003; Druckman et al. 2006; Druckman and Lupia 2012) , psychology (Shadish 2002) , a me ke kulekele (Shadish and Cook 2009) . Kekahi wahi o ka 'epekema kaiapili kahi kula hoʻokolohua koke lilo koʻikoʻi he lahui ulu. No ka mea, he maikaʻi loiloi 'ia o ka mea hana ma loko? Aneia iainiiaaiea ike Banerjee and Duflo (2009) , a no ka pilikia kūpono ike Deaton (2010) . No ka mea, i ka loiloi 'ana o kēia hana i pili aupuni science ike Humphreys and Weinstein (2009) . Eia ke oki, i ka ethical ho'āʻo pae'āina i kula hoʻokolohua ua kaupaona i nā i pili aupuni 'epekema (Humphreys 2015; Desposato 2016b) a? Acaeoey? Aneia iainiiaaiea (Baele 2013) .

Ma ka mokuna, I? Aaei e mālama '-lapaʻau' ike e hiki ai ke hoʻohana 'ia e hoʻoikaika i ka miomio o aayoaeuiinoe lapaʻau waiwai, aka, aia no kekahi hoopaapaa e pili ana i keia hoʻokokoke mai: Freedman (2008) , Lin (2013) , a me Berk et al. (2013) ; ike Bloniarz et al. (2016) no ka 'ike hou aku.

• Neʻe ma o aku o na mea ho okolohua (Pauku 4.4)

I Fashion wae e kālele ana ma luna o ekolu manaÿo: validity, heterogeneity o ka lapaau waiwai, a me na maomeka. Këia mau manaÿo he okoa inoa i loko oʻano kula. No ka mea, laʻana, psychologists kamaʻilioʻana e holo i kela aoao o na mea hoʻokolohua ma ka hoʻomōhala maluna uao a me na moderators (Baron and Kenny 1986) . I ka manaʻo o ka 'uao i ka mea i pio i ka mea aʻu i kapa aku ai maomeka, a me ka manaʻo o nā moderators ua pio i ka mea aʻu i kapa aku mawaho validity (e like me, e, e okoa ka hopena o ka hoʻokolohua, ina ua holo ia maʻano wahi, i hiki) a me heterogeneity o ka lapaʻau loa ( like, he nā waiwai nui no kekahi poe kanaka ma mua o nā kānaka).

No ka ike maopopo ana o Schultz et al. (2007) hōʻike mai ana i ka nohona theories e hiki ai ke hoʻohana i ka manao pono ka uao ai '. No ka mea, he oi nui hoʻopaʻapaʻa e pili ana i ke kūlana o ka kumumanaʻo i loko o ke hoʻololi i ka hoʻokō uao ai ', e ike Walton (2014) .

• Validity (Pauku 4.4.1)

I ka manaÿo o ka kūloko ma, a mawaho validity ua mua wahahee iloko o Campbell (1957) . Ike Shadish, Cook, and Campbell (2001) no ka mea, he oi aku ka piha moʻolelo, a me ka akahele Anoanoaaiii ana helu hopena validity, na validity,? Ieoaeunoai validity, a mawaho validity.

No ka mea, i ka manaʻo o nā kumuhana pili i ana helu hopena validity i ka ho okolohua ike Gerber and Green (2012) (no ka mea, i ka nohona science kuanaʻike) a Imbens and Rubin (2015) (no ka mea, ke ana helu kuanaʻike). Kekahi kumuhana o ana helu hopena validity e ala mai manaʻoi ma online, kula hoʻokolohua nā nīnūnē e like me computationally efficient epekema no ka mea e pili ana i hilinai aiʻanuʻu me kaukaʻi ikepili (Bakshy and Eckles 2013) .

Na validity hiki e paʻakikī e hamama ai ma luna kula hoʻokolohua. E nānā, no ka mea, laʻana, Gerber and Green (2000) , Imai (2005) , a me Gerber and Green (2005) no ka mea, hoopaapaa e pili ana i ka manaʻo o ka luna 'kula hoʻokolohua e pili ana i ke koho ana. Kohavi et al. (2012) , a me Kohavi et al. (2013) hoʻolako i ka pōkole i loko o nā hana nui loa ia o ka wā mawaena validity ma online, kula hoʻokolohua.

Hana nui, ae hoʻomaopopoʻia me na validity, he pilikia i randomization. ala hanakimi e potentially huai ae hoʻopilikia me ka randomization oia e hoohalike i ka lapaau a me ka hooponopono hui ma observable kāna moʻolelo. I keia ano o ka nane hea kapaʻia he koena kaha. Ike Hansen and Bowers (2008) no ke ana helu kokoke loa i kahi e kaulike E kaha makau i, ae ike Mutz and Pemantle (2015) no ka pili e pili ana i koena, e kaha. No ka mea, laʻana, me he koena huli Allcott (2011) i ike ai aia no kekahi hōʻike i ole hoʻokō i ka randomization, ua pololei ka ekolu o na hoao i loko o kekahi o ka OPower ho okolohua (e nānā Papa 2: kahua 2, 6, a me 8). No ka mea, na hoʻokokoke mai, ike Imbens and Rubin (2015) , ka Mokuna 21.

Other nui pili pili i na validity he: 1) hookahi-mńlualua ole-kaʻaeʻana, ma kahi i pau i loko o ka iapaau ana pūʻulu maoli loaa ka iapaau ana, 2) elua mńlualua ole-kaʻaeʻana, ma kahi i pau i loko o ka lapaau pūʻulu loaa ka lapaau, a me kekahi poe kanaka ma o ka mana pūʻulu loaa i ka iapaau ana, 3) attrition, kahi nā haumāna i ole ana no kekahi i komo, a me 4) 'ākeʻakeʻa, kahi a ka iapaau ana hookahe no maluna iho mai nā kānaka i loko o ka iapaau ana ano i na kanaka, ma ka mana kulana. Ike Gerber and Green (2012) mokuna 5, 6, 7, a me 8 No ka mea, nui ma luna o kela a me keia o ia mau kumuhana.

No ka mea, hou ma? Ieoaeunoai validity, ike Westen and Rosenthal (2003) , a no ka oi ma? Ieoaeunoai validity i ka nui ikepili kumu, Lazer (2015) , a me ka Mokuna 2 o keia buke.

Hana, ao ke ano o ka mawaho validity oia i ke kahua, kahi i 'uao, ua ikea ai. Allcott (2015) i kekahi akahele theoretical a me empirical hana o ke kahua koho bias. Keia mea hilo i hoi kūkākūkā ma Deaton (2010) . A iâ lâkou e ka replicated ma na kahua, ka Home Energy Report 'uao i no hoi ua kū kaʻawale? Anoee e mau noiʻi hui (e like me, Ayres, Raseman, and Shih (2013) ).

• Heterogeneity o ka lapaʻau 'ole (Pauku 4.4.2)

No ka mea, he manaʻo maikaʻi o ka heterogeneity o ka lapaʻau waiwai ma kula hoʻokolohua, ike ka Mokuna 12 o Gerber and Green (2012) . No ka mea, hoʻokomo 'ia' i heterogeneity o ka lapaʻau waiwai ma ka 'inikua hoao ana, ike Kent and Hayward (2007) , Longford (1999) , a me Kravitz, Duan, and Braslow (2004) . Heterogeneity o ka lapaʻau waiwai nui kālele ana ma luna o nä mea ma muli o ka mālama '-kanaka ano. Ina oe e hoihoi i ka heterogeneity muli o kia-lapaʻau nā haumāna, a laila, ua lawa ka nui o luna 'hookokoke mai ana, e like me poʻo kumu a mea stratification (Frangakis and Rubin 2002) ; ike Page et al. (2015) i ka moʻolelo loiloi.

Nui noiʻi kiÿeleo i ka heterogeneity o ka lapaʻau 'ole ka hoʻohana laina regression, aka, polokalamu kele epekema hilinai aku maluna o mīkini aʻo, no ka mea hoʻohālike Green and Kern (2012) , Imai and Ratkovic (2013) , Taddy et al. (2016) , a me Athey and Imbens (2016a) .

Aia no kekahi skepticism e pili ana i loaʻaa o ka heterogeneity o na waiwai, no ka lau e like pilikia, a me "lawaiʻa." Aia hoi, heʻano o ka ana helu hoʻokokoke mai i ke kōkua aae pili e pili ana mau nane (Fink, McConnell, and Vollmer 2014; List, Shaikh, and Xu 2016) . Hanakimi kokoke i pili e pili ana i "lawaia" ua mālama '-kakau ana, i ka mea i lilo increasingly, he pono ole i ka psychology (Nosek and Lakens 2014) , ka manaʻo politika epekema (Humphreys, Sierra, and Windt 2013; Monogan 2013; Anderson 2013; Gelman 2013; Laitin 2013) , a me na 'aoʻao hoʻokele waiwai (Olken 2015) .

Ma ka like ana o Costa and Kahn (2013) ua hiki ke e ua hoʻopili aku i ka poe HI ike wale no ka hapalua o ka hale i loko o ka ho okolohua. Heluhelu hoihoi i loko o nā lāliʻi, a hiki hoʻopilikia me keia Ka Ikepili e kapa aku i ka palapala pepa.

• Maomeka (Pauku 4.4.3)

Maomeka ua incredibly nui, aka, aole lakou huli mai i ka mea paʻakikī i ka hoʻopaʻa haʻawina. Research e pili ana i maomeka pili loa aku i ka hoʻopaʻa 'ana o uao i psychology (aka, ike no hoi VanderWeele (2009) no ka mea, i kūlike loa e like ma waena o nā manaʻoʻelua). Ana helu hoʻokokoke mai i loaa maomeka, e like me ka hele ana iloko o Baron and Kenny (1986) , he loa, he pono ole. Eia naʻe, ua huli aku ia mau kaʻina hana hilinaʻi ma kekahi assumptions ikaika (Bullock, Green, and Ha 2010) , a me koʻu hoomanawanui ana'ku ia wa, he mau maomeka, e like me ka mea i manaʻo ai, ma na wahi, i hiki (Imai and Yamamoto 2013; VanderWeele and Vansteelandt 2014) . Imai et al. (2011) , a me Imai and Yamamoto (2013) e kaumaha aku i kekahi mau iaono? Ienoai ana helu ano. Eia, VanderWeele (2015) e kaumaha ana i ka buke-loa lapaʻau me ka nui o ka nui hualoaʻa, e like me ka hulina kokoke i ka pā lonoaʻili Ka Ikepili.

A hoʻokaʻawale kokoke loa i kahi e kālele ana ma ia mea e hoao mai ae e manipulate i keʻano papa hana pono (e like me, e haawi ana na luina vitamin C). Eia naʻe, i loko o nā 'epekema kaiapili napoo ana no hoi, he pinepine mau maomeka, a he mea paakiki ka manao kūlana kanaka e hoololi hou no ia i kekahi me ka hoʻololi i na mea e ae. Kekahi hoʻokokoke mai e experimentally ua hoakakaia hoololiia'na maomeka ma Imai, Tingley, and Yamamoto (2013) , Ludwig, Kling, and Mullainathan (2011) , a me Pirlott and MacKinnon (2016) .

Eia ke oki, maomeka hoi i ka loihi moolelo i loko o ke akeakamai o ka 'epekema e like me ia i ho'ākāka' ia e Hedström and Ylikoski (2010) .

• E ho ohana i na hana kūpono (Pauku 4.5.1.1)

No ka mea, nui ma ka hoʻohana 'ana o nā leka i haʻawina a me ka hooia haʻawina e ana ka hoʻokae' ike Pager (2007) .

• Kukulu iho i ko oukou mau hoʻokolohua (Pauku 4.5.1.2)

Ka mea, he pono ole ala e Imi i komo i ka ho okolohua e hana nei oukou i ka mea Amazon Mechanical Tureke (MTurk). No ka mea, MTurk mimics, ao ke ano o ka loina Keʻena ho okolohua-ka uku poʻe kānaka e hoʻopiha i hana ia lakou, aole e hana no ka paa ole-nui noiʻi hoʻohana Turkers (na hana ma MTurk) e like me i komo i loko o kanaka kumuhana ho okolohua kūpono i ka wikiwiki a me ka cheaper aeaiiuo ohi ma mua o nā loina i mua hoʻomaka ma-ka pā kula keʻena hoʻokolohua ho okolohua (Paolacci, Chandler, and Ipeirotis 2010; Horton, Rand, and Zeckhauser 2011; Mason and Suri 2012; Rand 2012; Berinsky, Huber, and Lenz 2012) .

Ka nui ikaika o ka ho okolohua me ka i komo pūʻali koa i mea mai MTurk, ua logistical; lakou i hiki noiʻi e Imi i komo koke aku ai e like me ka pono. 'Oiai, Keʻena ho okolohua e lawe hebedoma, e holo a me ka mahinaʻai ia mea ke lawe malama e kau-up, ho okolohua me ka i komo pūʻali koa i mea mai MTurk hiki ke holo i loko o nā lā. No ka mea, laʻana, Berinsky, Huber, and Lenz (2012) ua hiki ke Imi i 400 kanaka i loko o ka hoʻokahi lā e komo i loko o ka 8 minuke hoʻokolohua. Eia, keia i komo hiki ke pūʻali koa i mea no ka mea, virtually kekahi mea (me ana a me ka nuipa a laulima, e like me kūkākūkā ma na Mokuna 3 a 5). Keia noho nanea oe o ka Recruitment o ia hoʻi, ka noiʻi ke holo kaʻina o nā hoʻokolohua ma kokololio succession.

Ma mua o kāna i komo mai MTurk no ko oukou mau ho okolohua, aia he eha nui na mea e ike. Mua, he nui noiʻi i ka pili-kekahi skepticism o ka ho okolohua pili i Turkers. No ka mea, o keia skepticism ka mea, aole i kekahi, he mea paakiki e loan me ka hoike. Eia naʻe, ma hope o kekahi mau makahiki o nā haʻawina me ka hoʻohana 'Turkers, ia kakou ke ano manao nei makou, o keia skepticism ka mea, aole i oi loa aku hoi pono. Ka mea, ua nui haʻawina kapakai a na HI o Turkers i na populations a me na haʻawina hoʻohālikelikeʻia nā hualoaʻa o ka ho okolohua me Turkers i nā hualoaʻa ma kekahi populations. Ua haawiia keia hana a pau, ke manao nei au i ka pono alanui no ka mea, ia oukou e noonoo i ka mea, he mea ia Turkers poe, he kumu pono hāpana, nui e like me nā haumāna aka iki aku like (Berinsky, Huber, and Lenz 2012) . Pela, e like me nā haumāna he he kupono heluna no kekahi aka, aole i papa hana noiʻi, Turkers poe, he kupono heluna no kekahi aka, aole i pau noiʻi. Ināʻoukou e hele aku e hana me Turkers, a laila ka mea i pono, e heluhelu na mea he nui o keia mau haʻawina a me ka hoomaopopo i ko lakou nuances.

Lua, ua ulu noiʻi maikai-hana no ka hoʻoulu na validity o Tureke hoʻokolohua, aʻoukou e aʻo mai e pili ana, ae hahai ia mau mea maikaʻi-hana (Horton, Rand, and Zeckhauser 2011; Mason and Suri 2012) . No ka mea, laʻana, noiʻi me Turkers i hooluoluia mai ai e hoʻohana i iaʻo e hoopau inattentive i komo (Berinsky, Margolis, and Sances 2014; Berinsky, Margolis, and Sances 2016) (aka, ike no hoi DJ Hauser and Schwarz (2015b) a DJ Hauser and Schwarz (2015a) ). Ina aole oe e wehe inattentive i komo, a laila, hiki ke holoi kekahi kanawai o ka iapaau ana mai ma ka walaʻau wahahee mai inattentive i komo, a me ka hana o ka nui o ka inattentive i komo hiki e nui loa. I ka ike maopopo ana Huber, a me nā hoapili (2012) ma kahi o 30% o ka i komo ole walaʻauʻana noonoo iaʻo. Pilikia, he pono ole me Turkers kekahi mea 'ole-naʻaupō i komo (Chandler et al. 2015) .

Kolu, pili ia i kekahi mau ano e ae o kikohoʻe hoʻokolohua, MTurk ho okolohua hiki ole ka pālākiō, Stewart et al. (2015) koho ia i kekahi manawa, aia no wale no ma kahi o 7,000 poe kanaka maluna o MTurk.

Eia ke oki, e ike ia MTurk he kaiāulu a me kona mau rula a me na kūmau (Mason and Suri 2012) . I ka mea a oukou e hoao aku i ka loaa mai e pili ana i ka nohona o ka aina, kahi a oukou i hele e holo i kou ike maopopo ana i na mea ala, oukou e hoao aku i ka loaʻa mai e pili ana i ka moʻomeheu a me kūmau o Turkers (Salehi et al. 2015) . A, oukou e ike ai i ka Turkers e e kamailio ana no ko oukou hoʻokolohua, ina e hana i kekahi mea kūpono 'ole' unethical (Gray et al. 2016) .

MTurk ka mea, he incredibly kupono ala e Imi i komo i ko oukou mau hoʻokolohua, ina paha lakou ua Keʻena-me, e like me Huber, Hill, and Lenz (2012) , a oi mahinaʻai, e like, e like me Mason and Watts (2009) , Goldstein, McAfee, and Suri (2013) , Goldstein et al. (2014) , Horton and Zeckhauser (2016) , a me Mao et al. (2016) .

• Kukulu iho i ko oukou mau huahana (Pauku 4.5.1.3)

Ināʻoe e manaʻo ana ho'āʻo e ho okumu i ko oukou mau huina hoonui, I paipai aku nei ia oukou heluhelu i ka'ōlelo aʻo kaumaha ma ka MovieLens pae ma Harper and Konstan (2015) . A kiʻike mai ko lakou hoao ana, he ia no kela a me keia holomua papahana, ua nui, he nui hoʻolako 'ole. No ka mea, laʻana, na MovieLens hui a hemo aku kekahi huahana e like me GopherAnswers i ua paʻa hoʻolako 'ole' (Harper and Konstan 2015) . Kekahi kumu o ke kanaka noiʻi nele i ho'āʻo e hana i ka huina hoonui o Edward Castronova ke hoao e hana i online no hihiu kapaia Arden. I loko nō $ 250.000 iloko o kālā,ʻo ka papahana he flop (Baker 2008) . Papahana e like me GopherAnswers, a me Arden ua Akā naʻe, ua nui aku, he pono ole ma mua o nā papahana e like me MovieLens. Eia ke oki, i ka wa aʻu i olelo aku ai au i ike i kekahi noiʻi mea i ana kukulu nā huahana no ka mea, hai experimentation maanei no koʻu koi a: 1) i komo hoʻohana i ka huina hoonui no ka mea ua 'oi aku ia lakou (e like me, aole lakou i uku ia a lakou, aole hana manawaleʻa e kōkua science) a me 2) ka huina hoonui ua hoʻohanaʻia no ka oi aku mamua o hookahi okoa hoʻokolohua ( 'o ia hoʻi, aole ka ia hoʻokolohua mau manawa me ka okoa i komo aiaiaiia). Ina oe ike o na ano he kumu hoʻohālike, ke noi aku nei ike.

• Hana like me ka mana (Pauku 4.5.2)

I Fashion lohe i ka manaʻo o ko Pasteur ka Quadrant kūkākūkā pinepine i kaʻenehana poe, a ua kokua hoonohonoho noiʻiʻana ma Google (Spector, Norvig, and Petrov 2012) .

Bond, a me nā hoapili 'kuka nei (2012) Ho'āʻo' o ia no hoi, e huai ae i ke kanawai o kēia nohoʻana ma o na makamaka o ka poe i loaa ia lakou. No ka mea, o ka manao no ia o ka hoʻokolohua, keia mau spillovers ua paʻakikī e huai ae cleanly; hoihoi poʻe heluhelu e ike Bond et al. (2012) no ka mea, ke kūkākūkā hou hoʻokolokolo pono. Keia hoʻokolohua he hapa o ka moolelo loihi o ka ho okolohua i pili 'epekema ma ka pono' ana e paipai i koho ma (Green and Gerber 2015) . Keia mau E-mai-ka-koho hoʻokolohua, he pono ole i ka hapa, no ka mea, iloko o Pasteur ka Quadrant. Ka mea hoi, ua nui na kanaka i ua ohohia e hoomahuahua i ke koho, a me ke koho hiki lilo i hoʻonanea 'ana, e hoao hou nui theories e pili ana i hana loli a me ka nohona keiki.

Other noiʻi, ua hoomakaukau aʻo e pili ana i holo kula hoʻokolohua me ka hoapili ia a papa hana e like me ka pili aoao, NGOs, a me nā pāʻoihana (Loewen, Rubenson, and Wantchekon 2010; List 2011; Gueron 2002) . Kekahi, ua kaumaha aʻo e pili ana i ke kūkaʻi 'ana, me ka hoʻonohonohoʻia hiki hopena noiʻi manao (Green, Calfano, and Aronow 2014; King et al. 2007) . E hiki no hoi hui alakai i ethical nīnau (Humphreys 2015; Nickerson and Hyde 2016) .

• Design aʻo (Pauku 4.6)

Ina e holo ana ho okumu i ka Ka Ikepili papahana mua o holo i kou hoʻokolohua, I paipai nei oukou i hoʻomaka ma ka heluhelu hōʻike 'ana no nā kulekele. O ka hoopili aku la (hoʻohui a hoʻopaʻa Koo hōʻike 'ana no o na hoao ana) lula ua? Ac? Ma ka lapaau (Schulz et al. 2010) a mau ala no ka lawelawe noiʻi (Mayo-Wilson et al. 2013) . A hai kau o na lula, ua hoʻomohala e na me ka luna hoʻoponopono o ka Pai o Hoʻokolohua Kalaiaina Science (Gerber et al. 2014) (ike no hoi Mutz and Pemantle (2015) , a me Gerber et al. (2015) ). Eia ke oki, hoike mai i ka lula ua hoʻomohala ma psychology (Group 2008) , a ua ike no hoi Simmons, Nelson, and Simonsohn (2011) .

Inā e loaʻa mai ana he Ka Ikepili manao ia oukou e noonoo i ka mālama 'ke kau helu keleponaʻana ia mea, no ka mālama'-kakau, e hoomahuahua i na manao pono a pau e ae i loko o koʻoukou mau hualoaʻa. Eia, ina oe e hana ana me he hoalawepu, ia e ko oukou hoa ke akamai e hoololi i ke Ka Ikepili mahope iho o ka ike ana i na hualoaʻa kaupalena. Mālama '-kakau ana, ua lilo increasingly, he pono ole i ka psychology (Nosek and Lakens 2014) , ka manaʻo politika epekema (Humphreys, Sierra, and Windt 2013; Monogan 2013; Anderson 2013; Gelman 2013; Laitin 2013) , a me ka' aoʻao hoʻokele waiwai (Olken 2015) .

Oiai e pili ana i koʻoukou mālama '-Ka Ikepili manao oukou e e makaala i kekahi noiʻi no hoi hoʻohana' regression, a me nā hoʻokokoke mai, e hoʻoikaika i ka miomio o ka aayoaeuiinoe lapaau kanawai, a aia no kekahi hoopaapaa e pili ana i keia hoʻokokoke mai: Freedman (2008) , Lin (2013) , a me Berk et al. (2013) ; ike Bloniarz et al. (2016) no ka 'ike hou aku.

Design aʻo manaʻoi no ka mea, online, kula hoʻokolohua, ua hoi hōʻike ma Konstan and Chen (2007) , a me Chen and Konstan (2015) .

• Ho okumu iʻole iaaanu lilo ikepili (Pauku 4.6.1)

No ka mea, hou maluna o ka MusicLab hoʻokolohua, ike Salganik, Dodds, and Watts (2006) , Salganik and Watts (2008) , Salganik and Watts (2009b) , Salganik and Watts (2009a) , a me Salganik (2007) . No ka mea, nui ma ka uwa-lawe-na aha kanaka, ike Frank and Cook (1996) . No ka mea, e oi aku maluna o untangling pomaikai, a me ke akamai hou nui, ike Mauboussin (2012) , Watts (2012) , a me Frank (2016) .

Aia no kekahi kokoke i eliminating i komo uku i noiʻi e hoʻohana i ka haʻalele: conscription. Ma na online no kula hoʻokolohua i komo ua basically koho i loko o ia mea, a aole loa compensated. 'O kekahi keia hoʻokokoke mai nā Restivo, a me van de Ka Rijt ke (2012) hoʻokolohua ma uku ma Wikipedia a me Bond, a me colleague ka (2012) ma ka hooikaika ana kanaka e koho hoʻokolohua. Mau ho okolohua e, aole anei he 'Aʻohe iaaanu lilo, ua loaaʻole iaaanu lilo i ka noiʻi. 'Aʻole nō naʻe i ka lilo ana o na mea he nui o keia mau mea, ua like liilii i kela a me keia i komo, liilii koina kau mea nui kēia helu o na komo nui ana ke hooloihi i koke. Noiʻi holo papakaukau online, hoao pinepine ho'āpono aku i ka nui o ka poe liilii, ka manaʻo lapaʻau waiwai ma ka olelo ana i keia mau waiwai uuku hiki i ka wa a noi aku i na kanaka he nui lilo nui. Ka mau ia manaʻo pili i na koina a pau noiʻi auhau ia maluna i komo. Ina oukou ho okolohua ke kumu hookahi miliona kanaka, e pepehi i kekahi minuke, i ka hoʻokolohua ka mea, aole loa nā palapala i kekahi kanaka, aka, ma ka 'Ohi ka mea, ua hoomaunauna aku ai i aneane elua makahiki o ka manawa.

Kekahi kokoke i piliʻole iaaanu na lilo uku i ka i komo mea e hoʻohana i ka lottery, he kokoke mai i hoʻohana no hoi ma anamanaʻo noiʻi (Halpern et al. 2011) . Eia ke oki, no ka mea, he nui e pili ana i ke hoʻololi i ka puaʻa hoʻohana-hana ike Toomim et al. (2011) .

• Kuapo, Hoʻohāiki, ae emi (Pauku 4.6.2)

Eia ke kumu wehewehe o ka poe ekolu R, mai Russell and Burch (1959) :

"Hope o ia hoʻi, ka substitution no ka mea, manao ola holoholona kiekie o insentient mea. Ka ho'ēmiʻana o ia hoʻi, ka ho'ēmiʻana i loko o ka heluna o na holoholona hoʻohana 'e loaa' ike no ka haawi nui a me ka miomio. Ano hoohanohano a 'o ia hoʻi kekahi emi iloko o ka incidence a me ka oolea o inhumane kaʻina hana pili i ka poe holoholona a nō ka mea e e hoohana ia. "

Ka ekolu R ka mea au i manao ole 'aʻe' i na kumu mua ethical i ho'ākāka 'ia ma ka Mokuna 6. Akā, ua, he oi elaborated hoʻokolohua kekahi o ia mau kumu-beneficence-ua hōʻike hewa no ka hoonoho ana o kanaka ia mea.

I ka wa e manao 'ehaʻeha o Contagion, aia he ekolu ole-ethical nīnūnē e malama i ka manaʻo i ka wa ka unuhi i kēia ho'āʻo. Mua, he mea i ike i ka mea kiko'ī maoli o ka ho okolohua hoʻohui i ka theoretical kuleana; i na olelo, e haule ana na ninau e pili ana i? ieoaeunoai validity. He mea i maopopo i ka olelo a me ka hopena maikaʻi'ōlelo helu, ua lilo ke anakahi maikaʻi o ka 'ehaʻeha o ana o na i komo no ka 1) ka mea, aole i ike i na olelo a kanaka Mail he anakahi maikai o ko lakou mau Emotions, a me 2) ka mea, aole maopopo i ka nui o kēia mau olelo la ka Ikepili 'ana i kūpono i ka noiʻi hoʻohana ua hiki ke reliably infer Emotions (Beasley and Mason 2015; Panger 2016) . Ma kekahi olelo, aole i lilo kekahi ino ana o ka biased hoailona. Lua, o ka manao a me Ka Ikepili o ka hoʻokolohua e haʻi mai i kekahi mea e pili ana i ka mea i loa ka hopena i hōʻike ( 'o ia hoʻi, ka mea, aohe Ka Ikepili o ka heterogeneity o ka lapaau waiwai) a me ka mea a keʻano papa hana e lilo. Ma keia hihia, ka noiʻi he nui o ka 'ike e pili ana i ka poe i komo, aka, uaʻano nuiʻia e like me hāmeʻa ma ke Ka Ikepili. Ekolu, ka nui kanawai ma keia ho okolohua, ua uuku loa; i ka likeʻole o ka iapaau ana a me ka hooponopono ana hoi ma kahi o 1 ka 1,000 mau olelo. Iloko o ko lakou mau pepa, Kramer a me nā hoapili i ka hihia a pau i kanawai o keia nui, he nui no ka haneri o na miliona o na kanaka e nānā i ko lakou News E hanai oe i kela la keia la. Ma na olelo e, ka mea hoʻopaʻapaʻa i hiki ole i ka poe uuku no kela a me keia kanaka ka mea, ua nui ka 'Ohi. Ina oe e lawe mai i kekahi manaʻo hoʻopiʻi, he mea naʻe i maopopo, inā he kanawai o keia nui, he nui e pili ana i ka 'epekema nīnau hou loa e pili ana' ehaʻeha o contagion. No ka mea, nui maluna o na wahi, i hiki ai liʻiliʻi 'ole he nui ike Prentice and Miller (1992) .

I olelo o ka R mua (hope), ke kapakai a ka 'ehaʻeha o Contagion hoʻokolohua (Kramer, Guillory, and Hancock 2014) , a me ka' ehaʻeha o contagion maoli hoʻokolohua (Coviello et al. 2014) e kaumaha ana i kekahi mau aʻo e pili ana i ka oihana-mua aʻe nei pae'āina i neʻe mai ho okolohua i maoli ho okolohua (a me na hoʻokokoke mai e like'ālike ia ho'āʻoʻana i ka ho okolohua He kokekau wale ma 'ole-mana aeaiiua, ike Mokuna 2). Ma waho aʻeo ka ethical pomaikai, ka hoʻololiʻana mai ka mana i ole-papa hana haʻawina no hoi mau hiʻohiʻona nō noiʻi e hoʻopaʻa haʻawina kūlana kanaka a lakou i ua logistically hiki ia lakou ke deploy. Keia mau ethical a logistical pomaikai e hele mai i ka waiwai e lilo aku, nae. Me maoli ho okolohua noiʻi i emi hooponopono maluna o na mea e like me Recruitment o na komo, randomization, a me ke ano o ka lapaau. No ka mea, laʻana, hookahi i kaupalena 'ii iai e like me ka hana ka mea ia ia na maoli he positivity a emi negativity. Ma ka mana o? Anoee, nae, ua hiki ke hoʻomaʻa positivity a me ka negativity kū kaʻawale Kramer a me nā hoapili.

Ka nui o kēia mau kokoke loa i kahi hoʻohanaʻia Coviello et al. (2014) ua hou elaborated ma Coviello, Fowler, and Franceschetti (2014) . No ka mea, i hoike no vocal aiaiiuo ike Angrist and Pischke (2009) (emi olelo) a Angrist, Imbens, and Rubin (1996) (oi olelo). No ka mea, he-Liloa appraisal o vocal aiaiiuo ike Deaton (2010) , a no ka mea, i hoike no vocal aiaiiuo me ka nawaliwali kani (ua mea he nawaliwali mea kani), ike Murray (2006) .

More nui, he maikai hoike i maoli ho okolohua he Dunning (2012) , a me Rosenbaum (2002) , Rosenbaum (2009) , a me Shadish, Cook, and Campbell (2001) kaumaha maikai manaʻo e pili ana i aayoaeuiinoe causal loa me ka ho okolohua.

I olelo o ka lua o ka R (hoohanohano a), he poe akeakamai, a me logistical kalepa-aʻe nei, i ka wa e noonoo pono hoʻololi i ka manao o ka 'ehaʻeha o Contagion mai ka wahi pou i boosting pou. No ka mea, laʻana, malia paha e hana i ka hana a ka oaoieei-manaʻo o ka News Feedʻo ia ka mea ua 'ano hiki pono o ka hana hoao i wahi pou mamua hoao me boosting pou (e hoailona oukou i hoao me ka wahi pou hiki ke hoʻokō e like me ka ahu iho ma luna o ka News Feed 'ole kekahi e pono ai no ka hoololi ana i ka nń ku System). Scientifically, nae, ke kumumanaʻo i hōʻike 'ia ma ka hoʻokolohua, aole i maopopo paipai kekahi manao ma luna o ka poʻe kekahi.

Eia naʻe,ʻO wau nō i kaʻike i nā lula ma mua noiʻi e pili ana i ka pili hana maikai la a wahi a boosting maʻiʻo ma ke News Feed. Hoi, aole au i ike ole ai i ka noiʻi e pili ana i hoohehee maemae au ia kūlana kanaka e hana lakou i emi mai nā palapala; Hoʻokahi nō hoʻokoe 'o Jones and Feamster (2015) , ka mea e hoomanao i ka hihia o ke ana o ka Internet censorship (he kumuhana I kamaʻilio ma ka Mokuna 6 i pili i ka Encore hoʻopaʻa (Burnett and Feamster 2015; Narayanan and Zevenbergen 2015) ).

I olelo o ka R kolu (ka ho'ēmiʻana), he maikai hoike i nā loina mana Ka Ikepili o Cohen (1988) . Mālama '-lapaʻau covariates hiki ke komo iloko o ke manao ke kahua, a me ka Ka Ikepili kahua o nā hoʻokolohua; Mokuna 4 o Gerber and Green (2012) 'oi aku ka maikai hoike i na hoʻokokoke mai, a me Casella (2008) ' oi aku ka nui ma-hohonu lapaau. 'Ana i kūpono a pau e hoʻohana i kēia mālama'-lapaau 'ike i loko o ka randomization ua nō kāu kapaʻia kekahi wahi mana manao, a stratified mana mau manao (ka mea i hoʻohana consistently ka terminology ma ka noho' ana); keia mau 'ana i kūpono, ua nui loa aku i ka stratified mea hōʻikeʻikeʻuʻuku' ana i kūpono kūkākūkā i loko o ka Mokuna 3. E nānā Higgins, Sävje, and Sekhon (2016) no ka oi ma ka hoʻohana 'ana i kēia mau manao ma ka papakaukau hoʻokolohua. Mālama '-lapaʻau covariates no hoi e hiki ke komo iloko o ke Ka Ikepili ke kahua. McKenzie (2012) kaupaona i nā ke koena unuhi-ma-oko kokoke i ke kālailai kula hoʻokolohua ma nui au mamuli. Ike Carneiro, Lee, and Wilhelm (2016) no ka mea oi maluna o ka oihana-mua aʻe nei ma waena o okoa hoʻokokoke mai, e mahuahua miomio i koho o ke kanaka waiwai. Hope, ina paha e hoao e komo mālama '-lapaau covariates ma ka manao a me Ka Ikepili ke kahua (ai' ole 'elua), i ka wa ka hoʻoholo', he he he mau kumu e noonoo. Ma ka 'okoʻa noiʻi makemake kahi e hoike ia lakou, aole "lawaiʻa" (Humphreys, Sierra, and Windt 2013) , me ka mālama'-lapaau covariates ma ka manao ke kahua e hiki ai e kōkua (Higgins, Sävje, and Sekhon 2016) . In wahi, i hiki kahi i komo hiki sequentially, o ka oi aku online, kula hoʻokolohua, me ka mālama '-lapaau' ike i loko o ka manao ke kahua e paʻakikī logistically, ike no ka hoʻohālike Xie and Aurisset (2016) .

Pono ke kuai ohui iki o ke kilo no ke aha la oko-i-oko ao ke e no ka nui loa aku kona mana mamua oko-i-mea. He nui na online no nā haumāna i kiʻekiʻe loa ku ei (nānā IAOEIAaO, Lewis and Rao (2015) , a me Lamb et al. (2015) ) a ua kŘpa a paa ma luna o ka manawa. Ma keia hihia, ke hoʻololi 'manual e loaa kūpono kou uuku ei, mahuahua i ka mana o ke ana helu hōʻike. keia kokoke mea i hoʻohana 'oi aku pinepine hookahi kumu no ia ma mua i ka mīkini elemakule, he mea i pono ole ai e loaa mālama'-lapaau nā haumāna. A oi pōhaku punaʻano a pau e manao ana he mea e manao wale hoao e ana ina kekahi hoounauna oaeouee kumu no ke kaumaha me keia poho. Ina oe e hana i ka oko-i-mea e hoʻokokoke mai, ko oukou olelo, e loaa variability e hele mai ana mai ka variability i loko o paona, ma o ka heluna kanaka. Ina e hana kekahi oko-i-like kokoke loa i kahi, nae, ka mea maoli i hana oko a ma ka paonaʻia wehe 'a ia oe e hiki loa ke puana huai ae hookaawale i ma ka iapaau ana.

Hanakimi i emi ka nui o na i komo iloko o ko oukou hoʻokolohua nuiʻaoʻao mea e alakai i ka mana Ka Ikepili, a Kramer a me nā hoapili hiki i hana ma muli o ka nui maoli a kanawai i ka malamaia, mai ka hoʻokolohua maoli ma Coviello et al. (2014) ai 'ole mamua ole-papa hana noiʻi ma Kramer (2012) (ma ka mea mau, he mau hana ma ka hopena o kēia mokuna). Kaʻina Hana i keia hoʻohanaʻana i ka mana Ka Ikepili he iki okoa mamua o KOLOHE. Ma ka Analog makahiki, noiʻi nui i hana mana Ka Ikepili e mālama pono ana i ko lakou kuka nei, aole loa ka nui ( 'o ia hoʻi, ma lalo o-mana). Ano, naʻe, noiʻi e hana mana Ka Ikepili e mālama pono ana i ko lakou kuka nei, aole he nui ( 'o ia hoʻi, ma luna-mana).

Eia ke oki, I noonoo ohui i ka ha R: Repurpose. Ka mea, ina noiʻi loaa ia lakou iho me ka oi mana ike mamua o ko lakou pono e kamaʻilio ko lakou mau hoa noiʻi nīnau, e repurpose i nā ikepili e ninau hou nīnau. No ka mea, kumu, ke manao wale e Kramer, a me nā hoapili i hoʻohana paleʻana ma-ma-oko ao estimator, a loaa ia lakou iho me ka oi aeaiiua mamua e pono e kamaʻilio i ko lakou noiʻi nīnau. E aho ia mamua, aole me ka 'ikepili i ka kuhelu, ia lakou i hiki i aʻo i ka nui o ke kanawai, e like me kekahi papa e mālama'-kanaka 'ehaʻeha o ka manaʻo. E like me ka Schultz et al. (2007) loaa ai ke kanawai o ka iapaau ana heʻokoʻa no ka mālamalama, a me nā mea hoʻohana kaumaha, paha i ka hopena o ka News Feed heʻokoʻa ka poʻe iʻohana mua e Mail hauʻoli (ai 'ole ke kaumaha) memo. Repurposing hiki alakai i "lawaia" (Humphreys, Sierra, and Windt 2013) a "pa-hacking" (Simmons, Nelson, and Simonsohn 2011) , aka, i keia, ua pili kēia addressable me ka huihui 'ana o ka pono mai, a hōʻike (Simmons, Nelson, and Simonsohn 2011) , ka mālama 'kakau ana (Humphreys, Sierra, and Windt 2013) , a me ka mīkini aʻo epekema i hoao ai e pale aku maluna-pono.