Ọzọkwa nkọwa

Ngalaba a na-iji ike mee ka a kwuru okwu ya, kama na-agụ dị ka a kọrọ.

• Okwu Mmalite (Nkebi 4.1)

Ajụjụ banyere causality na-elekọta mmadụ nnyocha na-abụkarị ndị mgbagwoju na mgbagwoju anya. N'ihi na a foundational obibia causality dabeere na causal grafụ,-ahụ Pearl (2009) , na n'ihi na a foundational obibia dabeere nwere ga esi, na-ahụ Imbens and Rubin (2015) (na teknuzu odide ntụkwasị ke ibuot emi). N'ihi na a tụnyere n'etiti abụọ ndị a na-eru nso, na-ahụ Morgan and Winship (2014) . N'ihi na a iwu obibia akọwapụta a confounder,-ahụ VanderWeele and Shpitser (2013) .

Na isi, m kere ihe yiri a na-egbuke egbuke dị n'etiti anyị inwe ike ime ka causal atụmatụ e mere si ibuo na-abụghị nnwale data. N'eziokwu, m na-eche na ihe dị iche na blurrier. Dị ka ihe atụ, onye ọ bụla na-anabata na ise siga na-akpata cancer ezie na anyị emetụbeghị a enweghị usoro na-achịkwa nnwale na-amanye ndị mmadụ ka na-aṅụ sịga. N'ihi magburu onwe n'akwụkwọ ogologo agwọ ọrịa na-eme causal atụmatụ e mere si na-abụghị ibuo data ahụ Rosenbaum (2002) , Rosenbaum (2009) , Shadish, Cook, and Campbell (2001) , na Dunning (2012) .

Isi nke 1 na nke 2 nke Freedman, Pisani, and Purves (2007) na-enye a doro anya iwebata n'ime ọdịiche dị n'etiti ndị nwere, na-achịkwa nwere, na enweghị usoro-achịkwa nwere.

Manzi (2012) na-enye a na-adọrọ mmasị na nke ogugu iwebata n'ime nkà ihe ọmụma na mgbakọ na mwepụ siruru n'ike nke enweghị usoro na-achịkwa nwere. Ọ na-akpali real-ụwa ihe atụ nke ike nke experimentation na azụmahịa.

• Gịnị bụ ihe ndị nwere? (Nkebi 4.2)

Casella (2008) , Box, Hunter, and Hunter (2005) , Athey and Imbens (2016b) na-enye ezi okwu mmeghe na mgbakọ na mwepụ akụkụ nke ibuo imewe na analysis. Ọzọkwa, e nwere ndị magburu onwe agwọ ọrịa nke eji nwere dị iche iche ubi: akụnụba (Bardsley et al. 2009) , sociology (Willer and Walker 2007; Jackson and Cox 2013) , akparamaagwa (Aronson et al. 1989) , ndọrọ ndọrọ ọchịchị ọkà mmụta sayensị (Morton and Williams 2010) , na-elekọta mmadụ iwu (Glennerster and Takavarasha 2013) .

Mkpa nke iso na mbanye (eg, nlele) bụ mgbe n'okpuru-esịtekọm ke ibuo nnyocha. Otú ọ dị, ọ bụrụ na mmetụta nke ọgwụgwọ bụ heterogeneous na ndị bi na, mgbe ahụ, ụfọdụ bụ oké egwu. Longford (1999) na-eme ka a doro anya na mgbe ọ ekwu ná nkwado maka eme nnyocha na-eche echiche nke nwere ka a bi nnyocha e mere na haphazard nlele.

• Abụọ akụkụ nke nnwale: lab-ubi na analọg-dijitalụ (Nkebi 4.3)

The dichotomy na m ọkọnọ n'etiti ụlọ nyocha na ubi nwere bụ a bit mfe. N'ezie, ndị ọzọ na-eme nnyocha chọrọ ọzọ zuru ezu typologies, akpan akpan ndị na-ekewapụ ndị dị iche iche nke ubi nwere (Harrison and List 2004; Charness, Gneezy, and Kuhn 2013) . Ọzọkwa, e nwere abụọ ọzọ ụdị nwere rụrụ site na-elekọta mmadụ ọkà mmụta sayensị na-ekwesịghịkwa neatly n'ime ụlọ nyocha na ubi dichotomy: nnyocha e mere nnwale na-elekọta mmadụ nwere. Survey nwere na-nwere iji akụrụngwa nke ẹdude nnyocha ndị e mere na tụnyere Ibọrọ ọzọ nsụgharị nke otu ajụjụ (ụfọdụ nnyocha e mere nnwale na-ọkọnọ ke Ibuot 3); n'ihi na ihe on nnyocha e mere nnwale ahụ Mutz (2011) . Social nwere ndị nwere ebe ọgwụgwọ bụ ụfọdụ na-elekọta mmadụ iwu na naanị ike emejuputa atumatu oru site a ọchịchị. Social nwere na-nwere njikọ chiri anya Mmemme nwale. Maka ozi on iwu nwere, lee Orr (1998) , Glennerster and Takavarasha (2013) , na Heckman and Smith (1995) .

A ọnụ ọgụgụ nke akwụkwọ nwere tụnyere ụlọ nyocha na ubi nwere nkịtị (Falk and Heckman 2009; Cialdini 2009) na na okwu nke ga esi kpọmkwem nwere na ndọrọ ndọrọ ọchịchị ndị ọkà mmụta sayensị (Coppock and Green 2015) , akụnụba (Levitt and List 2007a; Levitt and List 2007b; Camerer 2011; Al-Ubaydli and List 2013) na akparamaagwa (Mitchell 2012) . Jerit, Barabas, and Clifford (2013) awade a mara mma research imewe n'ihi atụnyere nsonaazụ ndị si ụlọ nyocha na ubi nwere.

Nchegbu banyere sonyere agbanwe àgwà ha n'ihi na ha ma na ha na-anya hụrụ na-akpọ mgbe ụfọdụ ina mmetụta, na ha na e mụọ na akparamaagwa (Orne 1962) na akụnụba (Zizzo 2009) . Ọ bụ ezie ukwuu n'ime metụtara ụlọ nyocha nwere, ndị a otu ihe ahụ okwu nwere ike ime ka nsogbu maka ubi nwere dị ka nke ọma. N'ezie, ina mmetụta na-akpọ mgbe ụfọdụ Hawthorne mmetụta, a okwu na-erite site a ubi nnwale, kpọmkwem na ama ìhè nwere na malitere na 1924 na Hawthorne Works nke Western Electric Company (Adair 1984; Levitt and List 2011) . Ma ina mmetụta na hawthorn mmetụta na-nwere njikọ chiri anya echiche nke reactive mmesho anyị tụlere n'Isi nke 2 (lee kwa Webb et al. (1966) ).

The akụkọ ihe mere eme nke ubi nwere kọwawo na akụnụba (Levitt and List 2009) , ndọrọ ndọrọ ọchịchị ọkà mmụta sayensị (Green and Gerber 2003; Druckman et al. 2006; Druckman and Lupia 2012) , akparamaagwa (Shadish 2002) , na ọha na eze iwu (Shadish and Cook 2009) . Otu n'ime mmadụ sayensị ebe ubi nwere ngwa ngwa ghọrọ ma ama bụ mba development. N'ihi na a mma nyochaa nke na-arụ ọrụ n'ime akụnụba-ahụ Banerjee and Duflo (2009) , na n'ihi na a oké egwu ntule ahụ Deaton (2010) . N'ihi na a nyochaa nke ọrụ a na ndọrọ ndọrọ ọchịchị ndị ọkà mmụta sayensị na-ahụ Humphreys and Weinstein (2009) . N'ikpeazụ, usoro ziri ezi ịma aka ndị aka na ubi nwere na e enyoba na ndọrọ ndọrọ ọchịchị ndị ọkà mmụta sayensị (Humphreys 2015; Desposato 2016b) na mmepe akụnụba (Baele 2013) .

Na isi, m tụrụ aro na tupu ọgwụgwọ ọmụma a pụrụ iji melite nkenke nke na-eme atụmatụ ọgwụgwọ mmetụta, ma e nwere ụfọdụ arụmụka banyere nke a obibia: Freedman (2008) , Lin (2013) , na Berk et al. (2013) ; -ahụ Bloniarz et al. (2016) maka ozi ndị ọzọ.

• Akpụ akpụ n'ofè mfe nwere (Nkebi 4.4)

M na-họọrọ ka-elekwasị anya na atọ banye n'eluigwe: ndaba, heterogeneity ọgwụgwọ mmetụta, na usoro. Ndị a echiche nwere aha dị iche iche na iche iche ubi. Dị ka ihe atụ, ndị ọkà n'akparamàgwà mmadụ na-akpali n'ofè mfe nwere site n'ilekwasị anya ka ndị ogbugbo na moderators (Baron and Kenny 1986) . Echiche nke ndi ogbo na-weghaara ihe m na-akpọ usoro, na echiche nke moderators na-weghaara ihe m na-akpọ mpụga ndaba (eg, ga-arụpụta ọtụtụ ihe nnwale dị iche iche ma ọ bụrụ na ọ na-agba ọsọ dị iche iche ọnọdụ) na heterogeneity ọgwụgwọ mmetụta ( eg, bụ mmetụta ibu n'ihi na ụfọdụ ndị karịa ndị ọzọ).

The nnwale nke Schultz et al. (2007) na-egosi otú mmadụ chepụtara ike ga-eji na-chepụta irè adọ. N'ihi na a ọzọ n'ozuzu arụmụka banyere ọrụ dịịrị onye nke tiori na emebe irè adọ, na-ahụ Walton (2014) .

• Ndaba (Nkebi 4.4.1)

Echiche nke esịtidem na mpụga ndaba e mbụ ẹkenam ke Campbell (1957) . Lee Shadish, Cook, and Campbell (2001) n'ihi na a ọzọ zuru ezu n'akụkọ ihe mere eme na a ịkpachara anya elaboration nke mgbakọ na mwepụ ọgwụgwụ ndaba, esịtidem ndaba, rụọ ndaba, na mpụga ndaba.

N'ihi na ihe nnyocha nke metụtara mgbakọ na mwepụ ọgwụgwụ ndaba na nnwale ahụ Gerber and Green (2012) (n'ihi na a na-elekọta mmadụ na sayensị n'ọnọdụ) na Imbens and Rubin (2015) (n'ihi na a mgbakọ na mwepụ n'ọnọdụ). Ụfọdụ mbipụta nke mgbakọ na mwepụ ọgwụgwụ ndaba na-ebilite kpọmkwem na online ubi nwere agụnye mbipụta dị ka computationally oru oma ụzọ maka eke obi ike etiti oge na-dabeere data (Bakshy and Eckles 2013) .

Esịtidem ndaba pụrụ isi ike iji hụ na mgbagwoju ubi nwere. Lee ihe atụ, Gerber and Green (2000) , Imai (2005) , na Gerber and Green (2005) n'ihi na arụmụka banyere mmejuputa a mgbagwoju ubi nnwale banyere ịtụ vootu. Kohavi et al. (2012) na Kohavi et al. (2013) na-enye okwu mmeghe n'ime ihe ịma aka nke nkeji ndaba na online ubi nwere.

One nchegbu bụ isi na esịtidem ndaba bụ nsogbu na randomization. Otu ụzọ isi ike nwere ike ịchọpụta nsogbu na randomization bụ ịmata ma ọgwụgwọ na ịchịkwa iche iche na ịhụ anya àgwà. Nke a na ụdị tụnyere a na-akpọ a itule ego. Lee Hansen and Bowers (2008) n'ihi na a mgbakọ na mwepụ obibia n'ọnọdụ kwesịrị ekwesị ndenye ego, na-ahụ Mutz and Pemantle (2015) maka nchegbu banyere itule ndenye ego. Dị ka ihe atụ, na-eji a itule elele Allcott (2011) chọpụtara na e nwere ụfọdụ na-egosi na randomization e emejuputa atumatu n'ụzọ ziri ezi na atọ nke nwere ụfọdụ nke OPower nwere (lee Isiokwu 2; saịtị 2, 6, 8). N'ihi na ndị ọzọ-eru nso, na-ahụ Imbens and Rubin (2015) , Chapter 21.

Ndị ọzọ na isi na-akpata nchegbu metụtara esịtidem ndaba bụ: 1) otu kwadoro na-abụghị nnabata, ebe bụghị onye ọ bụla na ọgwụgwọ otu n'ezie natara ọgwụgwọ, 2) abụọ kwadoro na-abụghị nnabata, ebe bụghị onye ọ bụla na ọgwụgwọ otu na-enweta ọgwụgwọ na ụfọdụ ndị mmadụ na-achịkwa otu enweta ọgwụgwọ, 3) attrition, ebe a ga esi na-adịghị tụrụ maka ụfọdụ sonyere, na 4) nnyonye anya, ebe ọgwụgwọ akwafuo n'elu si ndị mmadụ na ọgwụgwọ ọnọdụ ka ndị mmadụ na-achịkwa ọnọdụ. Lee Gerber and Green (2012) Isi nke 5, 6, 7, na nke 8 n'ihi na ihe ọ bụla nke ndị a okwu.

Maka ozi on mmepụta ya ndaba,-ahụ Westen and Rosenthal (2003) , na n'ihi na ihe on mmepụta ya ndaba na nnukwu data isi mmalite, Lazer (2015) na Isi nke 2 nke akwụkwọ a.

Otu akụkụ nke mpụga ndaba bụ ọnọdụ ebe ihe aka a na-anwale. Allcott (2015) na-enye a akpachara anya n'ọnụ na ahụrụ anya ọgwụgwọ nke saịtị nhọrọ echiche ọjọọ. A nke a na-atụle n'isiokwu na Deaton (2010) . Na mgbakwunye na-replicated na ọtụtụ ndị na saịtị, na Home Energy Report aka nwekwara e n'adabereghị mụọ site na otutu nnyocha iche iche (eg, Ayres, Raseman, and Shih (2013) ).

• Heterogeneity ọgwụgwọ mmetụta (Nkebi 4.4.2)

N'ihi na ihe ndị magburu onwe nnyocha nke heterogeneity ọgwụgwọ mmetụta na ubi nwere, gụọ Isi nke 12 nke Gerber and Green (2012) . N'ihi okwu mmeghe na-heterogeneity ọgwụgwọ mmetụta na ọgwụ ọnwụnwa,-ahụ Kent and Hayward (2007) , Longford (1999) , na Kravitz, Duan, and Braslow (2004) . Heterogeneity ọgwụgwọ mmetụta n'ozuzu elekwasị anya na iche dabere na tupu ọgwụgwọ e ji mara. Ọ bụrụ na i nwere mmasị na heterogeneity dabeere na post-omume ga esi, mgbe ahụ, ihe mgbagwoju approachs na-mkpa dị ka isi stratification (Frangakis and Rubin 2002) ; -ahụ Page et al. (2015) n'ihi na a nyochaa.

Ọtụtụ ndị nnyocha na-eme atụmatụ na heterogeneity ọgwụgwọ mmetụta eji linear nlaghachi azụ, ma ọhụrụ ụzọ ịdabere na igwe mmụta ihe atụ Green and Kern (2012) , Imai and Ratkovic (2013) , Taddy et al. (2016) , na Athey and Imbens (2016a) .

E nwere ụfọdụ nwere obi abụọ, banyere nchọpụta ndị nke heterogeneity nke mmetụta n'ihi multiple tụnyere nsogbu na "igbu azụ." E nwere a dịgasị iche iche nke mgbakọ na mwepụ na-eru nso nke nwere ike inyere adreesị nchegbu banyere multiple tụnyere (Fink, McConnell, and Vollmer 2014; List, Shaikh, and Xu 2016) . One obibia nchegbu banyere "ịkụ azụ" bụ tupu registration, nke na-aghọ esiwanye nkịtị na akparamaagwa (Nosek and Lakens 2014) , ndọrọ ndọrọ ọchịchị ọkà mmụta sayensị (Humphreys, Sierra, and Windt 2013; Monogan 2013; Anderson 2013; Gelman 2013; Laitin 2013) , na akụnụba (Olken 2015) .

Ke ọmụmụ ndị Costa and Kahn (2013) naanị ihe dị ka ọkara nke ndị ezinụlọ na nnwale ndị ike ike jikọrọ ndị ozi igwe mmadụ. -Agụ akwụkwọ nwere mmasị na nkọwa na o kwere omume nsogbu na nke a analysis kwesịrị ezo aka na mbụ akwụkwọ.

• Usoro (Nkebi 4.4.3)

Usoro bụ incredibly mkpa, ma, ha si gaa na-adị nnọọ ike ịmụ ihe. Research banyere usoro njikọ chiri anya ọmụmụ nke ndị ogbugbo na akparamaagwa (ma lee kwa VanderWeele (2009) n'ihi na a kpọmkwem tụnyere n'etiti abụọ echiche). Statistical-abịaru nso ka achọta usoro, dị ka obibia mepụtara na Baron and Kenny (1986) , bụ nnọọ nkịtị. O di nwute na, ọ na-enyo na ndị na usoro adabere na ụfọdụ dị ike echiche (Bullock, Green, and Ha 2010) na-ata ahụhụ mgbe e nwere otutu usoro, dị ka otu nwere ike na-atụ anya ọtụtụ mgbe, (Imai and Yamamoto 2013; VanderWeele and Vansteelandt 2014) . Imai et al. (2011) na Imai and Yamamoto (2013) na-enye ụfọdụ mma mgbakọ na mwepụ usoro. Ọzọkwa, VanderWeele (2015) awade a akwụkwọ-ogologo ọgwụgwọ na a ọnụ ọgụgụ nke dị mkpa results, gụnyere a keukwu obibia uche analysis.

A iche iche obibia na-elekwasị anya nwere na-agbali iji megharịa usoro ozugbo (eg, na-enye ọrụ ụgbọ mmiri na vitamin C). N'ụzọ dị mwute, ọtụtụ ndị na-elekọta mmadụ na sayensị ntọala e nwere ọbụna ọtụtụ usoro na ọ bụ ike chepụta agwọ ọrịa na-agbanwe onye na-enweghị na-agbanwe ndị ọzọ. Ụfọdụ na-eru nso ka experimentally mejupụtara usoro na-kọwara na Imai, Tingley, and Yamamoto (2013) , Ludwig, Kling, and Mullainathan (2011) , na Pirlott and MacKinnon (2016) .

N'ikpeazụ, usoro na-enwe ogologo akụkọ ihe mere eme na nkà ihe ọmụma nke sayensị ka a kọwara site Hedström and Ylikoski (2010) .

• Iji ẹdude gburugburu (Nkebi 4.5.1.1)

Maka ozi na ojiji nke ozi ọmụmụ na oditi ọmụmụ tụọ ịkpa ókè-ahụ Pager (2007) .

• -Ewu gị onwe gị na nnwale (Nkebi 4.5.1.2)

Kacha nkịtị ụzọ n'oru sonyere na nwere na ị na-ewu bụ Amazon Mechanical Turk (MTurk). N'ihi na MTurk mimics akụkụ nke omenala lab nwere-akwụ ụgwọ ndị mezue ihe aga-eme na ha agaghị eme maka free-ọtụtụ nnyocha maliteworị iji Turkers (ndị ọrụ on MTurk) dị ka sonyere na ụmụ mmadụ na-achị nwere dapụtara na ngwa ngwa na dị ọnụ ala karịa data collection ka omenala on-campus laabu nwere (Paolacci, Chandler, and Ipeirotis 2010; Horton, Rand, and Zeckhauser 2011; Mason and Suri 2012; Rand 2012; Berinsky, Huber, and Lenz 2012) .

Nnukwu ike nke nwere na sonyere esi n'obodo MTurk bụ logistical: ha na-ekwe na-eme nnyocha na-n'oru sonyere ngwa ngwa na dị ka mkpa. Ebe lab nwere ike iwe ọtụtụ izu na-agba ọsọ na ubi nwere ike iwe ọtụtụ ọnwa iji na-elu-, nwere na sonyere esi n'obodo MTurk nwere ike na-agba ọsọ na ụbọchị. Dị ka ihe atụ, Berinsky, Huber, and Lenz (2012) na-enwe ike n'oru 400 achị na otu ụbọchị isonye na ihe 8 nkeji nnwale. Ọzọkwa, ndị a sonyere nwere ike recruited n'ihi nta ka ọ bụla nzube (gụnyere nnyocha ndị e mere na uka mmekota, dị ka e kwuru n'Isi nke 3 na 5). Nke a ala nke mbanye pụtara na-eme nnyocha nwere ike na-agba ọsọ usoro nke yiri nwere ibe ha ngwa ngwa.

Tupu ew sonyere na MTurk n'ihi na gị onwe gị nwere, e nwere ebe anọ dị mkpa ihe mara. Nke mbụ, ọtụtụ ndị nnyocha nwere a na-abụghị kpọmkwem skepticism nke nwere metụtara Turkers. N'ihi na nke a skepticism abụghị kpọmkwem, ọ bụ ike ịrụ ụka na-egosi. Otú ọ dị, mgbe ọtụtụ afọ nke ọmụmụ iji Turkers, anyị nwere ike ugbu a kwubie na a ịrụ eziokwu ụka adịghị karịsịa mkpa. E nwere ọtụtụ ọmụmụ atụnyere demographics nke Turkers ndị ọzọ bi na ọtụtụ ọmụmụ atụnyere ihe ndị nwere na Turkers na-arụpụta ihe ndị ọzọ bi na. Nyere ndị niile ọrụ a, m na-eche na ụzọ kasị mma n'ihi na ị na-eche banyere ya bụ na Turkers bụ a ezi uche na mma sample, dị nnọọ ka ụmụ akwụkwọ ma ubé ihe iche iche (Berinsky, Huber, and Lenz 2012) . N'ihi ya, dị nnọọ ka ụmụ akwụkwọ ndị a nwere ezi uche na onu ogugu maka ụfọdụ ma ọ bụghị ihe niile ibuo research, Turkers bụ a ezi uche bi n'ihi na ụfọdụ ma ọ bụghị ihe niile research. Ọ bụrụ na ị na-aga na-arụ ọrụ Turkers, mgbe ahụ, ọ bụ ihe ezi uche na-agụ ọtụtụ n'ime ndị a comparative ọmụmụ na-aghọta ha nuances.

Nke abụọ, na-eme nnyocha mepụtara kasị mma-omume na-amụba esịtidem ndaba nke Turk nwere, na ị kwesịrị ị na-amụta banyere na-eso ndị a kasị mma-omume (Horton, Rand, and Zeckhauser 2011; Mason and Suri 2012) . Dị ka ihe atụ, na-eme nnyocha iji Turkers na-ume iji screeners wepụ inattentive sonyere (Berinsky, Margolis, and Sances 2014; Berinsky, Margolis, and Sances 2016) (ma lee kwa DJ Hauser and Schwarz (2015b) na DJ Hauser and Schwarz (2015a) ). Ọ bụrụ na ị adịghị ewepụ inattentive sonyere, mgbe ahụ ọ bụla a pụrụ isi kwuo nke ọgwụgwọ nwere ike na-asa ya site mkpọtụ ẹkenam si inattentive sonyere, na na na omume ọnụ ọgụgụ nke inattentive sonyere nwere ike ịbụ ọkpụrụkpụ. Na nnwale nke Huber na ibe (2012) ihe 30% nke ndị keere òkè na-emezughị isi anya screeners. Nsogbu ọzọ nkịtị na Turkers bụ ndị na-abụghị ghọgbuo sonyere (Chandler et al. 2015) .

Nke atọ, ikwu na ụfọdụ ndị ọzọ iche-iche nke dijitalụ nwere, MTurk nwere ike n'ọtụtụ na; Stewart et al. (2015) na-eme atụmatụ na mgbe ọ bụla e nyere n'oge e nwere nanị mmadụ 7,000 on MTurk.

N'ikpeazụ, unu mara na MTurk bụ a obodo na ya iwu na norms (Mason and Suri 2012) . N'otu ụzọ ahụ ị ga-agbalị chọpụta ihe banyere omenala nke mba ebe ị na-aga na-agba ọsọ gị nwere, ị ga-agbalị chọpụta ihe banyere omenala na norms nke Turkers (Salehi et al. 2015) . Na, ị ga-amarakwa na Turkers ga-ekwu okwu banyere gị nnwale ma ọ bụrụ na ị na-eme ihe na-ekwesịghị ekwesị ma ọ bụ unethical (Gray et al. 2016) .

MTurk bụ ihe incredibly ntụsara ahụ isi n'oru sonyere gị nwere, ma ha bụ ndị lab-dị ka, dị ka Huber, Hill, and Lenz (2012) , ma ọ bụ karịa ubi-dị ka, dị ka Mason and Watts (2009) , Goldstein, McAfee, and Suri (2013) , Goldstein et al. (2014) , Horton and Zeckhauser (2016) , na Mao et al. (2016) .

• -Ewu gị onwe gị na ngwaahịa (Nkebi 4.5.1.3)

Ọ bụrụ na ị na-eche echiche na-agbalị ike gị ngwaahịa, m nwere ike ikwu na ị na-agụ ndụmọdụ awa site MovieLens ìgwè na Harper and Konstan (2015) . A isi ghọta ihe ndị ha maara bụ na nke ọ bụla ịga nke ọma oru ngo e nwere ọtụtụ, ọtụtụ ọdịda. Dị ka ihe atụ, ndị MovieLens otu ulo oru ndị ọzọ na ngwaahịa ndị dị otú ahụ dị ka GopherAnswers ya na ha zuru ezu ọdịda (Harper and Konstan 2015) . Ihe atụ ọzọ nke a na-eme nchọpụta gwụsịrị mgbe mbọ iji wuo a ngwaahịa bụ Edward Castronova mgbalị na-ewu ihe online game akpọ Arden. N'agbanyeghị $ 250,000 na ego, ndi oru ngo bụ a flop (Baker 2008) . Projects ka GopherAnswers na Arden bụ dị mwute ikwu na ọtụtụ ihe ndị ọzọ na-ahụkarị ihe oru ngo ka MovieLens. N'ikpeazụ, mgbe m kwuru na m na-amaghị nke ọ bụla ọzọ na-eme nnyocha nke na-aga nke ọma wuru ngwaahịa maka ugboro ugboro experimentation ebe a bụ m ibiere: 1) sonyere eji ngwaahịa n'ihi ihe ọ na-enye ha (eg, ha na-adịghị akwụ ụgwọ na ha na-adịghị ọrụ afọ ofufo na-enyere ndị ọkà mmụta sayensị) na 2) ngwaahịa e ji mee ihe maka ihe karịrị otu iche nnwale (ie, abụghị otu nnwale otutu ugboro dị iche iche na soò na ọdọ mmiri). Ọ bụrụ na ị maara nke ihe atụ ndị ọzọ, biko mee ka m mara.

• Mmekọ ahụ dị ike na (Nkebi 4.5.2)

M nụrụ na echiche nke Pasteur si Quadrant tụlere ugboro ugboro na tech ụlọ ọrụ, na ọ na-enyere hazie research mbo Google (Spector, Norvig, and Petrov 2012) .

Nkekọ na ndị ọrụ ibe 'ọmụmụ (2012) na-agbali iji chọpụta mmetụta nke ndị a agwọ ọrịa na ndị enyi nke ndị na-natara ha. N'ihi na nke imewe nke nnwale, ndị a spillovers ndị siri ike ịchọpụta cleanly; nwere mmasị na-agụ akwụkwọ ga-ahụ Bond et al. (2012) n'ihi na a ọzọ ntụle zuru ezu banyere. Ihe a bụ akụkụ nke a ogologo omenala nke nwere na ndọrọ ndọrọ ọchịchị ndị ọkà mmụta sayensị na mgbalị na-agba ume ịtụ vootu (Green and Gerber 2015) . Ndị a ịba si-the-votu nwere idi ọsọ ke akụkụ n'ihi na ha nọ Pasteur si Quadrant. Ya bụ, e nwere ọtụtụ ndị na-akpali ntọt ịtụ vootu na ịtụ vootu pụrụ ịbụ na-akpali omume iji nwalee ihe n'ozuzu chepụtara banyere omume mgbanwe na-elekọta mmadụ mmetụta.

Ndị nnyocha ndị ọzọ nyere ndụmọdụ banyere na-agba ọsọ ubi nwere na onye òtù dị ka ndọrọ ndọrọ ọchịchị ọzọ, otu oru Ngo, na ulo (Loewen, Rubenson, and Wantchekon 2010; List 2011; Gueron 2002) . Ndị ọzọ nyere ndụmọdụ banyere otú amụma mmekọrịta ya na òtù dị iche iche nwere ike mmetụta research aghụghọ (Green, Calfano, and Aronow 2014; King et al. 2007) . Partnership nwekwara ike ịkpata usoro ziri ezi ajụjụ (Humphreys 2015; Nickerson and Hyde 2016) .

• Design ndụmọdụ (Nkebi 4.6)

Ọ bụrụ na ị na-aga ịmepụta otu analysis atụmatụ tupu agba ọsọ gị nnwale, m atụ aro na ị na-amalite site n'ịgụ akuko ụkpụrụ nduzi. The nwunye (Consolidated Standard Ide akụkọ Ọnwụnwa) ụkpụrụ nduzi e mepụtara na nkà mmụta ọgwụ (Schulz et al. 2010) na gbanwetụrụ na-elekọta mmadụ research (Mayo-Wilson et al. 2013) . A yiri set nke ụkpụrụ nduzi e mepụtara site na ndị editọ nke Journal of Na Political Science (Gerber et al. 2014) (lee kwa Mutz and Pemantle (2015) na Gerber et al. (2015) ). N'ikpeazụ, akuko ụkpụrụ nduzi e mepụtara na akparamaagwa (Group 2008) , na-ahụ na- Simmons, Nelson, and Simonsohn (2011) .

Ọ bụrụ na ị ịmepụta otu analysis atụmatụ i kwesịrị ịtụle tupu ịdenye ya n'ihi na tupu ndebanye ga-amụba obi ike na ndị ọzọ nwere na gị pụta. Ọzọkwa, ọ bụrụ na ị na-arụ ọrụ na-a onye, ​​na ọ ga-amachi onye gị nwere ikike ịgbanwe analysis mgbe ha hụrụ na ya pụta. Tupu ndebanye na-aghọ esiwanye nkịtị na akparamaagwa (Nosek and Lakens 2014) , ndọrọ ndọrọ ọchịchị ọkà mmụta sayensị (Humphreys, Sierra, and Windt 2013; Monogan 2013; Anderson 2013; Gelman 2013; Laitin 2013) , na akụnụba (Olken 2015) .

Mgbe na-eke gị tupu analysis atụmatụ ị kwesịrị ịmara na ndị nnyocha ụfọdụ na-ejikwa nlaghachi azụ na yiri eru nso iji melite nkenke nke na-eme atụmatụ ọgwụgwọ mmetụta, na e nwere ụfọdụ arụmụka banyere nke a obibia: Freedman (2008) , Lin (2013) , na Berk et al. (2013) ; -ahụ Bloniarz et al. (2016) maka ozi ndị ọzọ.

Design ndụmọdụ kpọmkwem maka online ubi nwere na-adade ke Konstan and Chen (2007) na Chen and Konstan (2015) .

• Mepụta efu variable eri data (Nkebi 4.6.1)

N'ihi na ihe na MusicLab nwere, lee Salganik, Dodds, and Watts (2006) , Salganik and Watts (2008) , Salganik and Watts (2009b) , Salganik and Watts (2009a) , na Salganik (2007) . Maka ozi on eto eto-na--ahia niile,-ahụ Frank and Cook (1996) . Maka ozi on untangling chioma na nkà ihe n'ozuzu,-ahụ Mauboussin (2012) , Watts (2012) , na Frank (2016) .

E nwere ihe ọzọ obibia iwepu soò na ịkwụ ụgwọ na-eme nnyocha ga-eji na-elezi anya: conscription. N'ọtụtụ online ubi nwere sonyere na-ihu ọma debara aha n'ime nnwale na mgbe ahụ ụgwọ atụrụ. Ihe atụ nke a obibia agụnye Restivo na van de Rijt si (2012) nnwale on ụgwọ ọrụ na Wikipedia na Bond na onye otu na- (2012) nnwale on agba ume ka ndị mmadụ votu. Ndị a nwere na-adịghị n'ezie nwere efu variable na-eri, ha nwere efu variable-eri na-eme nnyocha. Ọ bụ ezie na ndị na-eri ọtụtụ n'ime ndị a nwere bụ nnọọ obere ka ọ bụla soò, obere akwụ ụgwọ amanye onwe ya ịgbaso buru ibu ọnụ ọgụgụ nke ndị keere òkè ike ịgbakwunye ngwa ngwa. Na-eme nnyocha na-agba ọsọ oke online nwere mgbe ziri ezi na mkpa nke obere e mere atụmatụ ịbụ ọgwụgwọ mmetụta site n'ikwu na ndị a obere mmetụta pụrụ ịghọ ihe dị mkpa ma e jiri ya na ọtụtụ ndị. Kpọmkwem otu echiche na-emetụta na-akwụ ụgwọ na-eme nnyocha wetara sonyere. Ọ bụrụ na gị nwere na-akpata otu nde mmadụ na-ala n'iyi n'otu nkeji, na nnwale bụghị nnọọ na-emerụ ihe ọ bụla akpan akpan onye, ​​ma na rụpụta mkpokọta nchịkọta ọ egbughị fọrọ nke nta afọ abụọ nke oge.

Ọzọ obibia eke efu variable na-eri ugwo ka sonyere bụ iji lọtrị, na-abịaru nso na nwekwara e ji mee ihe na nnyocha e mere nnyocha (Halpern et al. 2011) . N'ikpeazụ, n'ihi na ihe banyere emebe atọ ụtọ ọrụ na onye-ahụmahụ ahụ Toomim et al. (2011) .

• Dochie, Nụchaa, na Belata (Nkebi 4.6.2)

Lee ndị mbụ nkọwa nke atọ R, si Russell and Burch (1959) :

"Nnọchi pụtara na nnọchi maka adịghị ama bi elu anụmanụ nke insentient ihe onwunwe. Mbelata pụtara gụnyere ya ná nọmba nke ụmụ anụmanụ ji mee ihe iji nweta ozi nke a nyere ego na nkenke. Ukpụhọde pụtara bụla ọnụ na omume ma ọ bụ ịdị njọ nke inhumane na usoro etinyere anụmanụ ndị nke ka na-enwe na-eji. "

The atọ R si na M gwa Abigel adịghị akpagbu ndị metụtara ụkpụrụ ụkpụrụ a kọwara n'Isi 6. Kama nke ahụ, ha na-a ọzọ kọwakwuru version otu n'ime ụkpụrụ ndị ahụ-beneficence-kpọmkwem maka ọnọdụ nke ụmụ mmadụ nwere.

Mgbe atụle mmetụta uche Contagion, e nwere atọ na-abụghị usoro ziri ezi mbipụta iburu n'uche mgbe na ikọ isi ihe a. Nke mbụ, ọ bụ anya otú ahụ n'ezie nkọwa nke nnwale jikọọ na usoro iwu na-ekwu; ndị ọzọ okwu, e nwere ajụjụ banyere mmepụta ya ndaba. Ọ bụghị doro anya na nti na-adịghị mma okwu adabere ndị bụ n'ezie-egosi na nke mmetụta uche steeti sonyere n'ihi na 1) ọ bụghị doro anya na okwu na ndị mmadụ na biputere na--egosi mmetụta ha na 2) Ọ bụ anya na akpan akpan mmetụta analysis Usoro nke na-eme nnyocha na-eji na-enwe ike reliably infer mmetụta uche (Beasley and Mason 2015; Panger 2016) . Ndị ọzọ okwu, e nwere ike inwe a ọjọọ ọtụtụ nke a abụchaghị eziokwu mgbaàmà. Nke abụọ, imewe na analysis nke nnwale na-agwa anyị ihe ọ bụla banyere onye kasị impacted (ie, ọ dịghị analysis of heterogeneity ọgwụgwọ mmetụta) na ihe usoro pụrụ ịbụ. Na nke a, ndị nnyocha ahụ nwere nza nke ihe ọmụma banyere ndị sonyere, ma ha na-nnoo emeso dị ka wijetị na analysis. Nke atọ, mmetụta size na nnwale a bụ nnọọ obere; ihe dị iche n'etiti ọgwụgwọ na ịchịkwa ọnọdụ bụ ihe 1 na 1000 okwu. Ha akwụkwọ, Kramer na ibe-eme ka ikpe na mmetụta nke a size dị mkpa n'ihi na ọtụtụ narị nde ndị mmadụ nweta ha News Nri kwa ụbọchị. Ya bu, ha na-ekwu na ọbụna mmetụta na ndị bụ obere n'ihi na onye ọ bụla ha bụ nnukwu na rụpụta mkpokọta nchịkọta. Ọ bụrụ na unu na-anabata arụmụka a, ọ bụ ka anya ma ọ bụrụ na mmetụta nke a size dị mkpa banyere ihe n'ozuzu ajụjụ nkà mmụta sayensị banyere obi contagion. Maka ozi na ọnọdụ ebe obere mmetụta ndị dị mkpa na-ahụ Prentice and Miller (1992) .

Na okwu nke mbụ R (Nnọchi), atụnyere mmetụta uche Contagion nnwale (Kramer, Guillory, and Hancock 2014) na nke mmetụta uche contagion eke nnwale (Coviello et al. 2014) awade ụfọdụ n'ozuzu ihe banyere ahia-offs-aka na-akpụ akpụ site na nwere na-eke nwere (na ndị ọzọ na-eru nso ka kenha na-anwa ịgụta nwere na-abụghị ibuo data, gụọ Isi nke 2). Na mgbakwunye na usoro ziri ezi uru, ịmafe si ibuo na-abụghị ndị ibuo ọmụmụ na-enyere ndị nnyocha na-amụ agwọ ọrịa na ha bụ ndị logistically enweghị ike idokwa. Ndị a usoro ziri ezi na logistical uru na a na-eri, Otú ọ dị. Na eke nwere na-eme nnyocha nwere obere achịkwa ihe dị ka mbanye nke sonyere, randomization, na ọdịdị nke ihe omume. Dị ka ihe atụ, otu mmachi nke mmiri ozuzo dị ka a ọgwụgwọ bụ na ọ ma enwekwu positivity na mbelata negativity. Na ibuo ọmụmụ Otú ọ dị, Kramer na ndị ọrụ ibe nwee ike ịgbanwe positivity na negativity onwe ha.

The akpan akpan obibia ji site Coviello et al. (2014) kọwakwuru kọwakwuru na Coviello, Fowler, and Franceschetti (2014) . N'ihi okwu mmeghe na-instrumental variables-ahụ Angrist and Pischke (2009) (erughị anụmanụ na) ma ọ bụ Angrist, Imbens, and Rubin (1996) (ọzọ iwu). N'ihi na a obi abụọ ịmata otú mmekọrịta instrumental variables-ahụ Deaton (2010) , na maka okwu mmeghe na-instrumental variables na-adịghị ike ngwá (mmiri ozuzo bụ a na-adịghị ike ngwá),-ahụ Murray (2006) .

More n'ozuzu, a mma iwebata ka eke nwere bụ Dunning (2012) , na Rosenbaum (2002) , Rosenbaum (2009) , na Shadish, Cook, and Campbell (2001) na-enye ezi echiche banyere estimating causal mmetụta na-enweghị nwere.

Na okwu nke-abua R (ukpụhọde), e nwere na nkà mmụta sayensị na logistical ahia-offs mgbe atụle agbanwe imewe nke mmetụta uche Contagion si igbochi posts ka boosting posts. Dị ka ihe atụ, o nwere ike ịbụ ikpe na teknuzu mmejuputa iwu-nke News Feed-eme ka ọ bụ n'ụzọ mfe ime nnwale na igbochi posts kama nnwale na boosting posts (mara na nnwale na igbochi posts nwere ike na-emejuputa atumatu dị ka a oyi akwa on n'elu nke News Feed usoro na-enweghị ihe ọ bụla mkpa alterations nke kpatara usoro). Ndị ọkà mmụta sayensị, Otú ọ dị, ozizi agwa okwu na nnwale mere n'ụzọ doro anya na-atụ aro otu imewe na ndị ọzọ.

O di nwute na, adịghị m maara nke ọkpụrụkpụ tupu nnyocha banyere ikwu kpọkwara igbochi na boosting ọdịnaya News Feed. Ọzọkwa, m na-ahụbeghị ihe nnyocha banyere anụcha agwọ ọrịa na-eme ka ha na-erughị emerụ; otu dị iche bụ Jones and Feamster (2015) , nke weere banyere amata ọtụtụ Internet nnyocha (a isiokwu m atụle n'Isi nke 6 na mmekọrịta ahụ encore ọmụmụ (Burnett and Feamster 2015; Narayanan and Zevenbergen 2015) ).

Na okwu nke-atọ R (Ibenata mmasi), a dị mma iwebata ka omenala ike analysis bụ Cohen (1988) . Tupu ọgwụgwọ covariates pụrụ itinye ná imewe ogbo na analysis ogbo nke nwere; Isi nke 4 nke Gerber and Green (2012) na-enye a ezi iwebata ka ma-eru nso, na Casella (2008) na-enye a ọzọ na-omimi ọgwụgwọ. Usoro ndị na-eji nke a tupu ọgwụgwọ ọmụma na randomization na a na-akpọ ma-egbochi ibuo aghụghọ ma ọ bụ stratified ibuo aghụghọ (okwu e adịghị mee anọgide gafee obodo); ndị a usoro na-miri metụtara na stratified ụfọdụ usoro tụlere n'Isi nke 3 Lee Higgins, Sävje, and Sekhon (2016) n'ihi na ihe na iji ndị a na aghụghọ na oke nwere. Tupu ọgwụgwọ covariates nwekwara ike-esịne ke analysis ogbo. McKenzie (2012) explores ihe dị iche-in-iche obibia nyochasịrị ubi nwere n'ụzọ sara mbara karị. Lee Carneiro, Lee, and Wilhelm (2016) n'ihi na ihe na ahia-offs n'etiti dị iche iche na-eru nso na-amụba nkenke na atụmatụ nke ọgwụgwọ utịp. N'ikpeazụ, mgbe ekpebi ma ha na-agbalị na-agụnye tupu ọgwụgwọ covariates na imewe ma ọ bụ analysis ogbo (ma ọ bụ ma), e nwere ihe ole na ole ga-atụle. Na a ọnọdụ ebe na-eme nnyocha chọrọ igosi na ha abụghị "ịkụ azụ" (Humphreys, Sierra, and Windt 2013) , na-eji tupu ọgwụgwọ covariates ke imewe ogbo pụrụ inye aka (Higgins, Sävje, and Sekhon 2016) . Na ọnọdụ ebe sonyere na-abata sequentially, karịsịa online ubi nwere, iji tupu ọgwụgwọ ọmụma ke imewe ogbo nwere ike isi ike logistically,-ahụ ihe atụ Xie and Aurisset (2016) .

Ọ bụ uru na-agbakwụnye a bit nke nsinammuo banyere ihe mere ihe dị iche-in-iche nwere ike ịbụ ya mere nnọọ irè karịa dị iche-in-n'aka. Ọtụtụ online esi nwere nnọọ elu ndịiche (lee eg, Lewis and Rao (2015) na Lamb et al. (2015) ) na ndị dịtụ ufọk ufene karịrị oge. Na nke a, mgbanwe akara ga-enwe n'ụzọ nta ndịiche, ịba ụba na ike nke mgbakọ na mwepụ ule. Otu ihe mere nke a bịakwutere adịghị mee ihe ọzọ mgbe bụ na tupu digital afọ na ọ bụghị nkịtị nwere tupu ọgwụgwọ a ga esi. A ọzọ ihe n'ụzọ na-eche banyere ya dị iche n'echiche nnwale tụọ ma a kpọmkwem usoro mmega akpata arọ ọnwụ. Ọ bụrụ na ị na-eme ka a dị iche-in-esi bịaruo nso, gị atụmatụ ga-enwe ike mgbanwe nke na-abịa site na mgbanwe na igwe na ndị bi na. Ọ bụrụ na ị na-eme ka a dị iche-in-iche obibia Otú ọ dị, na-adịkarị na-ewere ọnọdụ mgbanwe igwe ego n'anya wepụrụ ma ị nwere ike mfe ịchọpụta a dị iche kpatara ọgwụgwọ.

Otu ụzọ dị mkpa iji belata ọnụ ọgụgụ nke ndị keere òkè gị nnwale bụ ịhụ na omume a ike analysis, nke Kramer na ibe nwere ike mere nke dabeere na mmetụta nha hụrụ si eke nnwale site Coviello et al. (2014) ma ọ bụ tupu-abụghị ibuo nnyocha site Kramer (2012) (N'eziokwu a bụ omume na njedebe nke isiokwu a). Rịba ama na a iji ike analysis bụ a bit dị iche iche karịa ahụkarị. Na analọg afọ, na-eme nnyocha n'ozuzu mere ike analysis iji jide n'aka na ha na-amụ ọ bụghị kwa obere (ie, n'okpuru-kwadoro). Ugbu a, Otú ọ dị, na-eme nnyocha ga-eme ike analysis iji jide n'aka na ha na-amụ adịghị kwa nnukwu (ie, n'elu-kwadoro).

N'ikpeazụ, m na-ewere na-agbakwunye a anọ R: repurpose. Ya bụ, ma ọ bụrụ na nnyocha ahụ onwe ha na ihe ibuo data karịa ha mkpa iji lebara ha ná nnyocha ajụjụ, ha kwesịrị repurpose ihe data ịjụ ọhụrụ ajụjụ. Dị ka ihe atụ, were ya na Kramer na ndị ọrụ ibe ji a dị iche-in-iche estimator na hụrụ onwe ha na ndị ọzọ data karịa ihe a chọrọ iji lebara ha research ajụjụ. Kama ọ bụghị iji data eme ihe ruo n'ókè, ha gaara amụ size nke mmetụta ka otu ọrụ-eji ọgwụgwọ nke mmetụta uche okwu. Dị nnọọ ka Schultz et al. (2007) chọpụtara na mmetụta nke ọgwụgwọ dị iche n'ọnọdụ ìhè, na ịṅụ oké ọrụ, ikekwe mmetụta nke News Feed ndị dị iche iche maka ndị na-ama yiri ka biputere obi ụtọ (ma ọ bụ mwute) messages. Repurposing nwere ike 'azu " (Humphreys, Sierra, and Windt 2013) na" p-hacking " (Simmons, Nelson, and Simonsohn 2011) , ma ndị a na-n'ụzọ dị ukwuu addressable na a Nchikota n'eziokwu akuko (Simmons, Nelson, and Simonsohn 2011) , tupu ndebanye (Humphreys, Sierra, and Windt 2013) , na igwe mmụta ụzọ na-agbali iji zere n'elu-efe Chineke kwesịrị ekwesị.