Abinda za a karanta a gaba

• Gabatarwa (sashe na 4.1)

Tambayoyi game da lalacewa a cikin bincike na zamantakewar mutane suna da rikicewa da mahimmanci. Domin hanyar da za a samo asali game da zane-zane bisa ga zane-zane, duba Pearl (2009) , da kuma tsarin da aka samo asali bisa ga sakamako mai kyau, duba Imbens and Rubin (2015) . Don kwatanta tsakanin waɗannan hanyoyin biyu, duba Morgan and Winship (2014) . Domin hanyar da aka dace don gano VanderWeele and Shpitser (2013) , duba VanderWeele and Shpitser (2013) .

A cikin wannan babin, na halicci abin da ya kasance kamar mai haske a tsakanin iyakokinmu na yin ƙididdigar ƙira daga gwaji da wadanda ba na gwaji ba. Duk da haka, ina tsammanin cewa, a gaskiya, bambanci ya fi damuwa. Alal misali, kowa da kowa na yarda cewa shan taba yana haifar da ciwon daji, ko da yake babu gwajin gwaji wanda ya sa mutane su taba shan taba an riga an yi. Don kyakkyawan magani a cikin littafi na tsawon lokaci don yin abubuwan da aka ƙaddara daga wadanda ba na gwaji ba ne ga Rosenbaum (2002) , ( ??? ) , Shadish, Cook, and Campbell (2001) da kuma Dunning (2012) .

Sashe na 1 da 2 na Freedman, Pisani, and Purves (2007) suna ba da gabatarwa bayyananne ga bambancin dake tsakanin gwaje-gwaje, gwaje-gwaje masu sarrafawa, da kuma gwajin gwagwarmaya.

Manzi (2012) yana ba da labari mai ban mamaki da kuma iya fadadawa ga tsarin ilimin falsafa da na ilimin lissafi na gwaje-gwajen da aka sarrafa. Har ila yau, yana bayar da misalai masu ban sha'awa na duniya game da ikon gwaji a kasuwanci. Issenberg (2012) yana bayar da gabatarwa mai ban sha'awa ga amfani da gwaji a cikin yakin siyasa.

• Menene gwaje-gwaje? (sashi na 4.2)

Box, Hunter, and Hunter (2005) , @ casella_statistical_2008, da Athey and Imbens (2016b) bayar da kyakkyawar gabatarwa ga tsarin ilimin lissafin gwaji da bincike. Bugu da ƙari, akwai maganganun mahimmanci game da yin amfani da gwaje-gwaje a wurare daban-daban: tattalin arziki (Bardsley et al. 2009) , ilimin zamantakewa (Willer and Walker 2007; Jackson and Cox 2013) , ilimin kimiyya (Aronson et al. 1989) , kimiyyar siyasa (Morton and Williams 2010) , da kuma manufofin zamantakewa (Glennerster and Takavarasha 2013) .

Muhimmancin mai daukar ma'aikata (misali, samfurin samfuri) ana jin dadin zama a cikin bincike na gwaji. Duk da haka, idan sakamako na jiyya yana da bambanci a cikin yawan jama'a, to, samfurin yana da muhimmanci. Longford (1999) ya bayyana wannan a fili lokacin da yake bada shawara ga masu bincike suna tunanin gwaje-gwaje a matsayin binciken jama'a tare da samfurin samfur.

• Darasi biyu na gwaje-gwaje: Lab-filin da analog-dijital (sashi na 4.3)

Na nuna cewa akwai ci gaba a tsakanin jarrabawa da gwaje-gwajen gwaje-gwajen, kuma wasu masu bincike sun ba da cikakkun siffofin da suka dace, musamman wadanda suka raba nau'o'i daban-daban na gwaje gwaje-gwaje (Harrison and List 2004; Charness, Gneezy, and Kuhn 2013) .

Yawancin takardu sunyi la'akari da lab da gwaje-gwajen gwaje-gwaje a cikin litattafai (Falk and Heckman 2009; Cialdini 2009) da kuma sakamakon sakamakon gwaje-gwaje na musamman a kimiyyar siyasa (Coppock and Green 2015) , tattalin arziki (Levitt and List 2007a, 2007b; Camerer 2011; Al-Ubaydli and List 2013) , da kuma ilimin halin mutum (Mitchell 2012) . Jerit, Barabas, and Clifford (2013) suna ba da kyakkyawar zane-zane na bincike don gwada sakamakon daga lab da gwaje-gwajen gwaje-gwaje. Parigi, Santana, and Cook (2017) kwatanta yadda nazarin gwaje-gwaje na yanar gizo ya hada wasu alamomi na gwaji da gwaji.

Damuwa game da mahalarta canza halin su domin sun san cewa ana lura da su a wasu lokutan ana kiran su sakamakon illa , kuma ana nazarin su cikin ilimin halayyar mutum (Orne 1962) da tattalin arziki (Zizzo 2010) . Kodayake mafi yawancin hade da gwaje-gwajen gwaje-gwajen, waɗannan batutuwa guda ɗaya na iya haifar da matsalolin gwaje-gwajen filin. A gaskiya, bukatar effects ma wani lokacin kira Hawthorne effects, a wani ajali wanda ya sami asali da sanannen haske gwaje-gwajen da ya fara a 1924 a Hawthorne Works of Western Electric Company (Adair 1984; Levitt and List 2011) . Dukkan abubuwan da ake bukata da kuma halayen Hawthorne suna da nasaba da ra'ayin da za a gwada su a cikin babi na 2 (duba kuma Webb et al. (1966) ).

Gwajin gwaje-gwaje na da tarihi mai tsawo (Levitt and List 2009) , kimiyyar siyasa (Green and Gerber 2003; Druckman et al. 2006; Druckman and Lupia 2012) , ilimin halayyar mutum (Shadish 2002) , da kuma manufofin jama'a (Shadish and Cook 2009) . Wani bangare na kimiyyar zamantakewa inda gwaje-gwaje na hanzari ya zama shahararren ci gaban kasa da kasa. Domin nazari mai kyau game da wannan aikin a cikin tattalin arziki gani Banerjee and Duflo (2009) , kuma don ƙidayar kalubalen duba Deaton (2010) . Don nazarin wannan aikin a kimiyyar siyasa ya ga Humphreys and Weinstein (2009) . A karshe, an gano matsalolin gwagwarmaya daga binciken gwaje-gwaje a cikin batun kimiyyar siyasa (Humphreys 2015; Desposato 2016b) da kuma tattalin arziki (Baele 2013) .

A cikin wannan sashe, Na nuna cewa za'a iya amfani da bayanin kulawa da rigakafi don inganta yanayin ƙayyadaddun magani, amma akwai wasu muhawara game da wannan hanya; duba Freedman (2008) , W. Lin (2013) , Berk et al. (2013) , da Bloniarz et al. (2016) don ƙarin bayani.

A ƙarshe, akwai wasu nau'o'in gwaje-gwajen da wasu masana kimiyya na zamantakewa suka yi da su ba su dace da juna ba tare da jigilar hankulan su: binciken gwaje-gwaje da gwaje-gwajen zamantakewa. Gwaje-gwajen bincike sunyi gwaje-gwaje ta yin amfani da hanyoyin bincike na yanzu kuma sun kwatanta martani ga wasu matakan tambayoyin (wasu binciken da aka yi nazari an gabatar a Babi na 3); don ƙarin bayani game da gwaje-gwajen binciken duba Mutz (2011) . Binciken da aka yi na zamantakewar jama'a shine gwaje-gwajen inda magunguna ke da wasu manufofin zamantakewar da gwamnati za ta iya aiwatarwa kawai. Binciken na jama'a suna da nasaba da shirin gwajin. Don ƙarin bayani game da gwaje-gwajen manufofin, duba Heckman and Smith (1995) , Orr (1998) , da @ glennerster_running_2013.

• Komawa bayan gwaje-gwaje masu sauki (sashi na 4.4)

Na zaba don mayar da hankali kan abubuwa uku: inganci, bambancin yanayin maganin, da kuma hanyoyin. Wadannan ra'ayoyi sunaye daban-daban a cikin wurare daban-daban. Alal misali, masana kimiyya suna matsawa bayan gwaje-gwaje masu sauki ta hanyar mayar da hankali ga masu jarida da masu adawa (Baron and Kenny 1986) . Mahimmancin masu sana'a suna kama da abin da nake kiran su, kuma ana tunanin ra'ayin masu dacewa ta abin da na kira inganci na waje (misali, sakamakon gwajin zai zama daban idan an gudanar da shi a yanayi daban-daban) da kuma bambancin yanayin maganin ( misali, sune mafi girma ga wasu mutane fiye da wasu).

Gwajin da Schultz et al. (2007) nuna yadda za a iya amfani da ka'idodin zamantakewa don tsara tasirin tasiri. Don ƙarin bayani game da muhimmancin ka'idar a tsara zayyana tasiri, duba Walton (2014) .

• Tabbatarwa (ɓangare 4.4.1)

Manufofin na ciki da na waje sun fara gabatarwa da Campbell (1957) . Dubi Shadish, Cook, and Campbell (2001) don cikakken cikakken tarihin da kuma nazari akan ƙididdigar ƙididdigar ƙididdigar lissafi, inganci na ciki, ingantaccen aiki, da kuma ƙwarewar waje.

Don ƙarin bayani game da al'amurra da suka danganci ƙididdigar lissafi a cikin gwaje-gwajen duba Gerber and Green (2012) (daga hangen nesa na zamantakewa) da Imbens and Rubin (2015) (daga hangen nesa). Wasu al'amurra na ƙididdigar ƙididdigar lissafi da ke fitowa musamman a cikin binciken gwaje-gwajen yanar gizo sun hada da al'amurra kamar su ingantattun hanyoyin da za su samar da kwakwalwa tare da bayanan dogara (Bakshy and Eckles 2013) .

Tabbatar da ciki na iya zama da wahala don tabbatar da gwaje-gwaje a cikin gwaji. Dubi, alal misali, Gerber and Green (2000) , Imai (2005) , da Gerber and Green (2005) don muhawara game da aiwatar da gwaji game da batun jefa kuri'a. Kohavi et al. (2012) da Kohavi et al. (2013) samar da gabatarwa a cikin kalubale na dacewa tazarar a cikin shafukan yanar gizo na gwaji.

Ɗaya daga cikin barazanar barazana ga inganci na ciki ita ce yiwuwar kasawar ƙira. Wata hanyar da za a iya gane matsalolin da ke tattare da ƙaddamarwa shine don kwatanta magungunan kulawa da kulawa a kan al'amuran da ake gani. Irin wannan kwatanci ana kiransa duba ma'auni . Dubi Hansen and Bowers (2008) don yin la'akari da daidaituwa don daidaita daidaito da Mutz and Pemantle (2015) saboda damuwa game da kulawa da ma'auni. Alal misali, ta yin amfani da ma'auni, Allcott (2011) sami wasu shaidun shaida cewa ba a aiwatar da shi ba a cikin uku na gwajin Opower (duba tebur 2, shafuka 2, 6, da 8). Don wasu hanyoyi, duba babi na 21 na Imbens and Rubin (2015) .

Sauran manyan damuwa da suka danganci ingancin ciki shine: (1) wanda ba a yarda da shi ba, inda ba duk wanda ke cikin magungunan jiyya ya karbi magani ba, (2) rashin amincewa guda biyu, inda ba kowa a cikin rukuni na karbar magani da wasu mutane ba ƙungiya mai kulawa ta karbi magani, (3) attrition, inda ba a auna sakamakon ba ga mahalarta, da kuma (4) tsangwama, inda magungunan ya karu daga mutane a yanayin magani ga mutanen dake cikin yanayin kulawa. Duba surori 5, 6, 7, da 8 na Gerber and Green (2012) don ƙarin bayani game da waɗannan batutuwa.

Don ƙarin bayani a kan gina gine-gine, gani Westen and Rosenthal (2003) , kuma don ƙarin bayani game da ingantaccen gine-gine a cikin manyan bayanai, Lazer (2015) da babi na 2 na wannan littafin.

Ɗaya daga cikin ɓangaren ƙarancin waje shi ne wuri wanda aka gwada gwaji. Allcott (2015) ba da hankali ga fahimta da kuma kulawa da tsinkayen shafukan yanar gizo. Wannan batun kuma Deaton (2010) ya tattauna. Wani bangare na ingancin waje shi ne ko wasu hanyoyin aiki na wannan hanya zasu sami irin wannan sakamako. A wannan yanayin, kwatanta tsakanin Schultz et al. (2007) da Allcott (2011) nuna cewa gwaje-gwaje na Opower sunyi tasiri fiye da ƙwarewar da Schultz da abokan aiki suka yi (1.7% zuwa 5%). Allcott (2011) yi la'akari da cewa samfurori masu biyowa suna da ƙananan sakamako saboda hanyoyi da maganganun suka bambanta: imoticon da aka rubutun hannu a matsayin wani ɓangare na binciken da jami'a ke tallafawa, idan aka kwatanta da maƙasudin rubutun kalmomi a matsayin ɓangare na ƙaddamar da taro rahoton daga kamfanin sarrafawa.

• Hanyoyin maganin cutar (kashi 4.4.2)

Don kyakkyawan bayani game da bambancin ilimin maganin gwaje-gwajen gwaje-gwaje, duba bita na 12 na Gerber and Green (2012) . Don gabatarwa ga nau'o'in maganin magani a gwajin gwaji, duba Kent and Hayward (2007) , Longford (1999) , da Kravitz, Duan, and Braslow (2004) . Abubuwan da ke tattare da nau'o'in maganin cututtuka suna mayar da hankali ga bambance-bambance bisa ka'idodin magancewa. Idan kuna da sha'awar bambancin (Frangakis and Rubin 2002) bayan sakamakon maganin bayanan, to, ana buƙatar hanyoyin da za a yi amfani da shi, irin su magungunan farko (Frangakis and Rubin 2002) ; duba Page et al. (2015) don bita.

Mutane da yawa masu bincike sun kwatanta nau'o'in maganin cutar ta hanyar amfani da layi na linzamin kwamfuta, amma sababbin hanyoyin sun dogara da ilmantarwa na injiniya; duba, alal misali, Green and Kern (2012) , Imai and Ratkovic (2013) , Taddy et al. (2016) , da Athey and Imbens (2016a) .

Akwai wasu shakku game da sakamakon binciken da ke tattare da bambancin yanayi saboda matsalolin daidaitawa da kuma "kama kifi." Akwai hanyoyi masu yawa da zasu iya magance matsalolin da aka kwatanta da su (Fink, McConnell, and Vollmer 2014; List, Shaikh, and Xu 2016) . Daya daga cikin abubuwan da ake damu game da "kama kifi" shine pre-rajista, wanda ake samun karuwa a cikin ilimin kimiyya (Nosek and Lakens 2014) , kimiyyar siyasa (Humphreys, Sierra, and Windt 2013; Monogan 2013; Anderson 2013; Gelman 2013; Laitin 2013) , da kuma tattalin arziki (Olken 2015) .

A cikin binciken da Costa and Kahn (2013) kawai rabin rabin gidaje a cikin gwajin zasu iya danganta da bayanin alƙaluma. Masu karatu masu sha'awar wadannan bayanai ya kamata su koma ga takardun asali.

• Ra'ayoyin (sashe na 4.4.3)

Hanyoyin abubuwa sune mahimmanci, amma sun kasance da wuya suyi karatu. Bincike game da abubuwan da ke tattare da shi yana da nasaba da nazarin magoya bayansa a cikin ilimin halayyar mutum (amma ga yadda VanderWeele (2009) daidai don daidaitawa tsakanin ra'ayoyin biyu). Tattaunawar lissafi game da gano hanyoyin, irin su tsarin da aka samu a Baron and Kenny (1986) , suna da yawa. Abin baƙin ciki shine, wadannan hanyoyin sun dogara ne akan wasu tsammanin ra'ayi (Bullock, Green, and Ha 2010) kuma suna shan wahala lokacin da akwai hanyoyi masu yawa, kamar yadda mutum zai iya sa zuciya a yanayi da dama (Imai and Yamamoto 2013; VanderWeele and Vansteelandt 2014) . Imai et al. (2011) da Imai and Yamamoto (2013) suna ba da wasu hanyoyin da suka dace. Bugu da ƙari, VanderWeele (2015) yana bayar da cikakkiyar maganin rubutu tare da wasu mahimman sakamako, ciki har da cikakkiyar hanyar kula da bincike.

Hanyar da aka raba ta mayar da hankali kan gwaje-gwajen da suke ƙoƙarin sarrafa tsarin da kai tsaye (misali, bada masu bitamin C). Abin takaici, a yawancin saitunan zamantakewar zamantakewa, akwai lokuta masu yawa kuma yana da wuyar tsara zane wanda ya canza daya ba tare da canza wasu ba. Wasu hanyoyin da za a gwada aikin gwaje-gwaje sune Imai, Tingley, and Yamamoto (2013) , Ludwig, Kling, and Mullainathan (2011) , da Pirlott and MacKinnon (2016) .

Masu binciken da ke gudanar da gwaje-gwaje na ainihi zasu bukaci damuwa game da gwaji da yawa; duba Fink, McConnell, and Vollmer (2014) da List, Shaikh, and Xu (2016) don ƙarin bayani.

A ƙarshe, ma'anonin suna da tarihi mai tsawo a falsafar kimiyya kamar yadda Hedström and Ylikoski (2010) .

• Amfani da yanayin da ake ciki (sashe na 4.5.1)

Don ƙarin bayani game da yin amfani da takardun rubutu da nazarin binciken don auna nuna bambanci, duba Pager (2007) .

• Gina gwajin ka (sashe na 4.5.2)

Hanyar da ta fi dacewa don tara mahalarta zuwa gwaje-gwajen da kuke gina shine Amazon Mechanical Turk (MTurk). Tunda MTurk ta yi amfani da wasu nau'o'i na nazarin gargajiya na al'ada-biya mutane don kammala ayyukan da ba za su yi ba kyauta-da yawa masu bincike sun fara amfani da Turkers (ma'aikata a MTurk) a matsayin masu gwajin gwagwarmayar, wanda ya haifar da karfin tattara bayanai da yawa fiye da yadda za'a iya cimma a cikin nazarin gwaje-gwajen gargajiya na gargajiya (Paolacci, Chandler, and Ipeirotis 2010; Horton, Rand, and Zeckhauser 2011; Mason and Suri 2012; Rand 2012; Berinsky, Huber, and Lenz 2012) .

Kullum, amfanin mafi girma na amfani da masu halartar da aka karɓa daga MTurk su ne haruffa. Ganin cewa jarrabawar gwaje-gwaje na iya daukar makonni don farawa da gwaje-gwajen gwaje-gwaje na iya ɗaukar watanni zuwa kafa, gwaje-gwaje tare da mahalarta da aka karɓa daga MTurk za'a iya gudana cikin kwanakin. Alal misali, Berinsky, Huber, and Lenz (2012) sun iya daukar nauyin 400 abubuwa a rana guda don shiga cikin gwaji 8. Bugu da ari, za a iya tattara waɗannan mahalarta don kusan kowane dalili (ciki har da bincike da kuma haɗin gwiwar, kamar yadda aka tattauna a surori 3 da 5). Wannan sauƙi na daukar ma'aikata yana nufin cewa masu bincike zasu iya gudanar da gwaje-gwajen da suka shafi alaka da sauri.

Kafin kayiwa mahalarta daga MTurk don gwaje-gwajen ku, akwai abubuwa masu muhimmanci guda hudu da kuke buƙatar sani. Na farko, masu bincike suna da kwarewa na gwaje-gwajen da suka shafi Turkers. Saboda wannan shakku ba takamaiman ba ne, yana da wuyar magance hujja. Duk da haka, bayan shekaru da yawa na nazari da ake amfani da Turkers, zamu iya cewa yanzu wannan shakku ba shine wanda ya dace ba. Akwai nazari da yawa da aka kwatanta da mutanen Turkmen tare da sauran al'ummomi da kuma yawancin nazarin kwatanta sakamakon gwaje-gwaje da Turkers tare da wadanda daga sauran al'ummomi. Ganin dukan wannan aikin, ina ganin cewa hanya mafi kyau don ku yi tunani game da ita shi ne cewa Turkers suna da sauƙin samfurin, kamar ɗalibai amma kaɗan kaɗan (Berinsky, Huber, and Lenz 2012) . Saboda haka, kamar yadda dalibai suke da yawancin mutane, amma ba duka ba, bincike ne, Turkers suna da yawancin mutane, amma ba duka ba ne, bincike. Idan za ku yi aiki tare da Turkers, to, yana da mahimmanci don karanta yawancin waɗannan nazarin kwatanta da fahimtar nasu.

Abu na biyu, masu bincike sun kirkiro mafi kyawun ayyuka don inganta ƙwarewar MTurk na ciki, kuma ya kamata ka koyi game da bin waɗannan ayyuka mafi kyau (Horton, Rand, and Zeckhauser 2011; Mason and Suri 2012) . Alal misali, masu binciken da suke amfani da Turkers suna ƙarfafa su yi amfani da allon kwamfuta don cire masu halartar da ba su kula ba (Berinsky, Margolis, and Sances 2014, 2016) (amma ga yadda DJ Hauser and Schwarz (2015b) da DJ Hauser and Schwarz (2015a) ). Idan baza ka cire masu halartar da ba su da hankali, to duk wani tasiri na jiyya zai iya wankewa ta wurin hayaniya da suke gabatarwa, kuma a aikace yawancin mahalarta marasa sa ido zasu iya zama masu mahimmanci. A cikin gwaji na Huber da abokan aiki (2012) , kimanin kashi 30 cikin dari na mahalarta suka kasa kulawa da hankali. Sauran matsalolin da ke faruwa a lokacin da ake amfani da Turkmen su ne masu halartar bambamci (Chandler et al. 2015) da kuma haraji (Zhou and Fishbach 2016) .

Na uku, dangane da wasu nau'o'in gwaje-gwaje na dijital, gwaje-gwaje na MTurk ba za su iya sikelin ba; Stewart et al. (2015) la'akari da cewa a kowane lokacin da aka samu akwai kimanin mutane 7,000 a MTurk.

A ƙarshe, ya kamata ka san cewa MTurk wata al'umma ce da dokoki da ka'idojin kansa (Mason and Suri 2012) . Kamar yadda za ku yi kokarin gano game da al'adun kasar inda za ku gudanar da gwaje-gwajen ku, ya kamata ku yi kokarin gano ƙarin al'amuran al'adu da al'adu na Turkers (Salehi et al. 2015) . Kuma ya kamata ku sani cewa Turkmen za suyi magana game da gwajin ku idan kunyi wani abu mara dace ko rashin gaskiya (Gray et al. 2016) .

MTurk hanya ce mai matukar dacewa don karɓar masu halartar ku a cikin gwaje-gwajenku, ko suna kama da juna, irin su Huber, Hill, and Lenz (2012) , ko kuma mafi yawan filin wasa, irin su Mason and Watts (2009) , Goldstein, McAfee, and Suri (2013) , Goldstein et al. (2014) , Horton and Zeckhauser (2016) , da Mao et al. (2016) .

• Gina samfur naka (sashe na 4.5.3)

Idan kuna tunanin ƙoƙarin ƙirƙirar samfurinka, na bada shawara cewa ka karanta shawarar da kungiyar MovieLens ta bayar a Harper and Konstan (2015) . Gani mai mahimmanci daga kwarewarsu shine cewa ga kowane aikin ci gaba akwai matakai masu yawa, da dama. Alal misali, ƙungiyar MovieLens ta kaddamar da wasu samfurori, irin su GopherAnswers, wadanda suka kasance kasawa (Harper and Konstan 2015) . Wani misali na mai bincike da yayi kasa yayin ƙoƙari ya gina samfurin shi ne ƙoƙari na Edward Castronova na gina wani wasan intanet wanda ake kira Arden. Duk da $ 250,000 a cikin kudade, wannan aikin shi ne flop (Baker 2008) . Ayyuka kamar GopherAnswers da Arden suna da rashin alheri fiye da ayyukan kamar MovieLens.

• Abokan hulɗa da mai iko (sashi na 4.5.4)

Na ji ra'ayin Basteur na Quadrant da aka tattauna akai-akai a kamfanonin fasaha, kuma yana taimakawa wajen gudanar da bincike kan Google (Spector, Norvig, and Petrov 2012) .

Bond da kuma abokan aiki (2012) binciken) (2012) ma yayi ƙoƙarin gano sakamakon wadannan jiyya akan abokan waɗanda suka karbi su. Saboda zanewar gwaji, waɗannan rushewa suna da wuyar ganowa tsabta; masu sha'awar karatu su ga Bond et al. (2012) don tattaunawa sosai. Jones da abokan aiki (2017) sun gudanar da irin wannan gwagwarmaya a lokacin zabe na 2012. Wadannan gwaje-gwaje na daga cikin dogon lokaci na gwaje-gwaje a kimiyya na siyasa akan ƙoƙarin ƙarfafa yin zabe (Green and Gerber 2015) . Wadannan gwaje-gwaje-gwaje-gwaje ne na kowa, a wani ɓangare saboda suna a cikin Pasteur's Quadrant. Wato, akwai mutane da yawa waɗanda suke da ƙarfin yin ƙaura da kuma jefa kuri'a na iya zama hali mai ban sha'awa don gwada gwagwarmaya da yawa game da canza halin hali da kuma tasirin zamantakewa.

Don shawara game da gwajin gwaje-gwaje tare da kungiyoyi masu zaman kansu kamar jam'iyyun siyasa, kungiyoyi masu zaman kansu, da kasuwanni, ga Loewen, Rubenson, and Wantchekon (2010) , JA List (2011) , da kuma Gueron (2002) . Don tunani game da yadda haɗin gwiwa da ƙungiyoyi zasu iya tasiri na bincike, duba King et al. (2007) da Green, Calfano, and Aronow (2014) . Abokan hulɗa na iya haifar da tambayoyi masu kyau, kamar yadda Humphreys (2015) da Nickerson and Hyde (2016) tattauna.

• Shawara shawara (sashi na 4.6)

Idan kuna shirin kirkirar shirin kafin gudanar da gwajinku, ina bayar da shawarar cewa ku fara ne ta hanyar karanta jagoran rahoto. An tsara ka'idodin Ƙungiyar (Asalin Tattaunawa ta Gaskiya) da magani (Schulz et al. 2010) kuma an gyara shi don nazarin zamantakewa (Mayo-Wilson et al. 2013) . An tsara sassan jagororin da suka shafi jagororin jaridar Journal of Experimental Political Science (Gerber et al. 2014) (duba Mutz and Pemantle (2015) da kuma Gerber et al. (2015) ). A ƙarshe, an tsara ka'idodin rahoto a cikin ilimin halin mutum (APA Working Group 2008) , kuma ga Simmons, Nelson, and Simonsohn (2011) .

Idan ka ƙirƙiri wani tsari na bincike, ya kamata ka yi la'akari da rajista kafin yin rajistar shi saboda yin rajista zai ƙara ƙarfafawa cewa wasu suna cikin sakamakonka. Bugu da ari, idan kuna aiki tare da abokin tarayya, zai rage iyakar abokin ku don canza bincike bayan ganin sakamakon. Bayanin rajista yana ci gaba da kasancewa a cikin ilimin halin mutum (Nosek and Lakens 2014) , kimiyyar siyasa (Humphreys, Sierra, and Windt 2013; Monogan 2013; Anderson 2013; Gelman 2013; Laitin 2013) , da kuma tattalin arziki (Olken 2015) .

Shawarar shawara ta musamman don gwaje-gwaje na filin yanar gizo an gabatar da shi a Konstan and Chen (2007) da kuma Chen and Konstan (2015) .

Abin da na kira dabarun armada shine wasu lokuta ana kira aikin bincike ; duba Wilson, Aronson, and Carlsmith (2010) .

• Ƙirƙiri bayanan farashi maras nauyi (sashi na 4.6.1)

Don ƙarin bayani game da gwaje-gwajen Salganik, Dodds, and Watts (2006) , Salganik and Watts (2008) , Salganik and Watts (2009b) , Salganik and Watts (2009a) , da Salganik (2007) . Don ƙarin bayani game da kasuwanni masu cin nasara, duba Frank and Cook (1996) . Don ƙarin bayani game da sa'a da fasaha mafi yawa, duba Mauboussin (2012) , Watts (2012) , da kuma Frank (2016) .

Akwai wata hanya ta kawar da biyan kuɗi na masu shiga wanda masu bincike su yi amfani da hankali: kulla yarjejeniya. A yawancin gwaje-gwaje a kan layi mahalarta an tsara su a cikin gwaje-gwaje kuma ba a biya su ba. Misalai na wannan hanya sun haɗa da gwajin Restivo da na van de Rijt (2012) akan sakamako a Wikipedia da Bond da kuma abokin aikin (2012) akan ƙarfafa mutane su jefa kuri'a. Wadannan gwaje-gwajen ba su da wani nau'i mai tsada-maimakon haka, suna da tsada ga masu bincike . A irin waɗannan gwaje-gwajen, ko da koda kowane mai haɗin kuɗi yana da ƙananan ƙananan, farashin kuɗi zai iya zama babba. Masu bincike da ke gudanar da gwaje-gwaje a kan layi sau da yawa suna nuna muhimmancin kananan maganin maganin jiyya ta hanyar cewa waɗannan ƙananan illa zasu iya zama mahimmanci idan ana amfani da mutane da yawa. Hakanan tunani daidai ne akan halin da masu bincike suka gabatar a kan mahalarta. Idan fitinarka ya sa mutane miliyan daya su rabu da minti daya, gwajin ba shi da wata illa ga kowane mutum, amma a tara shi ya yi kusan kusan shekaru biyu.

Wata mahimmanci don ƙirƙirar farashin biyan kuɗi mai sauƙi ga mahalarta shine yin amfani da caca, wani mahimmanci da aka yi amfani dashi a binciken bincike (Halpern et al. 2011) . Don ƙarin bayani game da zayyana kwarewa masu amfani, duba Toomim et al. (2011) . Don ƙarin bayani game da amfani da buƙatu don ƙirƙirar gwajin gwajin da za a iya ɗaukar nauyin farashin ganin ( ??? ) .

• Sauya, Ƙasa, da Rage (sashi na 4.6.2)

Rikoki uku na R kamar yadda Russell and Burch (1959) sune:

"Sauyawa nufin canzawa ga m rayuwa mafi girma dabbobi da insentient abu. Saukarwa nufin rage a cikin lambobin dabbobi amfani da su samu bayanai daga wani da aka ba adadin da daidaici. Tsaftacewa nufin wani karu a faru ko tsananin tausayi hanyoyin amfani ga waɗanda dabbobi wanda har yanzu da za a amfani da su. "

Abubuwan uku na R da na ba da shawara ba su ƙetare ka'idodin ka'idodin da aka bayyana a babi na 6. Maimakon haka, sune fasali mafi mahimmanci daya daga waɗannan ka'idodin-amfana-musamman a cikin tsarin gwaje-gwajen mutum.

Game da na farko R ("maye gurbin"), kwatanta gwajin gwagwarmayar kwakwalwa (Kramer, Guillory, and Hancock 2014) da kuma tunanin da ke tattare da gwaji na halitta (Lorenzo Coviello et al. 2014) ba da wasu darussa game da masu cinikayya. a cikin gwaje-gwaje zuwa gwaje-gwaje na halitta (da kuma sauran hanyoyin kamar matsala da ƙoƙari na kimanta gwaje-gwaje a cikin bayanan gwaji ba; duba babi na 2). Bugu da ƙari, amfani mai amfani, sauyawa daga gwaji ga nazarin gwaji ba kuma ya sa masu bincike suyi nazarin maganin da basu iya aiki ba. Wadannan samfurori da kuma masu amfani da shi sun kasance a kan farashi, duk da haka. Tare da gwaje-gwaje na al'ada masu bincike basu da iko akan abubuwa kamar daukar nauyin mahalarta, ƙididdiga, da kuma yanayin magani. Alal misali, ƙuntataccen ruwan sama a matsayin magani shi ne cewa duka yana ƙaruwa da haɓakawa. A cikin gwajin gwajin, Kramer da abokan aiki sun iya daidaita daidaituwa da haɓaka kai tsaye. Lorenzo Coviello et al. (2014) ya kara bayani daga L. Coviello, Fowler, and Franceschetti (2014) . Don gabatarwa ga maɓalli na kayan aiki, wanda shine tsarin da Lorenzo Coviello et al. (2014) , ga Angrist and Pischke (2009) (m) ko Angrist, Imbens, and Rubin (1996) (mafi muni). Don ƙwararrakin ƙwarewar maɓalli na kayan aiki, duba Deaton (2010) , da kuma gabatarwa ga maɓalli na kayan aiki tare da kayan ƙarfi (ruwan sama wani abu ne mai rauni), duba Murray (2006) . Bugu da ƙari, Dunning (2012) , gabatarwa mai kyau ga gwaje-gwaje na halitta, yayin da Rosenbaum (2002) , ( ??? ) , da Shadish, Cook, and Campbell (2001) suna bayar da kyakkyawan ra'ayi game da kimanta sakamakon lalacewar ba tare da gwaje-gwajen ba.

Dangane da na biyu R ("tsaftacewa"), akwai fasaha na kimiyya da na kasuwanci a lokacin da suke canzawa da zane na Contagion na Muryar daga rufe matsaloli don tallafawa posts. Alal misali, yana iya kasancewa cewa sha'anin fasaha na News Feed yana da sauƙin yin wani gwaji wanda aka katange ginshiƙai fiye da ɗaya wanda aka ƙarfafa su (lura cewa ana iya aiwatar da gwajin da aka kulla da rufewa daga posts a matsayin Layer a saman Cibiyar Kula da Labaran Duniya ba tare da wani buƙatar gyare-gyare na tushen tsarin ba). Masanin kimiyya, duk da haka, ka'idar da aka yi ta gwaji ta hanyar gwaji ba ta bayar da ra'ayi daya akan juna ba. Abin baƙin cikin shine, ban sani ba game da ƙaddamar da bincike game da muhimmancin haɓakawa da bunkasa abubuwan ciki a cikin News Feed. Bugu da ƙari, ban gani da yawa bincike game da tsabtace jiyya ba don sa su zama marasa cutarwa; Daya daga cikinsu shine B. Jones and Feamster (2015) , wanda ya ɗauki la'akari da ƙwarewar yanar-gizon (wani batu na tattauna a babi na 6 a cikin dangantaka da Nazarin (Burnett and Feamster 2015; Narayanan and Zevenbergen 2015) ).

Dangane da na uku R ("rage"), Cohen (1988) (littafin) da Cohen (1992) ( Cohen (1988) (littafin) da Cohen (1992) sun bayar da kyakkyawar gabatarwar su a yayin da Gelman and Carlin (2014) ba da hangen nesa. Za a iya haɗa nauyin haɗin gwiwar rigakafi a cikin tsari da bincike na gwaje-gwajen; babi na 4 na Gerber and Green (2012) samar da kyakkyawan gabatarwar zuwa duka hanyoyin biyu, kuma Casella (2008) tana ba da ƙarin kulawa mai zurfi. Hanyoyin da ke amfani da wannan bayanin da aka riga sunadawa a cikin ƙididdigar sunaye ake kira ko dai an katange kayan gwajin ko samfurin gwaji (ba'a amfani da kalmar ba tare da amfani ba a cikin al'umma); waɗannan dabarun suna da alaƙa da alaka da samfurin samfurin samfurin da aka tattauna a babi na 3. Dubi Higgins, Sävje, and Sekhon (2016) don ƙarin amfani da waɗannan kayayyaki a cikin gwaje-gwaje masu yawa. Za a iya hada magungunan kwakwalwa a cikin aikin bincike. McKenzie (2012) bincika bambancin-bambancin da ke tattare da nazarin gwaje-gwaje a cikin zurfin bayanai. Dubi Carneiro, Lee, and Wilhelm (2016) don ƙarin bayani a kan cinikayya tsakanin hanyoyin daban-daban don ƙara ƙayyadewa cikin ƙididdigar sakamako. A ƙarshe, lokacin da za a yanke shawara ko ƙoƙari ya haɗa da haɗin gwargwadon maganin farko a tsarin zane ko mataki na bincike (ko duka biyu), akwai wasu dalilai da za a bincika. A wani wuri inda masu bincike suke so su nuna cewa basu "kama" (Humphreys, Sierra, and Windt 2013) , ta hanyar amfani da magungunan rigakafi a tsarin zane na iya taimaka (Higgins, Sävje, and Sekhon 2016) . A cikin yanayi inda masu halartar taron suka zo daidai, musamman nazarin filin yanar gizo, ta hanyar amfani da bayanan kulawa a cikin tsarin zane na iya zama mai wuyar gado; ga misali, Xie and Aurisset (2016) .

Yana da daraja ƙara ƙarin bayani game da dalilin da yasa bambancin bambanci-bambanci zai iya zama mafi tasiri fiye da bambanci-a-daya daya. Yawancin sakamako na intanet suna da matukar bambanci (ga misali, RA Lewis and Rao (2015) da kuma Lamb et al. (2015) ) kuma suna da kwanciyar hankali a tsawon lokaci. A wannan yanayin, zaɓin canji zai kasance da ƙananan ƙwayar cuta, ƙara ƙarfin jarrabawar lissafi. Ɗaya daga cikin dalilan da ba'a amfani dasu ba a lokuta mafi yawa shi ne cewa kafin shekarun dijital, ba al'ada ba ne don samun sakamako na rigakafi. Hanyar da ta fi dacewa don yin la'akari da wannan shine tunanin wani gwaji don auna ko wani kwarewa na yau da kullum yana sa asarar nauyi. Idan ka yi amfani da bambanci-a-ma'ana kusanci, ƙayyadadden ku zai sami canji wanda ya samo daga canji a ma'auni a cikin yawan jama'a. Idan kuna yin kuskuren bambance-bambance, duk da haka, an cire yanayin sauye-sauye a ma'aunin nauyi, kuma zaka iya samun sauƙin gane bambancin da magani ya haifar.

A ƙarshe, na yi la'akari da ƙara na huɗu R: "sake dawowa". Wato, idan masu bincike sun sami kansu da bayanan gwaji fiye da yadda suke buƙatar magance tambayoyin bincike na asali, sai su sake dawo da bayanan don tambayi sababbin tambayoyi. Alal misali, ka yi tunanin cewa Kramer da abokan aiki sun yi amfani da ƙididdigar bambance-bambancen da suka bambanta kuma suka sami kansu da bayanan da suka fi dacewa don magance tambayoyin bincike. Maimakon ba amfani da bayanai ba har abada, da sun iya nazarin girman sakamakon a matsayin aikin maganganun maganganu. Kamar yadda Schultz et al. (2007) gano cewa tasirin maganin ya bambanta ga masu haske da masu amfani, watakila sakamakon Labarai na da banbanci ga mutanen da suka riga sun kula da sakon farin ciki (ko bakin ciki). Rubucewar zai iya haifar da "kifi" (Humphreys, Sierra, and Windt 2013) da kuma "p-hacking" (Simmons, Nelson, and Simonsohn 2011) , amma waɗannan sun fi mayar da hankali tare da haɗin gaskiyar labarai (Simmons, Nelson, and Simonsohn 2011) , pre-rajista (Humphreys, Sierra, and Windt 2013) , da kuma hanyoyin ilmantarwa na na'ura waɗanda suke ƙoƙari su guje wa gwadawa.