4.4.1 Kev siv tau

Siv tau qhia txog ntau npaum li cas cov kev tshwm sim ntawm ib tug xyaum ua tej yam txhawb ib tug ntau general xaus.

Tsis xyaum ua tej yam yog zoo meej, thiab soj ntsuam ntawm muaj tsim ib qho uas nws kim heev cov lus los piav qhia txog tau teeb meem. Siv tau qhia txog rau cov neeg uas cov kev tshwm sim ntawm ib tug xyaum ua tej yam txhawb ib co ntau general xaus. Social zaum tau pom nws yuav pab tau phua validity rau hauv plaub lub ntsiab hom: statistical xaus validity, nrog validity, txua validity, thiab lwm validity (Shadish, Cook, and Campbell 2001, Ch 2) . Mastering cov tswv yim yuav muab rau koj puas siab puas ntsws daim ntawv rau critiquing thiab kev txhim kho cov kev tsim thiab tsom xam ntawm ib tug xyaum ua tej yam, thiab nws yuav pab koj sib txuas lus nrog lwm yam kev soj ntsuam.

Statistical xaus validity chaw zov me nyuam nyob ib ncig ntawm seb tus tsom ntawm qhov xyaum ua tej twb ua kom raug. Nyob rau hauv lub ntsiab lus teb ntawm Schultz et al. (2007) xws lo lus nug tej zaum yuav center ntawm seb lawv xoo lawv p-qhov tseem ceeb kom raug. Tsom yog tshaj lub Scope ntawm phau ntawv no, tab sis kuv yuav hais tias lub statistical hauv paus ntsiab lus uas yuav tsum tau los tsim thiab tsom xam thwmsim tsis tau hloov nyob rau hauv lub digital muaj hnub nyoog. Txawm li cas los, qhov sib txawv cov ntaub ntawv ib puag ncig nyob rau hauv cov kev tsis tsim tshiab statistical txuj (piv txwv li, siv lub tshuab kev kawm txoj kev los laij cov nqi heterogeneity ntawm kev kho mob los (Imai and Ratkovic 2013) ) thiab cov tshiab computational txoj kev sib tw (eg, thaiv cov hlab nyob rau hauv loj heev thwmsim (Higgins, Sävje, and Sekhon 2016) ).

Internal validity chaw zov me nyuam nyob ib ncig ntawm seb lub sim cov txheej txheem tau ua kom raug. Rov qab mus rau cov xyaum ua tej yam ntawm Schultz et al. (2007) , cov lus nug hais txog nrog validity yuav center nyob ib ncig ntawm lub randomization, tus me nyuam ntawm cov kev kho mob, thiab kev ntsuas ntawm kev tshwm sim. Piv txwv li, tej zaum koj yuav muaj kev txhawj xeeb hais tias cov kev tshawb fawb pab tsis nyeem hluav taws xob meters nti. Nyob rau hauv qhov tseeb, Schultz thiab lug txhawb cov miv twb txhawj xeeb txog qhov teeb meem no thiab lawv tau muaj ib tug qauv ntawm meters nyeem ob zaug; qhov zoo ces, soj ntsuam tau yeej tseem zoo tib yam. Nyob rau hauv kev, Schultz thiab lug txhawb cov miv 'xyaum ua tej yam zoo nkaus li muaj siab nrog validity, tab sis qhov no yog tsis ib txwm rooj plaub; complex teb thiab hauv internet thwmsim feem ntau khiav mus rau teeb meem ua tau xa txoj cai kev kho mob rau txoj cai neeg thiab ntsuas tus ua tau rau txhua leej txhua tus. Qhov zoo ces, cov muaj hnub nyoog yuav pab txo cov kev txhawj xeeb txog internal validity vim hais tias nws yuav ua rau nws yooj yim mus xyuas kom meej tias cov kev kho mob yog xa raws li tsim rau cov neeg uas yuav tsum tau txais nws thiab los ntsuas tau rau tag nrho cov neeg.

Tsim validity chaw ib ncig ntawm lub match nruab nrab ntawm cov ntaub ntawv thiab cov theoretical constructs. Raws li sib tham nyob rau hauv Tshooj 2, constructs yog paub daws teeb lub tswv yim uas social zaum yog vim li cas hais txog. Tu siab, cov paub daws teeb lub tswv yim yeej tsis muaj tseeb txhais cov ntsiab lus thiab ntsuam. Rov qab mus rau Schultz et al. (2007) , tus hais tias injunctive kev cai yuav txo hluav taws xob siv yuav tsum tau soj ntsuam los tsim ib tug kev kho mob uas yuav muab "injunctive kev cai" (piv txwv li, ib tug Emoticon) thiab ntsuas "hluav taws xob siv". Nyob rau hauv analog thwmsim, ntau soj ntsuam ntawm tsim lawv tus kheej kev kho mob thiab ntsuas lawv tus kheej ua tau. Qhov no mus kom ze kom hais tias, raws li ntau li ntau tau, lub thwmsim phim cov abstract constructs raug kawm. Nyob rau hauv cov thwmsim qhov twg soj ntsuam tus khub nrog cov tuam txhab los yog tsoom fwv xa kev kho mob thiab siv yeej ib txwm-rau cov ntaub ntawv systems los ntsuas tau, lub match ntawm qhov xyaum ua tej thiab lub theoretical constructs tej zaum yuav tsawg ntom. Yog li, kuv xav hais tias dlaim validity yuav yuav ib tug loj kev txhawj xeeb nyob rau hauv cov kev tshaj analog thwmsim.

Thaum kawg, lwm cov validity chaw zov me nyuam nyob ib ncig ntawm seb cov kev tshwm sim ntawm no xyaum ua tej yam yuav generalize mus rau lwm cov teeb meem. Rov qab mus rau Schultz et al. (2007) , ib tug yuav nug, yuav tib lub tswv yim-muab cov neeg ntaub ntawv hais txog lawv lub zog pab nyob rau hauv kev sib raug zoo rau lawv cov phooj ywg thiab ib lub teeb liab ntawm injunctive cai (xws li, ib tug Emoticon) -reduce zog pab yog hais tias nws tau ua nyob rau hauv ib tug txawv txoj kev nyob rau hauv ib tug txawv chaw? Rau feem ntau zoo-tsim thiab zoo-khiav thwmsim, kev txhawj xeeb txog lwm validity yog qhov nyuaj tshaj plaws rau qhov chaw nyob. Nyob rau hauv lub dhau los lawm, cov sib cav tswv yim hais txog lwm validity twb nquag cia li ib Rev ntawm cov neeg zaum nyob rau hauv ib chav tsev ua rau xav txog tej yam dab tsi yuav muaj tshwm sim yog hais tias tus txheej txheem tau ua nyob rau hauv ib tug txawv txoj kev, los yog nyob rau hauv ib qho chaw sib txawv, los yog nrog cov neeg sib txawv. Qhov zoo ces, cov muaj hnub nyoog enables soj ntsuam dhau mus cov ntaub ntawv-free speculations thiab ntsuam xyuas sab nraud validity empirically.

Vim hais tias cov kev tshwm sim los ntawm Schultz et al. (2007) thiaj li exciting, ib lub tuam txhab muaj npe Opower koom tes nrog hlauv taws xob nyob rau hauv lub tebchaws United States mus rau deploy cov kev kho mob ntau lug. Raws li cov qauv siv los ntawm Schultz et al. (2007) , Opower tsim Mekas Tsev Zog Reports uas muaj ob lub ntsiab modules, ib tug qhia ib tug neeg hauv tsev neeg lub hluav taws xob pab txheeb ze rau nws cov neeg nyob ze nrog ib tug Emoticon thiab ib tug muab lub tswv yim rau txo zog pab (Daim duab 4.6). Ces, nyob rau hauv kev koom tes nrog soj ntsuam, Opower khiav xaiv tswj thwmsim rau kev ntsuam xyuas tej yam uas cov tsev Zog Reports. Txawm tias cov kev kho mob nyob rau hauv cov thwmsim tau feem ntau tauj lub cev-feem ntau yog los ntawm qub fashioned qwj mail-lub sij hawm tau ntsuas siv cov pab kiag li lawm nyob rau hauv lub ntiaj teb no (piv txwv li, lub hwj chim meters). Es tsis manually sau cov ntaub ntawv no nrog kev tshawb fawb pab mus xyuas txhua lub tsev, lub Opower thwmsim tau tag nrho cov ua nyob rau hauv kev sib koom tes nrog lub hwj chim tuam txhab uas muag muag txog cai rau kev soj ntsuam mus saib tau lub hwj chim readings. Yog li, cov cov digital teb thwmsim tau khiav ntawm ib tug loj heev scale thaum uas tsis muaj nce mus nce los them tus nqi.

Daim duab 4.6: Lub Tsev Zog Reports nyob rau hauv Allcott (2011) muaj ib tug Social Piv Module thiab ib tug Action kauj ruam Module.

Nyob rau hauv ib tug thawj txheej ntawm thwmsim uas 600,000 tsev neeg tau txais kev pab los ntawm 10 nqi hluav taws xob tuam txhab uas muag nyob ib ncig ntawm lub tebchaws United States, Allcott (2011) nyob hauv lub Tsev Zog Daim ntawv qhia txog nws txo qis hluav taws xob noj los ntawm 1.7%. Nyob rau hauv lwm yam lus, cov kev tshwm sim los ntawm cov loj npaum li cas, ntau thaj nyuag miv txoj kev tshawb no twb qualitatively zoo xws li cov tau los ntawm Schultz et al. (2007) . Tab sis, cov nyhuv loj yog me me: nyob rau hauv Schultz et al. (2007) lub tsev neeg nyob rau hauv lub piav thiab injective cai mob (ib tug nrog lub Emoticon) txo lawv hluav taws xob pab los ntawm cov 5%. Cov leej yog vim li cas rau qhov no sib txawv yog tsis paub, tab sis Allcott (2011) speculated tias txoj kev txais ib lub handwritten Emoticon raws li ib feem ntawm ib txoj kev kawm tau kev txhawb nqa los ntawm lub tsev kawm ntawv tej zaum yuav muaj ib tug loj ntxim rau cov cwj pwm tshaj uas tau txais ib tug luam Emoticon raws li ib feem ntawm ib tug loj ua daim ntawv qhia los ntawm ib tug hwj chim lub tuam txhab.

Ntxiv mus, nyob rau hauv tom ntej kev tshawb fawb, Allcott (2015) qhia rau ib tug ntxiv 101 thwmsim uas ib tug ntxiv 8 lab tsev neeg. Nyob rau hauv cov tom ntej no 101 thwmsim lub tsev Zog Daim ntawv qhia txog txuas ntxiv mus ua rau neeg kom txo tau lawv hluav taws xob noj, tab sis cov teebmeem twb txawm me me thiab. Cov leej yog vim li cas rau qhov no poob yog tsis paub, tab sis Allcott (2015) speculated tias cov hauj lwm zoo ntawm daim ntawv qhia nyob rau yuav koos lub sij hawm vim hais tias nws twb tau thov rau ntau hom kev koom. Dua, hlauv taws xob nyob rau hauv ntau environmentalist chaw nyob yuav txais yuav cov kev pab cuam ua ntej lawm thiab lawv cov neeg muas zaub ntau teb rau cov kev kho mob. Raws li hlauv taws xob nrog tsawg ib puag ncig cov neeg muas zaub tau txais qhov kev pab cuam, nws cov hauj lwm zoo nyob rau poob. Yog li, cia li raws li randomization nyob rau hauv sim kawm kom hais tias cov kev kho mob thiab pab pawg neeg tswj yog zoo sib xws, randomization nyob rau hauv kev tshawb fawb chaw kom hais tias cov kev kwv yees yuav generalized los ntawm ib tug ib pab pawg neeg ntawm cov neeg mus rau ib tug ntau pejxeem (xav hais tias rov qab mus rau Tshooj 3 txog zauv). Yog hais tias kev tshawb fawb chaw tsis sampled across, ces generalization-txawm los ntawm ib txig tsim thiab ua xyaum ua tej yam-yuav ua tau problematic.

Ua ke, cov 111 thwmsim-10 nyob rau hauv Allcott (2011) thiab 101 nyob rau hauv Allcott (2015) -involved txog 8.5 lab cov tsev neeg los ntawm thoob plaws lub tebchaws United States. Lawv li qhia siv kuj qhia tias tsev Zog Reports txo nruab nrab hluav taws xob noj, ib tug tshwm sim uas txhawb nqa cov thawj uas nrhiav tau ntawm Schultz thiab lug txhawb cov miv los ntawm 300 lub tsev nyob rau hauv California. Tshaj ntawd cia li replicating cov thawj tau, tus ua raws-up thwmsim kuj qhia tau tias qhov luaj li cas ntawm cov nyhuv mas nws txawv ntawm qhov chaw nyob. Cov teeb no ntawm thwmsim kuj tso ob tug ntau general ntsiab lus hais txog cov cov teb thwmsim. Ua ntej, soj ntsuam yuav tsum tau empirically chaw nyob kev txhawj xeeb txog lwm validity thaum tus nqi ntawm khiav thwmsim yog tsawg tsawg, thiab qhov no yuav tshwm sim yog tias lub sij hawm twb raug ntsuas los ntawm ib tug ib txwm-rau cov ntaub ntawv system. Yog li ntawd, nws qhia tias kev tshawb fawb yuav tsum muaj nyob rau ntawm qhov zoo-tawm rau lwm nthuav thiab tseem ceeb cwj pwm uas twb raug kaw, thiab ces tsim thwmsim rau sab saum toj ntawm no uas twb muaj lawm xab infrastructure. Ob txhais, qhov no set ntawm thwmsim rau peb nco txog tias cov teb thwmsim tsis yog nyob hauv internet; nce Kuv xav hais tias lawv yuav qhov txhia chaw uas muaj ntau yam tshwm sim ntsuas los ntawm sensors nyob rau hauv lub ua tau ib puag ncig.

Cov plaub hom validity-statistical xaus validity, nrog validity, txua validity, lwm validity-muab ib tug puas siab puas ntsws daim ntawv los pab kev soj ntsuam ntsuam xyuas seb cov kev tshwm sim los ntawm ib tug kev xyaum ua tej yam txhawb ib tug ntau general xaus. Piv rau analog muaj hnub nyoog thwmsim, nyob rau hauv cov muaj hnub nyoog thwmsim nws yuav tsum yooj yim mus rau qhov chaw sab nraud validity empirically thiab nws yuav tsum tau yooj yim los xyuas kom meej nrog validity. Nyob rau lwm cov tes, tej teeb meem ntawm dlaim validity tej zaum yuav ntau nyuaj nyob rau hauv cov muaj hnub nyoog thwmsim (txawm hais tias hais tias yog tsis yog cov ntaub ntawv nrog rau lub Opower thwmsim).