大三上課件-統(tǒng)計(jì)error diagnostic test

Chp14

ClinicalMeasurementErrorandDiagnosisTest

臨床測(cè)量誤差與診斷試驗(yàn)

Dr.MinShenSchoolofPublicHealth

TongjiMedicalCollegeMedicalmeasurementpatientssexageweightheightbloodpressurebloodsugar1…………23Hypertension?Diabetes?hemomanometerchecker

glucosemonitorsMeasurementErrorAparticipant’sscoreonaparticularmeasureconsistsof2components:Observedscore=Truescore+MeasurementErrorTrueScore=scorethattheparticipantwouldhaveobtainedifmeasurementwasperfect—i.e.,wewereabletomeasurewithouterror(i.e.Height175cm)MeasurementError=thecomponentoftheobservedscorethatistheresultoffactorsthatdistortthescorefromitstruevalue(0.5cm)1.Sourcesofmeasurementerror2.Assessmentofmeasurementerror3.EvaluationofconsistencyClinicalMeasurementErrorSymptomsandsignsBiologicalvariationMeasurementdataSystematicerrorRandomerrorMedicaldiagnosisSystematicerrorRandomerrorMeasuremente=thetruevalue+systematicerror+biologicalvariation+randomerror+othererrorMeasurementerror1.SourcesofMeasurementError0101

SystematicerrorSystematicerrorsareerrorsthatproducearesultthatdiffersfromthetruevaluebyafixedamount

(thesamedirection).Theseerrorsresultfrombiasesintroducedbyinstrumentalmethod,orhumanfactorsSources:Instrumental,physicalandhumanlimitations.Example:Deviceisout-ofcalibration,type,manufactory,lotnumberHowtominimizeit?Carefulcalibration.BestpossibletechniquesAreTYPICALLYpresent?systematicerrorIfyouaskthepersonwhosellsfishatyourfavoritemarkettoweightapieceoffishseveraltimesforyou,andheputshisthumbonthescaleinawaythatmakesthefishseem2ouncesheavierthanitis,that’sbias--asystematictendencytooverorunderestimatethetruevalue.Noticethatsystematicerrordoesn’tmovearound

fromobservationtoobservation--that’swhatmakesitsystematic.

Randomerroristheirreproducibilityinmakingreplicatemeasurementsandaffectstheprecisionofaresult.

Source:

NaturebiologicalvariabilityMeasurementerrors

Howtominimizeit?

Takemoredata.

Randomerrors

canbeevaluatedthroughstatisticalanalysisandcanbereducedbyaveragingoveralargenumberofobservations.

RandomErrorHowtominimizeexperimentalerror:someexamplesTypeofErrorExampleHowtominimizeitRandomerrorsYoumeasurethemassofaringthreetimesusingthesamebalanceandgetslightlydifferentvalues:17.46g,17.42g,17.44gTakemoredata.Randomerrorscanbeevaluatedthroughstatisticalanalysisandcanbereducedbyaveragingoveralargenumberofobservations.SystematicerrorsTheclothtapemeasurethatyouusetomeasurethelengthofanobjecthadbeenstretchedoutfromyearsofuse.(Asaresult,allofyourlengthmeasurementsweretoosmall.)Theelectronicscaleyouusereads0.05gtoohighforallyourmassmeasurements(becauseitisimproperlytaredthroughoutyourexperiment).Systematicerrorsaredifficulttodetectandcannotbeanalyzedstatistically,becauseallofthedataisoffinthesamedirection(eithertohighortoolow).Spottingandcorrectingforsystematicerrortakesalotofcare.Howwouldyoucompensatefortheincorrectresultsofusingthestretchedouttapemeasure?（你將如何補(bǔ)償拉伸卷尺使用不正確的結(jié)果）Howwouldyoucorrectthemeasurementsfromimproperlytaredscale?（你如何??糾正不正確的測(cè)量值？）Accuracy:Incertainmeasurementconditions,thedegreeofagreementbetweentheaveragelevelsofmeasurementresultsandthetruevalue.systematicerroraccuracyHowclosemeanofmeasuredvaluesistotruevaluePrecision:

Incertainmeasurementconditions,thedegreeofdispersionbetweentheobservedvalues??.randomerrorprecision

RepeatabilityofmeasurementsAccuracyvs.PrecisionInordertobeprecise,atestmustbeaccurate;butprecisiondoesnotguaranteeaccuracy.PrecisenotaccurateAccuratenotpreciseNeitherprecisenoraccurateBothpreciseandaccurate

Scientistsalwayswantthemostpreciseandaccurate

experimentaldata.

Theprecisionandaccuracyarelimitedbytheinstrumentationanddatagatheringtechniques.

Outlier:Statisticaldatawhichisextremelydifferentfromtheothersinthesamesample.Instatistics,anoutlierisanobservationthatisnumericallydistantfromtherestofthedataOutlierscanoccurbychanceinanydistribution,buttheyareoftenindicativeeitherofmeasurementerrororthatthepopulationhasaheavy-taileddistribution.Statisticalanalysis:discardthemorusestatisticsthatarerobusttooutliersFigure1.BoxplotofdatafromtheMichelson-MorleyExperimentdisplayingoutliersinthemiddlecolumnBias:Asystematicerrorindataduetothecharacteristicsoftheprocessemployedinthecreation,collection,manipulation,andpresentationofdata,orduetofaultysampledesignoftheestimatingtechnique.MeasurementError=(observedvalue–truevalue)=systematicerror+randomerror

biasSelectionandinformationbiasesSourcesofSelectionBiasInappropriatepopulationstudiedInadequateparticipationChangeofclassificationofthedeterminantSelectionofmost‘a(chǎn)ccessible’orofvolunteersInadequateParticipationWewanttostudytheassociationofstigmawithdiagnosisofTBWeselectasampleofthepopulation20%ofthesampleagreetoparticipateWefindthatthereisnoassociationofstigmawithTBIsthistrue?InappropriatePopulationWewishtomeasuretheimpactofHIVontuberculosisWestudythetrendoftuberculosisinWuhanfrom1997to2001WefindnochangeinnotificationrateWeconcludethatHIVhasnoimpactonTBWereweright?ClassificationofDeterminantWewanttostudytheimpactofpovertyonthetrendoftuberculosisinWuhanWeselectapoordistrictandarichdistrictandcomparethenotificationofTBfrom1991to2000Inthemeantime,thereisan‘urbanrenewal’projectinthepoordistrictWefindnodifferencebetweenthedistrictsCanweconcludethatpovertyisnotrelatedtoTB?ParticipationofVolunteersWewanttodeterminetheprevalenceofHIVinfectioninWuhanWeaskforvolunteersfortestingWefindnoHIVIsitcorrecttoconcludethatthereisnoHIVinWuhan?variancecomponentanalysis(方差分量法）

signal-to-noiseratio(SNR)

2.Assessmentofmeasurementerror1)Variancecomponentanalysis:

Thebasicideaforvariancecomponentanalysisistoseparatethetotalvariationintovariancesfromdifferentsources,likeindividualvariabilityandmeasurementrepeatedvariabilityandtocomparetheratioofeachvariancecomponenttothetotalvariation.Iftheratioofthevariancecomponentofrepeatedmeasurementerrortothetotalvariationissmall,itindicatesthatitisameasurementofhighreliability.隨機(jī)誤差的度量同樣條件下多次重復(fù)測(cè)量，用全部觀察值結(jié)果的標(biāo)準(zhǔn)差作為測(cè)量誤差的度量:variancecomponentofindividualvariability.

個(gè)體之間的變異或誤差的方差分量:variancecomponentofrepeatedvariability.受試者內(nèi)多次測(cè)量結(jié)果的變異/誤差estimatemeasurementerroranalysisvarianceoftwomeasurementerrorExpectationofMSExample:例14-1血清載脂蛋白B在粥樣動(dòng)脈硬化、冠心病等疾病研究中具有重要意義。為了解其測(cè)量誤差，選擇8名受試者的一份血樣分成3份，分別進(jìn)行三次測(cè)量，試對(duì)其進(jìn)行評(píng)價(jià)。Table14-1threemeasurementsofSerumapolipoproteinB(mg/dl)NofirstsecondthirdTotal(B)Sumofsquare11201099932836028212612311836744929312112414138649898410013414037447556596979729028034610710510832034138711713511536745139813414012039451956total2826337678Example:8×3σ2w=S2w=MSe

The95%maximumerrorrangeofrepeatedmeasurementforeachsubject每一受試者重復(fù)測(cè)量的95%的最大誤差范圍：ICC(intraclasscorrelationcoefficient)組內(nèi)相關(guān)系數(shù)ICCisreliabilityindex.(1>ICC>0)（信度）ICC1:repeatability（測(cè)量誤差?。?/p>

K=3:timesofrepeatmeasurement=0.48onetime=0.735

Errorassessment---SNR(signalnoiseratio)TheSNRisusedtomeasuretheerrorundertheknowntruecondition.Thebasicideaistocorrectmeasurementerrorsbyalinearregressionmodeledbystandardsampleandactualmeasurement.

信噪比：通常是根據(jù)標(biāo)準(zhǔn)樣品對(duì)實(shí)際測(cè)量結(jié)果的誤差進(jìn)行校正，然后求出評(píng)價(jià)統(tǒng)計(jì)量SNR

Example例14-2試驗(yàn)?zāi)撤N新型抗血小板聚集藥物，選三家醫(yī)院作為試驗(yàn)中心，為考核各醫(yī)院血小板凝集試驗(yàn)測(cè)定方法的準(zhǔn)確性和穩(wěn)定性，取出9個(gè)標(biāo)準(zhǔn)血清試樣，分別測(cè)得如下數(shù)據(jù)（表14-4），試對(duì)測(cè)試結(jié)果進(jìn)行評(píng)價(jià)。Example:Table14-4theresultsofexperimentaltestsofplateletaggregationofthreehospitals.

Regressionequation:以各醫(yī)院的測(cè)量值為應(yīng)變量Yi(i=1,2,3)，以標(biāo)準(zhǔn)含量為自變量X作直線(xiàn)回歸分析SNRr為有效重復(fù)數(shù)

r:EffectivenumberofrepeatSNRSNRResultsof

comprehensiveanalysisofthreehospitals:SNR3.Evaluationofconsistency觀察結(jié)果的一致性評(píng)價(jià)consistency:Agreementorlogicalcoherenceamongthingsorparts.是指對(duì)同一個(gè)體用兩種儀器或從兩種角度進(jìn)行觀測(cè)，觀測(cè)值之間接近程度或測(cè)量效果相似性的一個(gè)指標(biāo).Kappakappacoefficient

評(píng)價(jià)分類(lèi)變量結(jié)果一致性和信度的重要指標(biāo)|κ|≤1KappaandKendallThekappastatisticisameasureofabsoluteagreementamongappraisers'ratings.Usekappawhenyourdataarenominal(category)withtwoormorelevelswithnonaturalordering.Kendall'sstatisticisameasureoftheassociationamongappraisers'ratings.OnlyuseKendall'sstatisticwhenyourdataarecontinuousorordinalwiththreeormorepossiblelevelswithnaturalordering,suchasstronglydisagree,disagree,neutral,agree,andstronglyagreeKappaistheproportionofagreementafterchanceagreementhasbeenremoved.Ifkappa=1,thereisperfectagreement.（完全一致）Ifkappa=0,theagreementisthesameaswouldbeexpectedbychance.(<0.4)（一致性較差）Thestrongertheagreement,thehigherthevalueofkappa.(>0.8)（極好的一致性）Negativevaluesoccurwhenagreementisweakerthanexpectedbychance,butthisrarelyhappens.(觀察一致性小于機(jī)遇一致性，無(wú)意義）PA:observedagreementPe:agreementofchancePA–Pe:actualagreementbeyondchance

1–Pe

:potentialagreementbeyondchance

Kapparange:[-1,+1]

IfKappa=+1therearecompleteagreementbetweentwocriteria.

IfKappa=–1twocriteriaaretotallydifferent.

IfKappa=0theagreementiscausedbychance.

IfKappa>0.8theagreementisverysatisfy.IfKappabetween0.6-0.8higheragreement

IfKappabetween0.4-0.6moderateagreement.

IfKappa<=0.4

theagreementisnotsosatisfy.Theconsistencytestfordichotomous:bExample:Supposethatyouwereanalyzingdatarelatedtopeopleapplyingforagrant.Eachgrantproposalwasreadbytwopeopleandeachreadereithersaid"Yes"or"No"totheproposal.Supposethedatawereasfollows,whererowsarereaderAandcolumnsarereaderB:Notethattherewere20proposalsthatweregrantedbybothreaderAandreaderB,and15proposalsthatwererejectedbybothreaders.Thus,theobservedpercentageagreementisPr(a)=(20+15)/50=0.70BYesNoAYes205No1015

TocalculatePr(e)(theprobabilityofrandomagreement)wenotethat:AsaidYesto25applicantsandNoto25applicants,theAsaidYesis50%ofthetimeBsaidyesto30applicantsandNoto15applicants,BsaidYesis60%ofthetimeThereforetheprobabilitythatbothofthemwouldsay"Yes"randomlyis0.5*0.6=0.3andtheprobabilitythatbothofthemwouldsay"No"is0.5*0.4=0.2,ThustheoverallprobabilityofrandomagreementisPr(e)=0.3+0.2=0.5SonowapplyingourformulaforCohen'sKappaweget:BBYesNoAYes205ANo1015consistencytestofmulti-classificationmeasurementresults:consistencytestofmulti-classificationmeasurementresults:ObservedagreementAgreementofchanceUsingCiwujiainjectiontreat418patientsofcoronaryheartdiseasewithangina,DoctorAanddoctorBevaluatetheeffectofthetreatment.Howistheagreementofevaluationofthetwodoctors?DoctorADoctorBtotalmarkedeffecteffectnoeffectmarkedeffect10540109effect2422020264noeffect063945total12923059418DiagnosisTest:Adiagnostictestisanykindofmedicaltestperformedtoaidinthediagnosisordetectionofdisease.Forexample:※todiagnosediseases,andpreferablysub-classifyitregarding,forexample,severityandtreatability.※toconfirmthatapersonisfreefromdisease.

StatisticalassessmentofDiagnosisTestDiagnostictestshelpphysiciansrevisediseaseprobabilityfortheirpatients.Alltestsshouldbeorderedbythephysiciantoansweraspecificquestion.The5mainreasonsforadiagnostictestareasfollows:◎Establishadiagnosisinsymptomaticpatients.Forexample,anECGtodiagnoseST-elevationmyocardialinfarction(STEMI)inpatientswithchestpain.◎Screenfordiseaseinasymptomaticpatients.Forexample,aprostate-specificantigen(PSA)testinmenolderthan50years.◎Provideprognosticinformationinpatientswithestablisheddisease.Forexample,aCD4countinpatientswithHIV.◎Monitortherapybyeitherbenefitsorsideeffects.Forexample,measuringtheinternationalnormalizedratio(INR國(guó)際標(biāo)準(zhǔn)化比值)inpatientstakingwarfarin(華發(fā)林).INR的值越高，血液凝固所需的時(shí)間越長(zhǎng)。這樣可以防止血栓形成，例如血栓導(dǎo)致的中風(fēng)。健康成年人，INR值大約1.0。有靜脈血栓的患者的INR值一般應(yīng)保持在2.0～2.5之間；有心房纖維性顫動(dòng)的患者的INR值一般應(yīng)保持在2.0～3.0之間?！駻testmaybeperformedtoconfirmthatapersonisfreefromadisease.Forexample,apregnancytesttoexcludethediagnosisof.Thecriterion(reference)standardtestdefinitivelydecideseitherpresenceorabsenceofadisease.Examplesofcriterionstandardtestsincludepathologicalspecimensformalignanciesandpulmonaryangiographyforpulmonaryembolism.However,criterionstandardtestsroutinelycomewithdrawbacks：theyareusuallyexpensive,lesswidelyavailable,moreinvasive,andriskier.Theseissuesusuallycompelmostphysicianstochooseotherdiagnostictestsassurrogatesfortheircriterionstandardtest.Goldstandardmethodofdiagnosiswhichisthemostreliable,mostaccurateandthebestincurrentclinicaldiagnosisofdisease.(Biopsy,Autopsy,Surgicalexploration)

ThetargetpopulationpatientNon-patient＋＋－－GoldstandardDiagnosisTestNumberofpeopleNon-patientD-PatientD+αβThresholdfalsepositivefalsenegativeTruepositive:

Sickpeoplecorrectlydiagnosedassick.Falsepositive:Healthypeopleincorrectlyidentifiedassick.Truenegative:

Healthypeoplecorrectlyidentifiedashealthy.Falsenegative:Sickpeopleincorrectlyidentifiedashealthy.

DiagnostictestresultsGoldstandardDiagnosistesttotalT+D-D+D-totalaba+b(n1)cdc+d(n2)a+cb+da+b+c+d(n)Se=an1Sp=n2dPV+=aa+cdPV-=b+dSensitivity(Se)measurestheproportionofactualpositiveswhicharecorrectlyidentifiedassuch.(e.g.thepercentageofsickpeoplewhoarecorrectlyidentifiedashavingthecondition)1-Se=β:1-Sedenotesthefalsenegativerate.measurestheproportionofnegativeswhicharecorrectlyidentified.(e.g.thepercentageofhealthypeoplewhoarecorrectlyidentifiedasnothavingthecondition)1-Sp=α:1-Spdenotesthefalsepositiverate.Specificity(Sp)ThesetwomeasuresarecloselyrelatedtotheconceptsoftypeIandtypeIIerrors.Atheoretical,optimalpredictionaimstoachieve100%sensitivity(i.e.predictallpeoplefromthesickgroupassick)and100%specificity(i.e.notpredictanyonefromthehealthygroupassick),howevertheoreticallyanypredictorwillpossessaminimumerrorboundknownastheBayeserrorrate.

Youden’sindex(J)Youden’sindexisonewaytoattemptsummarizingtestaccuracyintoasinglenumericvalue.J=Se+Sp-1(-1≤J≤1)

J1J≤0Positivepredictivevalue(PV+)Positivepredictivevalueistheproportionofpatientswithpositivetestresultswhoarecorrectlydiagnosed.negativepredictivevalue(PV-)Positivepredictivevalueistheproportionofpatientswithpositivetestresultswhoarecorrectlydiagnosed.Thepredictivevalueareassociatedwithnotonlysensitivityandspecificity,butalsotheprevalenceoftargetpopulation.predictivevaluePV+=P(D+∣T+)=SePSeP+(1-Sp)(1-P)PV-=P(D-∣T-)=Sp(1-P)Sp(1-P)+(1-Se)PThevalueofadiagnosticmethodmainlybasedonthePV+andPV-.predictivevalue1.0PVPPV-PV+Se-,Sp-;P↑→→PV+↑,PV-↓0.80.20.40.60PV+=aa+cPV-=b+dd診斷人群患病比率Figure:診斷人群患病比率與診斷結(jié)果預(yù)測(cè)值的關(guān)系Example:Electricpulptesttodetectthenecrosisofpulptissue,howaboutthesensitivity,specificity,Youdenindex,PV+andPV-？Pathologicalexamination(goldstandard)ElectricpulptestTotalT+T-D+451055D-16180196Total61190251DiagnosisindexResultsestimateStandarddeviation95%CISensitivity0.81820.05200.71630.9201Specificity0.91840.01960.88010.9567YoudenIndex0.73660.05560.62770.8455PV+0.7377PV-0.9474Sensitivityandspecificityarethebasicmeasuresofaccuracyofadiagnostictest;however,theydependonthecutpointusedtodefine“positive”and“negative”testresults.Asthecutpointshifts,sensitivityandspecificityshift.ROCCurve(receiveroperatingcharacteristiccurve)

ROCCurveRecei