Analysis and evaluation of the evidence of diagnostic test Clinical Trail Study CenterCao SumeiDiagnostic test are not just about diagnosisl Screeningl Determining severityl Optimally therapyl Prognosisl Monitor Examplel Carotid ultrasound can tell you the severity of the patients carotid stenosisl Carotid ultrasound can tell you the patients prognosis for stroke and deathl Carotid ultrasound can predict your patients likely responsiveness to therapyBasic principles of conducting diagnostic studiesl Apply the gold standard to determine whether or not the target condition is present Gold standard: The most recognized standard for clinician to diagnose the target conditionlPathological measurementlOperation findinglSpecial imaging detectionlLong-term follow-up lRecognized standardWhat if your test is more gold than the standardl May lead to underestimate of the diagnostic power of the evaluatingl One strategy for dealing with this problem is to use long-term follow-up as a gold standardTo Whom Should the Gold Standard Be Applied?to everyoneselective performing the gold standard on patients may result in “ verification bias ” or “workup bias”Recruit your participantsl Recruit the target-negative and target-positive participants identified by gold standard l characteristic of those to whom you will want to apply the test in clinical practicel Including a broad spectrum of the diseasedcase：from mildly to severelycontrol：a broad spectrum of competing conditionsl An alternative approach is that recruiting a consecutive sample of patientsMeasurement proceduresl Specifying test techniquel Reproducibilityl Blinding of the individual conducting or interpreting the test to the gold standardSelect statistical procedurelCalculating sample size Example: Assuming a sensitivity of 80%, specificity of 60% of ultrasonography for diagnosis of cholecystolithiasis. How many samples are needed ? Result evaluation indexl Example: 126 patients underwent independent, blind BNP measurement and echocardiography for diagnosis of LVD.l sensitivity：a/(a+c)=35/40=0.88, or 88%l specificity：d/(b+d)=29/86=0.34, or 34%l positive predictive value (PPV)：a/(a+b)=35/92=0.38, or 38%l negative predictive value (NPV)：d/(c+d)=29/34=0.85, or 85%l prevalence: (a+c) / (a+b+c+d)=40/126=0.32, or 32%Multilevel likelihood ratiosStability of the indexlStable : Sen, SpelRelatively stable: LR+, LR- lUnstable : PPV, NPV, prevalence:Receiver operating characteristic curves(ROC)l It illustrates the performance of a diagnostic test when you select different cut-points to distinguish “normal” from “abnormal” l It demonstrates the fact that any increase in sensitivity will be accompanied by a decrease in specificityl The closer the curve gets to the upper left corner of the display, the more the overall accuracy of the testl The closer the curve comes to the 45-degree diagonal of the ROC space ,the less accurate the testl The area under the curve provides an overall measure of a tests accuracy Fig A ROC for BNP as a diagnostic test for LVD Parallel test l A test B test Resultl + + +l + + l + +l -lReduction miss diagnosislExclude some disease lWhen prevalence is low, as the primary screening methodSerial testl A test B test Resultl + + +l + - l + -l -Sen = Sen A SenBSpe = Spe A + (1-Spe A) Spe Bl Misdiagnosis may cause nuisance effect l Confirmatory diagnosis Serial test with enzyme labeled compound assay for diagnosis of myocardial infarctionEnzyme labeled compound assaySenSpeCPK969657SGOT9174LDH87919191Multivariate analysislSEN lSPE single variable analysismarkermethodsSEN (%)spe (%)cutoffAREAaELISA90.788.80.19150.926bELISA77.373.20.20350.802cELISA74.270.90.09050.762dELISA78.481.61.080.836eELISA90.784.40.3560.932fELISA84.581.60.7990.899multivariate analysis using logistic regression Combined markersSEN（）SPE（）AREAa and b88.882.50.926a and c87.782.50.927a and e91.690.070.974a and f95.590.070.967a and d87.285.6.0936b and c78.876.30.837b and d87.786.60.934b and e83.882.50.900b and f82.781.40.863C and d87.785.60.946C and e88.385.60.926C and f81.679.40.854d and e89.486.60.946d and f88.386.60.952e and f87.785.60.933Prediction the probability of a diseasel Logit(P)= -0.934+4.797 x a +2.203 x eAvoiding overfittingl Overfitting occurs when a computer model identifies a “chance” pattern that discriminates cancer patients from non-cancer patients, perfectly fitting that dataset but not reproducible in other data sets.l One way to avoiding overfitting is to randomly split the data into separate training and test samples.The EBM steps for diagnostic testsl Looking for the most suitable study papers according to the clinical question Bring forward the question in cliniclExample 2 : if detection of serum forritin can diagnose Iron deficiency anemia? Search the computer information using the apposite key wordl“diagnose Iron deficiency anemia” and “diagnostic test” and “human”Evaluation of the scientificity of the papersl If compared with the gold standard independently and blindly