Histopathological diagnosis of cutaneous melanocytic lesions: blinded and nonblinded second opinions offer similar improvement in diagnostic accuracy.

Background: Previous studies of second opinions in the diagnosis of melanocytic skin lesions have examined blinded second opinions, which do not reflect usual clinical practice. The current study, conducted in the USA, investigated both blinded and nonblinded second opinions for their impact on diagnostic accuracy.

Methods: In total, 100 melanocytic skin biopsy cases, ranging from benign to invasive melanoma, were interpreted by 74 dermatopathologists.

Combining biomarkers by maximizing the true positive rate for a fixed false positive rate.

Biomarkers abound in many areas of clinical research, and often investigators are interested in combining them for diagnosis, prognosis, or screening. In many applications, the true positive rate (TPR) for a biomarker combination at a prespecified, clinically acceptable false positive rate (FPR) is the most relevant measure of predictive capacity. We propose a distribution-free method for constructing biomarker combinations by maximizing the TPR while constraining the FPR.

Adding Rigor to Biomarker Evaluations-EDRN Experience.

The cancer early-detection biomarker field was, compared with the therapeutic arena, in its infancy when the Early Detection Research Network (EDRN) was initiated in 2000. The EDRN has played a crucial role in changing the culture and the ways people conduct biomarker studies. The EDRN proposed biomarker developmental guidelines and biomarker pivotal trial study design standards, created biomarker reference sets and functioned as an unbiased broker for the field, implemented the most rigorous blinding policy in the biomarker field, developed an array of statistical and computational tools for early-detection biomarker evaluations, and developed a multidisciplinary team-science approach.

Statistical inference for net benefit measures in biomarker validation studies.

Referral strategies based on risk scores and medical tests are commonly proposed. Direct assessment of their clinical utility requires implementing the strategy and is not possible in the early phases of biomarker research. Prior to late-phase studies, net benefit measures can be used to assess the potential clinical impact of a proposed strategy.

Assessment of Second-Opinion Strategies for Diagnoses of Cutaneous Melanocytic Lesions.

Importance: Histopathologic criteria have limited diagnostic reliability for a range of cutaneous melanocytic lesions.

Objective: To evaluate the association of second-opinion strategies by general pathologists and dermatopathologists with the overall reliability of diagnosis of difficult melanocytic lesions.

Design, Setting, And Participants: This diagnostic study used samples from the Melanoma Pathology Study, which comprises 240 melanocytic lesion samples selected from a dermatopathology laboratory in Bellevue, Washington, and represents the full spectrum of lesions from common nevi to invasive melanoma.

Concordance and Reproducibility of Melanoma Staging According to the 7th vs 8th Edition of the .

Importance: The recently updated American Joint Committee on Cancer (AJCC) classification of cancer staging, the , 8th edition (), includes revisions to definitions of T1a vs T1b or greater. The Melanoma Pathology Study database affords a comparison,of pathologists' concordance and reproducibility in the microstaging of melanoma according to both the existing 7th edition ) and the new .

Objective: To compare and to examine whether changes to the definitions of T1a and T1b or greater are associated with changes in concordance and reproducibility.

Recommendation to use exact P-values in biomarker discovery research in place of approximate P-values.

Background: Biomarker candidates are often ranked using P-values. Standard P-value calculations use normal or logit-normal approximations, which may not be correct for small P-values and small sample sizes common in discovery research.

Methods: We compared exact P-values, correct by definition, with logit-normal approximations in a simulated study of 40 cases and 160 controls.

Pathologist characteristics associated with accuracy and reproducibility of melanocytic skin lesion interpretation.

Background: Diagnostic interpretations of melanocytic skin lesions vary widely among pathologists, yet the underlying reasons remain unclear.

Objective: Identify pathologist characteristics associated with rates of accuracy and reproducibility.

Methods: Pathologists independently interpreted the same set of biopsy specimens from melanocytic lesions on 2 occasions.

Pathologists' diagnosis of invasive melanoma and melanocytic proliferations: observer accuracy and reproducibility study.

 To quantify the accuracy and reproducibility of pathologists' diagnoses of melanocytic skin lesions. Observer accuracy and reproducibility study. 10 US states.

The diagnostic challenge of low-grade ductal carcinoma in situ.

Background: Diagnostic agreement among pathologists is 84% for ductal carcinoma in situ (DCIS). Studies of interpretive variation according to grade are limited.

Methods: A national sample of 115 pathologists interpreted 240 breast pathology test set cases in the Breast Pathology Study and their interpretations were compared to expert consensus interpretations.

A Randomized Study Comparing Digital Imaging to Traditional Glass Slide Microscopy for Breast Biopsy and Cancer Diagnosis.

Background: Digital whole slide imaging may be useful for obtaining second opinions and is used in many countries. However, the U.S.

Diagnostic Reproducibility: What Happens When the Same Pathologist Interprets the Same Breast Biopsy Specimen at Two Points in Time?

Background: Surgeons may receive a different diagnosis when a breast biopsy is interpreted by a second pathologist. The extent to which diagnostic agreement by the same pathologist varies at two time points is unknown.

Methods: Pathologists from eight U.

Evaluation of 12 strategies for obtaining second opinions to improve interpretation of breast histopathology: simulation study.

Objective:  To evaluate the potential effect of second opinions on improving the accuracy of diagnostic interpretation of breast histopathology.

Design:  Simulation study.

Setting:  12 different strategies for acquiring independent second opinions.

Early-Phase Studies of Biomarkers: What Target Sensitivity and Specificity Values Might Confer Clinical Utility?

Background: Many cancer biomarker research studies seek to develop markers that can accurately detect or predict future onset of disease. To design and evaluate these studies, one must specify the levels of accuracy sought. However, justified target levels are rarely available.

Variability in Pathologists' Interpretations of Individual Breast Biopsy Slides: A Population Perspective.

Background: The effect of physician diagnostic variability on accuracy at a population level depends on the prevalence of diagnoses.

Objective: To estimate how diagnostic variability affects accuracy from the perspective of a U.S.

The Net Reclassification Index (NRI): a Misleading Measure of Prediction Improvement Even with Independent Test Data Sets.

The Net Reclassification Index (NRI) is a very popular measure for evaluating the improvement in prediction performance gained by adding a marker to a set of baseline predictors. However, the statistical properties of this novel measure have not been explored in depth. We demonstrate the alarming result that the NRI statistic calculated on a large test dataset using risk models derived from a training set is likely to be positive even when the new marker has no predictive information.

Designing a study to evaluate the benefit of a biomarker for selecting patient treatment.

Biomarkers that predict the efficacy of treatment can potentially improve clinical outcomes and decrease medical costs by allowing treatment to be provided only to those most likely to benefit. We consider the design of a randomized clinical trial in which one objective is to evaluate a treatment selection marker. The marker may be measured prospectively or retrospectively using samples collected at baseline.

The Fundamental Difficulty With Evaluating the Accuracy of Biomarkers for Guiding Treatment.

Developing biomarkers that can predict whether patients are likely to benefit from an intervention is a pressing objective in many areas of medicine. Recent guidance documents have recommended that the accuracy of predictive biomarkers, ie, sensitivity, specificity, and positive and negative predictive values, should be assessed. We clarify the meanings of these entities for predictive markers and demonstrate that generally they cannot be estimated from data without making strong untestable assumptions.

Improving the quality of biomarker discovery research: the right samples and enough of them.

Background: Biomarker discovery research has yielded few biomarkers that validate for clinical use. A contributing factor may be poor study designs.

Methods: The goal in discovery research is to identify a subset of potentially useful markers from a large set of candidates assayed on case and control samples.

Diagnostic concordance among pathologists interpreting breast biopsy specimens.

Importance: A breast pathology diagnosis provides the basis for clinical treatment and management decisions; however, its accuracy is inadequately understood.

Objectives: To quantify the magnitude of diagnostic disagreement among pathologists compared with a consensus panel reference diagnosis and to evaluate associated patient and pathologist characteristics.

Design, Setting, And Participants: Study of pathologists who interpret breast biopsies in clinical practices in 8 US states.

Construction and analysis of the NCI-EDRN breast cancer reference set for circulating markers of disease.

Background: Many circulating biomarkers have been reported for the diagnosis of breast cancer, but few, if any, have undergone rigorous credentialing using prospective cohorts and blinded evaluation.

Methods: The NCI Early Detection Research Network (EDRN) has created a prospective, multicenter collection of plasma and serum samples from 832 subjects designed to evaluate circulating biomarkers for the detection and diagnosis of breast cancer. These samples are available to investigators who wish to evaluate their biomarkers using a set of blinded samples.

An approach to evaluating and comparing biomarkers for patient treatment selection.

Despite the heightened interest in developing biomarkers predicting treatment response that are used to optimize patient treatment decisions, there has been relatively little development of statistical methodology to evaluate these markers. There is currently no unified statistical framework for marker evaluation. This paper proposes a suite of descriptive and inferential methods designed to evaluate individual markers and to compare candidate markers.

Net risk reclassification p values: valid or misleading?

Background: The Net Reclassification Index (NRI) and its P value are used to make conclusions about improvements in prediction performance gained by adding a set of biomarkers to an existing risk prediction model. Although proposed only 5 years ago, the NRI has gained enormous traction in the risk prediction literature. Concerns have recently been raised about the statistical validity of the NRI.