Validating the Framingham Offspring Study equations for predicting incident diabetes mellitus.

Background: Investigators from the Framingham Offspring Study (FOS) recently proposed a new simple point score for estimating 8-year diabetes mellitus (DM) risk.

Objectives: To validate prediction models and to compare DM risk estimated by the point score with observed DM incidence.

Study Design: Longitudinal observational cohort study.

Methods: We identified 20,644 members of Kaiser Permanente Northwest (KPNW) who had no prior evidence of DM and who had all data elements necessary to estimate the models recorded between July 1, 1999, and June 30, 2000. Patients were followed up through June 30, 2007. We reestimated the FOS logistic regression models in the total KPNW sample (cumulative DM incidence, 16.5%) and in a randomly selected subsample with incidence identical to that in the FOS (5.1%). We also compared DM risk predicted by the FOS point score with actual 8-year DM incidence observed in the KPNW samples.

Results: The prediction models performed similarly in the FOS and KPNW samples, with almost identical odds ratios for independent variables and areas under the receiver operating characteristic curves for the models. The FOS point score substantially underestimated actual DM incidence in the total sample. In the subsample with DM incidence identical to that in the FOS sample, the point score was extremely accurate.

Conclusions: The accuracy of the point score requires that the underlying incidence of the individuals to whom it is applied be similar to the population from which it was derived. Accurate adaptation requires recalculating DM incidence at each point level.