Validation of new york operative mortality risk score for valve and valve/coronary artery bypass grafting operations.

Background: New York (NY) valve and valve/coronary artery bypass grafting (CABG) mortality risk models, developed from operations performed in 2007 to 2009, have just been published. These models were validated using NY data from 2004 to 2006. The authors stated that their models "should also be validated by testing them against non-New York populations." Thus, we validated the NY models with the Providence Health & Services-Swedish Health Services (PH&S-SHS) cardiac surgical data and also compared them with The Society of Thoracic Surgeons (STS) mortality risk models.

Methods: The PH&S-SHS validation data set contained 4,021 isolated valve and 2,406 valve/CABG operations, performed from 2008 to 2012. The risk models (NY logistic and score models and the STS models) were recalibrated to equalize the expected and observed number of deaths. Discrimination was tested by C statistics and calibration by Hosmer-Lemeshow statistics.

Results: PH&S-SHS operative mortality rates were 2.6% and 5.5% in the valve and valve/CABG operations, respectively, and were lower than the NY rates. The C statistics for the NY logistic valve and valve/CABG models were 0.777 and 0.727, respectively, and were very similar for the NY score models. Calibration was good for the NY valve model (p=0.85), but not for the NY valve/CABG model (p=0.01). The STS models had better discrimination than NY models and good calibration.

Conclusions: The NY logistic and score models for valve operations fit the PH&S-SHS data well with acceptable discrimination and good calibration. The NY models for valve/CABG operations fit the PH&S-SHS data with acceptable discrimination and poor calibration. STS logistic regression models fit the PH&S-SHS data somewhat better.