In a separate model, the research team collected NSQIP data on 9,182 patients who underwent general surgery between January 2017 and September 2022 at the medical center. They predicted patient outcomes by analyzing clinical data captured by the NSQIP Risk Calculator.
After comparing the two models, a third model was built, combining both preoperative intuition and the NSQIP Risk Calculator, to determine if this third model might outperform the other two models.
Ad Statistics
Times Displayed: 21862
Times Visited: 433 Stay up to date with the latest training to fix, troubleshoot, and maintain your critical care devices. GE HealthCare offers multiple training formats to empower teams and expand knowledge, saving you time and money
Key Findings
Nearly half of surgeons who responded to the survey (45.4%) indicated that their patient’s risk of any complication was average, with 40.3% responding higher than average risk and 14.4% responding lower than average risk.
Preoperative surgeon intuition was an independent predictor of postoperative complications. A model of preoperative surgeon intuition predicting complications had an area under the curve (AUC) of 0.70, where an AUC of 1.0 is perfect prediction and 0.5 is a random result.
Surgeon intuition in predicting any complication was less accurate than the ACS NSQIP risk calculator, which had an AUC of 0.83.
A combined model using both surgeon intuition and the NSQIP risk calculator did not do better, with an AUC of 0.83, than the NSQIP risk calculator alone.
A subset analysis showed that the intuition of more experienced attending surgeons in predicting the outcomes of patients was more accurate than less experienced residents.
“The value of surgical intuition for preoperative prediction was not improved by including human intuition in the model and this suggests that, at least for most presurgical prediction, the information that is gathered by the NSQIP Risk Calculator is better at predicting those outcomes than the gut feeling that surgeons have when looking at patients,” Dr. Brat said.
“Human intuition takes into account a lot of information that is not available to the calculator, but it doesn’t weight it in any explicit way. We don’t have a weighting system in our head that says, ‘We know that this piece of information is more important than another by a certain amount,’ whereas the NSQIP Risk Calculator does have that explicit weighting system. So, in certain situations, it’s the case that an explicit algorithm is going to be better at prediction. The value of the clinician is to integrate information that’s not available to the risk calculator.”
“Data from the NSQIP Risk Calculator are amongst the best for predicting outcomes – demonstrably superior to claims, billing, and administrative data. However, what is done with the data is the next important step,” said Clifford Y. Ko, MD, MS, MSHS, FACS, FASCRS, Director, ACS Division of Research and Optimal Patient Care, who wasn’t involved with the study. “It will be the surgeon and patient’s responsibility together, based on the data, to decide whether to proceed with surgery, and how best to prepare for the phases of care before, during, and after surgery.”
