A Google AI model is less effective at predicting acute kidney injury in women than in men.
Google's kidney damage AI yields inferior results in women
December 12, 2022
by John R. Fischer, Senior Reporter
A Google AI model trained with data predominantly from male veterans to predict acute kidney injury is overestimating the risk for women in certain circumstances and was less accurate in predicting the condition in them overall, according to researchers at University of Michigan.
Unveiled in 2019, the model was designed by Google business DeepMind Health. In an initial study, it accurately predicted nine out of ten VA patients would develop the condition and need dialysis, two days in advance, reported Stat.
A common killer of hospitalized patients, acute kidney injury is difficult to recognize and is responsible for 1.7 million annual deaths worldwide.
“If we have this problem, then half the population won’t benefit,” Jie Cao, a Ph.D. student at the University of Michigan and the lead author of the Michigan paper, told Stat.
In its study, Google included 6.38% of women, with detection lower in those with earlier stages of acute kidney failure. In Michigan, it was lower across all levels. In stage three patients, it was accurate in 84% of men, versus 71% of women.
Once trained with data from more women, it performed better in Michigan but worse with the VA, being successful with 83% of men and around 69% of women. This may be due to different treatment practices among VA sites, indicating a need for more uniformity, according to the researchers.
The VA planned to assess its performance in a clinical trial in its Palo Alto health system, but the status of that initiative is unclear.
Google researcher Alan Karthikesalingam told STAT that the AI prototypes need to be refined, but said the researchers assessed the lower-performing of two models designed by his team. The researchers chose this model because of limited computing capacity in the VA’s system.
Google has published protocols for implementing the better-performing model, along with open source code. Good Cloud is also making a pilot-ready version available to providers worldwide.
“We hope this enables other clinical researchers to build upon our promising initial findings and take further research steps to improve performance in representative populations in multiple clinical environments,” wrote Google.
In a statement , the VA said it is studying “various approaches before making a determination on the different models’ efficacy and/or suitability for any specific uses.”
The Michigan study was published in Nature.
Unveiled in 2019, the model was designed by Google business DeepMind Health. In an initial study, it accurately predicted nine out of ten VA patients would develop the condition and need dialysis, two days in advance, reported Stat.
A common killer of hospitalized patients, acute kidney injury is difficult to recognize and is responsible for 1.7 million annual deaths worldwide.
“If we have this problem, then half the population won’t benefit,” Jie Cao, a Ph.D. student at the University of Michigan and the lead author of the Michigan paper, told Stat.
In its study, Google included 6.38% of women, with detection lower in those with earlier stages of acute kidney failure. In Michigan, it was lower across all levels. In stage three patients, it was accurate in 84% of men, versus 71% of women.
Once trained with data from more women, it performed better in Michigan but worse with the VA, being successful with 83% of men and around 69% of women. This may be due to different treatment practices among VA sites, indicating a need for more uniformity, according to the researchers.
The VA planned to assess its performance in a clinical trial in its Palo Alto health system, but the status of that initiative is unclear.
Google researcher Alan Karthikesalingam told STAT that the AI prototypes need to be refined, but said the researchers assessed the lower-performing of two models designed by his team. The researchers chose this model because of limited computing capacity in the VA’s system.
Google has published protocols for implementing the better-performing model, along with open source code. Good Cloud is also making a pilot-ready version available to providers worldwide.
“We hope this enables other clinical researchers to build upon our promising initial findings and take further research steps to improve performance in representative populations in multiple clinical environments,” wrote Google.
In a statement , the VA said it is studying “various approaches before making a determination on the different models’ efficacy and/or suitability for any specific uses.”
The Michigan study was published in Nature.