IBM Watson for Oncology has had diagnosis problems, according to a report from STAT News.

Internal documents – slide decks – according to the news site, showed “multiple examples of unsafe and incorrect treatment recommendations” from the software.

The documents were part of June and July 2017 presentations by IBM Watson’s ex-deputy health chief Andrew Norden, and were said to have been distributed to IBM Watson Health management. Norden has since left the company, and had no comment when asked by tech news site Gizmodo about the STAT story.

The critique of the system was strong, according to Gizmodo, including advising that the “often inaccurate” medical advice from the software raises “serious questions about the process for building content and the underlying technology.”

In one case, Watson advised that a man, 65, with lung cancer and severe bleeding should get both chemotherapy and Bevacizumab, or Avastin. That drug can itself lead to severe or fatal hemorrhage, according to its literature – and should not be taken when there is severe bleeding.

A Memorial Sloan Kettering (MSK) Cancer Center spokesperson said they believed that this advice was part of system testing – and not a recommendation for a real patient, according to STAT.

The fault, according to the newly revealed documents, lies with the training the system got from IBM engineers and the doctors at MSK, who had been working with the system in 2012 to make it more doctor-like in its “thinking”.

At that time the doctors employed hypothetical – or “synthetic” – patient data to train Watson.

The reaction of doctors in the field reported in the documents seems to have been less than completely positive, according to Gizmodo.

“This product is a piece of sh--,” a doctor at Florida’s Jupiter Hospital said to IBM, according to the documents. “We bought it for marketing and with hopes that you would achieve the vision. We can’t use it for most cases.”

Watson for Oncology is in use at 230 hospitals globally and has “supported care for more than 84,000 patients,” an IBM spokesperson told Gizmodo in response to the STAT report.

At the same time, “we have learned, and improved Watson Health based on continuous feedback from clients, new scientific evidence, and new cancers and treatment alternatives,” the spokesperson advised, adding that “this includes 11 software releases for even better functionality during the past year, including national guidelines for cancers ranging from colon to liver cancer.”

Rather than looking at the criticisms in these documents as a sign of major trouble, Memorial Sloan Kettering spokesperson Caitlin Hool told STAT, that they are evidence of “the robust nature of the process” of development.

She underscored to the news site that “while Watson for Oncology provides safe treatment options, treatment decisions ultimately require the involvement and clinical judgement of the treating physician,” stressing that “no technology can replace a doctor and his or her knowledge about their individual patient.”

A STAT report on Watson for Oncology from September, 2017, found that “the supercomputer isn’t living up to the lofty expectations IBM created for it.”

It noted at the time that rather than Watson figuring out how to treat cancer automatically from its access to data, a “couple of dozen physicians at a single, though highly respected, U.S. hospital, Memorial Sloan Kettering Cancer Center in New York” are inputting their own advice into the system to help to guide its decision-making.

The report also noted that IBM “in its rush to bolster flagging revenue, unleashed a product without fully assessing the challenges of deploying it in hospitals globally. While it has emphatically marketed Watson for cancer care, IBM hasn’t published any scientific papers demonstrating how the technology affects physicians and patients.”