From the November 2019 issue of HealthCare Business News magazine
3. Clinical validation
Once developed, AI algorithms should ideally undergo clinical studies to test their robustness, accuracy and reproducibility.
Questions the radiologist should ask:
• What type of clinical study was used for the validation? Prospective studies using both case and non-case data outweigh the significance of retrospective studies.
• Did the clinical study involve single or multiple readers?
• Was it a single-center or multicenter clinical study?
• How does the performance of the AI algorithm compare to existing clinical practice? The AI solution should be equal to, or better than the current standard of practice.
4. Product regulation
Only solutions with regulatory clearance may be used in clinical practice. At present, 57 vendors have received regulatory approval for 77 machine learning algorithms for medical imaging from one of the four major regulatory bodies: US FDA (USA), CE Mark (Europe), PMDA (Japan), and MFDS (South Korea).
Questions the radiologist should ask:
• Does the vendor have regulatory approval from the US FDA? If so, what type of clearance has it received (i.e., 510(K), PMA, or de Novo)?
• Has the algorithm been cleared in-full, or only in-part? If in-part, which part is approved and why? Has this been clearly labelled for the end-user?
• Is the solution regulated in any other markets? For example, in neighboring countries or other major markets (e.g., CE Mark)?
• Does the vendor have further regulatory applications in the pipeline?
5. Workflow integration
The results from AI solutions need to integrate seamlessly into the radiologist’s preferred diagnostic viewer. Radiologists should be able to view, and potentially interact (edit / accept / reject), with the findings without compromising on productivity.
Questions the radiologist should ask:
• Are the AI results integrated into the PACS interface, or does the algorithm require a separate user interface to view the results?
• Can the AI results be edited, accepted or rejected based on the radiologist’s clinical judgment?
• How are the results displayed? Are they presented as an overlay on the image or as a separate report?
• Is the AI solution PACS/modality vendor-agnostic?
6. Return on investment
Related to clinical relevancy, radiologists and healthcare providers expect to see a return on investment (financial and non-financial reasons) for AI solutions.
Questions the radiologist should ask:
• Is the AI solution a productivity tool, or a solution focused on quality improvement? Healthcare providers with a shortage of radiologists may value AI algorithms that increase productivity, but other providers may find value in algorithms that improve the quality of diagnoses and treatment decisions.
• Does the AI solution alter the diagnostic pathway, providing value to both the patient and healthcare provider? For example, reduce the need for invasive procedures, and in turn, the risk to the patient.
• Does the AI solution alter the treatment pathway, resulting in cost savings for the healthcare provider?
