Pixel perfect – A new approach to annotation software

March 12, 2019
by Sean Ruck, Contributing Editor
At the conclusion of the Society for Imaging Informatics in Medicine’s 2018 conference, Dr. Akshay Goel and his team were awarded the Innovation Challenge grand prize. SIIM picked Goel’s project, “Radlearn.ai – A Cloud-Based Deep Learning Annotation-to-Analysis Platform” from among eight semi-finalists. HealthCare Business News reached out to Dr. Goel to learn about the potential SIIM saw in his project and to find out what the future holds and how the work he’s doing may better-serve imaging efforts in the future.

Goel started programming as a freshman in high school. In college, at Carnegie Mellon University, he was interested in pursuing medical school but maintained his love of computer science. “I took classes in biomedical engineering but decided to major in computer science after my strong natural engagement toward the programming assignments in our Data Structures and Algorithms course,” he said.

After his third year of medical school, Goel did a research year at UT Southwestern, sponsored by the Doris Duke Charitable Foundation, focusing on a tech-heavy project. His project was automatically computing aortic stiffness from cardiac MRIs in the Dallas Heart Study, and his experience meshed the two passions. Having just completed his third year of medical school, the project helped to cement his love of technology and medicine further. Years later, he found himself holding a novelty-sized check for a very real $10,000 in front of a SIIM crowd.

The inspiration for his platform, which he named Radlearn.ai, came from a very basic approach – speaking and observing radiology researchers. “A lot of people I spoke to were annotating radiology data in a way that felt tedious and inefficient. Some were even complaining about the process! At the time I couldn’t find a tool that solved this problem, so that’s how I got interested.”

Goel created a prototype that could manage the entire annotation process much more efficiently. He also continued his original approach of speaking with other researchers about their workflow, to understand their annotation process: from opening the study, finding a tool, setting up parameters for a tool, marking an annotation, and saving the result. “Every step adds up very quickly. People I observed had to complete a tedious 10-step process to annotate a single study. When you multiply that by hundreds of studies, you realize how time-consuming it can be,” he said.

Within five or six months, he had a highly functional prototype of his Radlearn.ai concept, far enough along to win the SIIM Innovation Challenge – a contest he didn’t initially even have in mind. What makes his accomplishments to-date even more impressive is that he developed the program with a small team of three developers in the beginning, and one designer later in the process. “I was fortunate to have recently organized a development team for a tutoring website called Medlearnity. It made the prospect of getting this project off the ground more feasible”, he said.

Radlearn.ai is built on top of a popular open-source package called Cornerstone. Since Cornerstone is used as its… cornerstone, essentially any DICOM data supported by Cornerstone is supported by Radlearn.ai, which means MRI, CT, ultrasound, etc. Radlearn.ai, essentially, helps manipulate pixel data in a browser environment. Radiologists can use the tools to annotate images based on a variety of characteristics. Goel provided a simple example using an aggressive brain tumor – Radlearn.ai allows someone to easily make distinct annotations of the necrotic tumor, and the amorphous edema around the primary tumor. “Capturing amorphous pathology like edema is a weak point in many annotation solutions, as the pathology is not confined by a clean border,” he explained.

Goel provided an explanation in more basic terms, “You could imagine Radlearn.ai is a fancy version of MS Paint for radiology data. Radlearn.ai takes the basic paintbrush tool a few steps further allowing you to precisely control annotation, based on specified characteristics, within the confines of a web browser.”

Ultimately, Goel believes that Radlearn.ai could be effective for any hospital or organization that is interested in creating radiology AI models with an optimal workflow. The platform will be able to be readily integrated into a hospital infrastructure so all data is secure and protected. “As they annotate the data, all of the information would be saved on an internal server within the hospital network. The labeled data can then be used for model development or other research interests,” he said.

Rollout for the platform would require an initial set-up by the Radlearn team. “We haven’t integrated our platform at any organizations yet, but we look forward to that milestone in the future. Fortunately, the radiology infrastructure is well defined and modular, so this makes the integration process much easier.”

Goel’s next-steps are testing his platform and optimizing and improving upon current annotation functionalities. He’s currently in the process of arranging a collaboration project with an organization, that will allow him to test his system with an extremely large amount of data. And his time in the SIIM spotlight isn’t over yet either. In June, he’ll be in front of the SIIM audience again to present an update on the platform. By that time, he hopes to share some preliminary results of the big data collaboration.