RSNA and ASNR have developed what they claim is the largest dataset of expert-annotated brain hemorrhage CT images.

RSNA and ASNR create largest collection of annotated brain hemorrhage CT images

April 30, 2020
by John R. Fischer, Senior Reporter
Two medical societies and more than 60 volunteer neuroradiologists have developed what they claim is largest public collection of expert-annotated brain hemorrhage CT scans to speed up the creation of machine learning algorithms for diagnosing and characterizing the condition.

Curated by the Radiological Society of North America (RSNA) and the American Society of Neuroradiology (ASNR), the formation of the data set is a product of the latest edition of the Radiology Society of North America Artificial Intelligence (AI) Challenge, in which participants were tasked with creating an algorithm to assist in identifying and characterizing intracranial hemorrhages from brain CT scans. The development process and observations of the data set were recorded in a paper.

“The value of this challenge is to create a data set that might lead to a generalizable solution, and the best way to do that is to train a model from data originating from multiple institutions that use a variety of CT scanners from various manufacturers, scanning protocols and a heterogeneous patient population,” said the paper’s lead author, Dr. Adam Flanders, a neuroradiologist and professor at Thomas Jefferson University Hospital, in a statement.

The organizers of the competition compiled the collection from scratch using the brain hemorrhage CT data sets of Stanford University in Palo Alto, California; Universidade Federal de São Paulo in Brazil; and Thomas Jefferson University Hospital in Philadelphia, Pennsylvania.

An open call for volunteers within the ASNR membership to annotate images attracted 140, from which 60 were selected to assess 874,035 brain hemorrhage CT images in 25,312 unique exams. Volunteers marked each image as normal or abnormal, with the hemorrhage subtype indicated on abnormal images.

Accurately diagnosing the presence and type of intracranial hemorrhage is important for selecting the correct treatment, as even a small hemorrhage can lead to death if it is in a critical location.

The release of the data set led to more than 22,000 submissions from 1,787 individual competitors in 1,345 teams from 75 countries in the challenge. The organizers expect its development by RSNA and a subspecialty like ASNR will help foster future collaborations.

“I was really impressed by the huge volunteer effort and the tremendous worldwide interest in this project,” Dr. Flanders said. “The data set we created for this challenge will endure as a valuable ML research resource for years to come.”

The data set will be used again in this year’s competition, which will be a collaboration between RSNA and the Society of Thoracic Radiology to detect and characterize pulmonary embolisms on chest CTs.

The paper was published in Radiology: Artificial Intelligence.