MIT team develops AI that can predict heart failures from chest X-rays with up to 90% accuracy

Heart failures are complex conditions, and detecting them accurately can be even more complicated. Manual diagnoses made by doctors are often inaccurate or inconsistent. However, as with most challenging tasks requiring volumes of data, AI can lend a helping hand. An MIT team did just that by developing an AI system capable of analyzing X-rays to accurately anticipate the onset of certain types of heart failures.

The team of researchers hailing from the MIT Computer Science and Artificial Intelligence Lab (CSAIL) explained that the by detecting signs of excess fluid in lungs (a condition called pulmonary edema), they can determine heart failure severity on a four-level scale with a high degree of accuracy. This particular edema is one of the most outstanding signs of heart failure, and is normally only visible to the expert eye from an X-ray image. Even experts, however, can sometimes miss out on the subtle features necessary to identify the edema. This leads to inconsistent diagnoses, treatment plans, and a pretty chaotic operation.

The CSAIL team solves this problem by developing an AI model that is trained on a massive number of chest radiographs and their associated radiology reports, along with edema severity labels (ranging from 0 to 3, with 3 being the most severe condition). During the inference phase, the model attempts to calculate edema severity from the input image, and even from the given reports.

Training the model required a total of 377,110 chest radiographs along with 227,835 associated radiology reports. Once the severity labels were extracted from the reports, and a variety of filters were applied to narrow down the dataset to specific congestive heart failure cases, the team was left with a total of 247,425 image-text pairs for training purposes.

The model’s performance was evaluated on hundreds of image-text pairs, with a certified radiologist and a team of domain experts reviewing and correcting the predicted labels as needed. Once trained, the system was able to detect level-3 edemas with an impressive 90% accuracy, and the level-1 and level-2 edemas were classified with 82% and 81% accuracy respectively.

“We trained it to minimize the difference between the representations of the X-ray images and the text of the radiology reports, using the reports to improve the image interpretation … These correlations will be valuable for improving search through a large database of X-ray images and reports, to make retrospective analysis even more effective,” explained co-author and Ph.D. student Geeticka Chauhan.