Scientists Apply a Novel Machine Learning Method to Help Diagnose Deadly Respiratory Illness

Article Content

An international team of scientists led by UC San Diego electrical and computer engineering professor Pengtao Xie has developed a new algorithm that shows promise in improving the detection of pneumonia from chest x-rays. The new approach includes a two-way confirmation system that could be used as a way to complement the work and expertise of physicians in ways that minimize both human and computer error. 

The team applied its artificial intelligence system to a dataset of 5000 images, showing promising results.  The findings regarding the new Neural Architecture Search (NAS) method were published in the July in the journal Scientific Reports.

The team includes researchers from UC San Diego, the Indian Institute of Technology, and the Indian Institute of Information Technology.

Pneumonia is one of the most fatal ailments around the world, especially for children, and became even more dangerous during the worldwide pandemic of COVID-19. But detecting cases of pneumonia through chest radiography has long proven difficult even for experts. 

We had a chance to speak with Abhibha Gupta, a co-author of the paper. During her internship,Gupta was a senior-year undergrad student pursuing a bachelors in technology in computer science and engineering at the Indian Institute of Information Technology. She is currently a first-year information science graduate student at the University of Pittsburgh. Gupta was responsible for designing and conducting the experiments, carrying out the ablation studies, compiling the results and drafting the paper, with valuable inputs from Xie and Parth Sheth. We discussed what these results mean for the future of fighting pneumonia. Our conversation is edited for clarity and length.

The Scientific Reports paper presents a new approach to using AI to detect pneumonia from chest x-rays. AI systems are already being used to help doctors identify pneumonia, so what specific challenges are you addressing here?

The contributions of our article are twofold. We first present a novel machine learning paradigm that improves upon the previous neural architecture search based methods. Second, we use this tweaked framework to search for the best performing neural architecture among the candidate set of architectures for the task of pneumonia detection.

Previous methods involving pneumonia detection involve computationally expensive and bulky models with many layers that have a high training and inference time. We propose a three-level optimisation paradigm that tweaks upon the previous neural architecture based methods by implementing the learning by teaching framework. 

Our setup improves upon the detection accuracy of pneumonia with significantly less model size. This reduces training time and underlying costs and also improves upon inference time significantly. We also compared our model’s performance with human labeling and found that it gives comparable accuracy. GRAD-CAM visualization on various X-Ray images have shown that our method is able to detect the regions of opacity accurately.

What is the fundamental technical challenge you're trying to solve?

The underlying concept behind our research is that we are trying to search for a neural architecture that is able to detect pneumonia from chest x-rays. We have tried to improve upon the task of neural architecture search by implementing the Learning By Teaching or

LBT framework. The framework uses semi-supervised learning to mimic a student-teacher relationship. Just as a human teacher learns, imparts knowledge, and then tests the student to assess their knowledge, our framework consists of a teacher and student model that train together in an end-to-end manner to improve their learning abilities. 

First, the teacher trains using a human-labeled dataset, then the student gains knowledge by the pseudo labels, such as predictions generated by the teacher model on unlabelled data. Finally, unseen data is employed to assess the student’s performance and update the teacher model. We use this framework to search for high-performing network topologies in order to detect pneumonia from chest x-rays.

What do you like about using medical data sets to improve ML/AI systems?

Medical datasets impart flexibility to researchers in terms of problem formulation and solving. 

There has been significant improvement in the volume and quality of medical datasets. But, since this field is vast, it is difficult to get cleaned and processed data which is truly representative of the population. Training ML models with one kind of data can give rise to biases among them which can lead to erroneous predictions.

Do you have any advice for people studying machine learning who are interested in getting involved in improving healthcare via AI/ML?

My advice for people would be to do a thorough literature review and interact with medical experts before formulating solutions to problems. One should also try to procure data from varied sources so that the ML model is truly able to handle all kinds of specimens with substantial accuracy. 

In the case of healthcare, where the decision of subject matter experts matters more, one can compare the model results with their labellings to get a better idea about the performance of the model and identify potential errors in predictions.

Can you describe the urgency of detecting pneumonia in light of what we’ve learned from fighting the COVID-19 pandemic?

The Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) is the pathogen responsible for the Coronavirus disease 2019 (COVID-19) pandemic. The new COVID-19 induced pneumonia causes severe inflammation in lungs. It damages cells and tissues of air sacs in lungs. These sacs are where the oxygen is processed and delivered to the blood. A study conducted shows that the mortality rate of patients suffering from COVID-19 induced pneumonia is 56%, showing that severe COVID-19 pneumonia is associated with very high mortality. 

There is an urgent need to develop new methods that aid in the effective identification of pneumonia in early stages to reduce patient mortality. In countries that lack medical resources, especially in the rural areas, there is a strong need for computer aided diagnosis systems. These artificial intelligence based systems can help radiologists detect pneumonia from chest x-ray images in the early stages itself.

What are some of the ways this work can help alleviate suffering?

Our framework acts as a two-way confirmation system that can then be confirmed by the attending physician, drastically minimizing both human and computer error. The results suggest that our method can be used to improve diagnosis relative to traditional techniques, which may improve the quality of treatment. 

Since our model is able to distinguish between instances containing pneumonia with high accuracy and has a low memory footprint it can be successfully deployed in real life settings to provide a priori diagnosis before the condition of the patient deteriorates further.

Paper: “Neural architecture search for pneumonia diagnosis from chest x-rays.” Authors include Abhibha Gupta, Department of Computer Science and Engineering, Indian Institute of Information Technology; Parth Sheth, Department of Electronics and Communication Engineering, Indian Institute of Technology; and Pengtao Xie, Department of Electrical and Computer Engineering, University  of California, San Diego. Gupta and Sheth contributed equally.