Deep Learning + COVID: How Vision Software is Detecting Infections

Medical images like X-rays are critical to confirming a COVID-19 diagnosis—giving doctors and radiologists visual evidence that laboratory tests are accurate. Moreover, deep learning software can lighten the workload of clinicians by analyzing thousands of medical images and identifying anomalies that refute or support a diagnosis.

There’s just one roadblock: The most popular open-source deep learning tools are difficult to use and require substantial programming expertise. It is not practical to expect healthcare workers, such as doctors, radiologists, and other clinicians, to master these tools.

A team of AI experts at Cognex set out to surmount this obstacle with a base hypothesis: Could Cognex’s industrial automation software provide an easy-to-use alternative that can match the performance of the world’s top open-source deep learning tools?

The opening test of this hypothesis showed strong potential. According to research by a five-person team of Cognex deep learning experts, the company’s state-of-the-art machine vision software equaled or surpassed the accuracy of the world’s leading open-source deep learning tools.

The study, titled “Detection of COVID-19 from Chest X-rays using Deep Learning: Comparing Cognex VisionPro Deep Learning 1.0 Software with Open Source Convolutional Neural Networks,” has drawn attention from major research publishers. The co-authors were Arjun Sarkar, Joerg Vandenhirtz, Jozsef Nagy, David Bacsa, and Mitchell Riley, all of whom work on Cognex’s Life Sciences team.

“We were surprised to learn that it’s easy for the software to differentiate between the pathologies that show up on X-rays,” said Vandenhirtz, Cognex’s Senior Business Development Manager for Life Sciences OEM Europe. “It’s is nearly impossible for humans to figure out differences in X-ray images with different pathologies. Five radiologists can give five different opinions on these kinds of images.”

Vandenhirtz coordinated the study to help extend the company’s premium machine-vision technology into the healthcare and life sciences space. The global coronavirus pandemic provided the urgency, while a large data set of COVID-19 X-rays provided test imagery for the study. He hired Sarkar, a master’s degree candidate studying biomedical engineering at FH Aachen, the University of Applied Sciences in Aachen, Germany, to conduct the experiment and summarize his findings in the research report.

Sarkar had a strong background using TensorFlow, the leading deep-learning platform from Google, Vandenhirtz said. TensorFlow requires programmers to build their models in a text-based terminal interface. VisionPro Deep Learning, by contrast, has a user-friendly GUI that requires no programming experience.

“If you can learn Microsoft Office, then you can learn VisionPro DL,” Vandenhirtz said. “Sarkar had no clue about VisionPro DL when we hired him. In two months, he learned the program, conducted the research, and wrote up his findings.” A conventional deep learning study might require years to build a network, develop models, and train algorithms. VisionPro DL dramatically reduces that time frame.

Software like VisionPro DL reflects the growing democratization of artificial intelligence, which is putting powerful learning algorithms in the hands of people with no background in programming or data science. 

“Picture a radiologist who knows nothing about TensorFlow or Linux or any of that,” Vandenhirtz said. “She lives in her radiology world; maybe she’s using an iPhone 5 on its factory settings. We can train her to use VisionPro DL in two to three days.”

Building on a Study of COVID and Deep Learning

Cognex’s study built on the findings of a research study at the University of Waterloo in Ontario, Canada. The study, titled “COVID-Net: A Tailored Deep Convolutional Neural Network Design for Detection of COVID-19 Cases from Chest X-Ray Images,” collected nearly 14,000 chest X-rays in a data set called COVIDx. Co-authors Linda Wang and Alexander Wong used open-source deep learning packages to build COVID-Net, a sophisticated neural network that analyzed the X-rays and learned to identify lungs that had tell-tale signs of COVID-19. 

A group of University of Waterloo researchers launched a startup called DarwinAI to develop commercial deep learning software to tap the value of resources such as COVID.Net, which holds immense promise but still faces a fundamental usability challenge.

“Right now, it's just a very technical kind of implementation that data scientists could leverage but certainly not a radiologist or healthcare worker, so it needs to be wrapped in a proper application UI that's fairly easy to use and that somebody who's not deeply technical can leverage,” DarwinAI CEO Sheldon Fernandez said in an interview with CDNet.

The Cognex researchers understood the implications of these limitations. VisionPro DL was developed for Cognex’s clients in manufacturing. Its developers designed it specifically so that factory managers and technicians could use deep learning to analyze images on their production lines to maintain quality control and keep defective and damaged products out of the marketplace.

In an automotive factory, for instance, Cognex’s machine vision cameras take digital pictures of parts like fenders and engine blocks. VisionPro DL scans these images for scratches, dents, and other anomalies that human inspectors often miss. Flagging these flaws in advance keeps production lines more productive and strengthens product quality. It can also be used to classify parts or defects, as well as locate parts and verify assemblies. These types of inspection tasks are often still done manually or not all because they essentially require the involvement of human judgment.

How VisionPro DL Performed on the COVIDx Data Set

A measurement called F-score assesses the overall accuracy of a deep learning system, which attempts to accurately predict patterns and anomalies on digital images.

Cognex’s researchers analyzed nearly 14,000 X-ray images in the COVID-Net data set. The images were divided into three categories: normal, non-COVID-19 pneumonia, and COVID-19.

As this table comparing multiple deep learning packages shows, COVID-Net generated strong predictive results, ranging from 92.6% on normal images to 94.7% for COVID-19 images. VisionPro Deep Learning 1.0 did even better, with 95.6% on normal X-rays and 97.0% on COVID-19 X-rays.

Of course, this is just one study. Though the Cognex team used industry-standard techniques for research and statistical analysis, it remains to be seen if other researchers can replicate the results. Moreover, the software has not been approved for medical use.

Vandenhirtz said the company’s primary short-term interest is telling the global medical community about the capabilities of this kind of software, which also has been showing promising results on CT (computed tomography) scans. It also could prove useful in areas like ophthalmology, which rely on pictures of the internal mechanisms of the eye.

For all their capabilities, deep learning algorithms cannot fully replace the wisdom of human clinicians, Vandenhirtz said. But, like the stethoscope or blood pressure cuff, it is a useful tool to help healthcare professionals perform their job at a high-level.

“We don’t believe, at least in the short to medium term, that AI will be capable of making a diagnosis,” he concluded. “VisionPro DL can make recommendations, but ultimately the radiologist has to decide what the image means.”

AI will not replace the radiologists, he added, but it will replace the radiologists who are not using AI.