Are Transformers Radiologists in Disguise?

The advent of GPT-4V

The Artificial Intelligence landscape has been buzzing with the recent release of OpenAI's GPT-4V, a variant of the GPT-4 model that exhibits interesting and surprising capabilities in various domains. OpenAI recently teased its GPT-4V model - a Multi-Modal Deep Learning model that seemingly possessed Human-like capability to understand images and voice data. While GPT-4V use cases like creating code from flow charts and adjusting bicycle seats have flooded Twitter, we believe some of the biggest gains from this current revolution of AI will come in Healthcare. So we at 5C were extremely keen to get our hands dirty on this latest breakthrough model and understand its capability to diagnose radiology scans.

Applicability of GPT4-V as Co-Pilot for Radiologists
In the intricate world of radiology, precision is paramount. Radiologists must often comb through dozens of images, looking for subtle abnormalities. Could GPT-4V, with image recognition capabilities, highlight potential concerns to radiologists?

Let's find the answer to it through an experiment by taking the case of a common pathology seen in Chest XRays - Pleural Effusion.

But first, what is a pleural effusion?

Pleural Effusion

Pleural effusion refers to the buildup of excess fluid between the layers of the pleura, the membranes surrounding the lungs.

1. Causes: Pleural effusion can result from various causes such as heart failure, pneumonia, malignancies, pulmonary embolism, and renal diseases among others.
2. Symptoms: It can present with symptoms like chest pain, cough, dyspnea (difficulty breathing), and decreased movement of the chest on the affected side.
3. Diagnosis: The initial diagnosis typically involves a combination of clinical examination and chest X-rays - and based on the results ultrasound, CT scans or sometimes analysis of the pleural fluid itself is advised.

Detecting and understanding the cause of pleural effusion is vital as it can be a sign of underlying medical conditions. Addressing it promptly can prevent complications and ensure the patient receives the appropriate treatment.

How is it Diagnosed?
Chest X-rays are the cornerstone of diagnosing pleural effusion. Within these X-rays, the effusion manifests as a distinctive white shade. However, its subtle manifestations can elude the human eye. Here, supplementary tools like CT scans and ultrasounds come into play, offering a more comprehensive diagnosis.

What are the Dangers of Missing it?
Untreated or improperly managed pleural effusion can lead to complications like:

• Empyema: Pus accumulates in the pleural space, usually due to an infection.
• Pleural fibrosis: Scarring of the pleural space which can restrict lung expansion.
• Respiratory Distress: Large effusions can compress the lung and lead to respiratory failure.

Experiment Structure and Results

To gauge GPT-4V's prowess in detecting pleural effusion, an experiment was conducted with 10 random chest X-rays known to have pleural effusion. The XRays were sent to GPT-4V, Radiologists as well as to 5C's AI.

Presented below are the results from the experiment.

The output from GPT-4V and from 5C's AI model for the Chest XRays are provided at the end of the post.

Conclusion and our thoughts

At 5C, we are massive believers in the transformative potential of AI in radiology, and our conclusion is that our experiment with GPT-4V and the above radiology images underscores the importance of specialized models. The experiment highlighted that despite GPT-4V's prowess in other domains, it might not yet be primed for radiological applications. That being said, vision transformers as a technology are going to be a key tool in the computer vision stack for AI engineers.

GPT-4V can surely help you fix your bicycle seat, but diagnosing your radiology scan? Not yet, atleast.

Warmly,

AI @ 5C Network