Not So Black and White: AI-Powered Imaging in Veterinary Medicine

And now for something that will bother everyone.

“If a physician can be replaced by a computer, then he or she deserves to be replaced by a computer.” -Warner Slack, Harvard Medical School

I’ve been a little slow to write a piece on diagnostic imaging in veterinary medicine.

I find myself between two opposing factions and developing an opinion that will likely bother both of them. On one side are the veterinary radiologists who, for a variety of reasons, sternly reject the wide application of artificial intelligence to image recognition. I took a crack at a literature review and assumed a confrontational stance of my own. I rejected the notion that AI needs to be perfect before being used. AI, like the real world, might not be as some tunnel-visioned radiologists see things.

But then I bristle at the decidedly grandiose promises of those peddling AI-powered radiology software. I have challenged the salespeople to provide studies and research backing up the claims. I received little follow-up. Some don’t call me back at all.

And then there are some radiologists like Ryan Appleby, who seems a moderate. He is certainly familiar with AI, writing one of the best papers I’ve found on its workings and use. It’s also one of the only ones found in a veterinary journal, and it’s a very good piece of work. It’s become dated, as articles about AI often are,¹ but the foundational principles are covered well.

I wrote this disclaimer of sorts in a previous article but I believe it still applies: I occasionally disagree with prevailing opinions in my profession, and I tend to disagree - shall we say - emphatically and enthusiastically. In general, I play the game at maximum velocity, and that, along with a habit of sledgehammer-direct communication, has frequently rubbed folks the wrong way. I don’t know that my career is better for it, but sometimes my conscience is. As with many things in life, I’m working to get better.

Anyway, when it comes to AI-powered radiology, I have some thoughts.

Posts with happy cartoon images seem to get better engagement.

Heads, We Win

Fairly or not, I’m going to use Dr. Ira Gordon and Dr. Eli Cohen’s paper on AI in Radiology as representative of the rejections of AI use in veterinary from radiologists. Dr. Cohen lectured on AI at the ACVIM last year, lectured on AI at the ACVIM last year and his webinar lecture is available on AVMA’s Axon.

In an overly simplified, I believe that in their work they overstate the problems with AI and make inadequate mention of potential. The uses of the technology are not limited to radiology, but the uses in diagnostic imaging are particularly exciting.

Two, they point out that AI is flawed but fail to acknowledge that, in this wicked world, everything is flawed. Even Mendel’s suspiciously agreeable pea plants didn’t fit perfectly Punnett square. Have you ever used a piece of technology that, at some point, didn’t work entirely correctly? I think you have because if you’re reading this piece on a computer or a phone, then I’m certain it hasn’t always worked exactly as it should.

Third, it sets what I regard to be an unattainable standard for the use of AI in veterinary medicine. I think it’s easy for folks who are often a step removed from the pill guns, germs, and steel of veterinary medicine to imagine that imperfection equals failure. But I work in a clinic, not behind a computer screen (mine, admittedly, doesn’t work perfectly), and sometimes the “gold standard” of care is what is “good enough.” And sometimes it isn’t.

Tails, You Lose

And then we’ve got these radiology companies that seem to believe that testing an AI on cases from their own training data means the AI has bested the radiologists.

There’s a big gaping chasm of difference between “it works” and “it works better than every expert.” The boastful claims were written with the enthusiastic optimism of a fifth grader who threw his first strikeout.

Geoffrey Hinton, the so-called godfather of AI, formerly of Google’s DeepMind team, predicted that we can stop training radiologists in five years. They will be obsolete. He said that in 2017. More than five years later, we very much need radiologists.

He further said in a recent interview that he thinks we will not radiologists in 10-15 years, rather defensively pointing out that he was only off by a factor of 3-5. I’m intolerant of this claim too. Tell me, dear colleagues, how often in your job can you afford to be off by a factor of 3-5 in an estimation of outcome? Generally, I can’t, but I only practice medicine for a living.

The advertising of the radiology services is a bit troubling. It’s more sales than science. One provider offers studies, presumably but not definitively, of their technology. The first is certainly promising, but it’s limited to dogs with pleural effusion. It doesn’t test the efficacy of the software in diagnosing literally anything else. The second is behind a paywall, but only studies cardiogenic pulmonary edema. That’s all the studies they have on their software, just those two conditions.

I expect that I am like most veterinarians in that I’d be hoping to use the software to help me evaluate more than two conditions and would like scientific studies to back up the novel technology I hope to use.

The screenshots below are from the websites of some of the purveyors of AI-powered radiology support.

Neat, but the research is very limited.

Also an exciting claim, but the research falls a bit short yet.

I’m not sure that a report of the findings is so much “transparency” as “what was promised by the software.”

This is where I think the radiologists have a right to be sounding off. These claims border on the outrageous, and they’re at least as nonsensical as the radiologists’ own demands for perfection and complete transparency. But if I were the radiologists, I’d let the marketing teams oversell it and scorch the earth. So the technology is imperfect? Let it fail, and it’ll be years before people come back to it. Piping up with righteous outrage doesn’t move the needle on public opinion, it just makes everybody look bad.

I want more studies. I want more research on various conditions. I want it to be published even if the technology is just okay. I really want to be published if the technology doesn’t work at all. I want more clarity on what is reliable and what is not.

I want science, not hype. And there should be nobody better at telling the difference between the two than scientists like doctors.

Science is a rigorous, systematic endeavor that organizes knowledge in the form of testable explanations and predictions about the world. Contemporary medicine applies science, research, genetics, and medical technology to diagnose, treat, and prevent injury and disease. We don’t get to stop because a new technology worked on diagnosing one or two conditions, we don’t get to make sweeping claims about said technology’s efficacy.

The Jagged Edge of the Coin

The functionality of artificial intelligence has been described as a “jagged edge.” It refers to the fact that, at present, there are some things that AI does better than humans and some things it worse. You can’t fall asleep at the wheel and hope for a good outcome, but it can be a powerful augmentation. 

There are studies that have shown that AI outperforms radiologists, evidence that radiologists outperform AI in the real world², that AI can confound radiologists, and that AI can enhance radiologists.

Radiologists would have it work perfectly before widespread use, but I believe that is unreasonable. Such a mandate far exceeds what radiologists are currently capable of doing. Studies have demonstrated that even the yet imperfect AI augmentation can lead to more accurate diagnoses and better clinical outcomes for patients. That doctors would seek to halt the use of a technology that demonstrably leads to improved patient outcomes is reprehensible. I think we should all take note of those who would protect or advance their own interests at the expense of the greater good.

I’m equally bothered by companies that would obscure the truth of their technology’s capacity by failing to perform or failing to publish scientific studies that would further our understanding. If the software is so very valuable and broadly useful as is claimed in their advertising materials, I eagerly await to see it proven by the robust and rigorous scientific testing that I believe is required to justify such claims.

I spoke with the president of Vetology, Eric Goldman.³ Mr. Goldman makes an important distinction between an AI-report being a “screening result” and not a “diagnostic.” I think it has suddenly become vital to know the difference. A diagnostic report can only be delivered by a veterinarian, a qualified person taking responsibility for the decision. That an AI-powered radiology company is led by someone with the wisdom to understand that difference is a very good sign for that company and for the profession.

Now that I’ve found my way to criticizing both sides of this debate, I’ll offer what I believe to be the best path toward advancing the use of radiology enhanced by artificial intelligence. We need more studies. We need studies on more than two pathologies. We need to work to understand the technology, its value, and its limitations. We need the expertise of radiologists to help develop it. The more we know and the more we understand, the better the correctness of diagnoses and, as a natural sequela, better clinical outcomes will be for our patients.

Things don’t need to be perfect to make things better, but good enough isn’t good enough if it’s a static end when it comes to the lives and well-being of our patients.

We have work to do.

1. It’s very difficult to write articles about artificial intelligence that have lasting relevance. Dr. Appleby’s paper was published in April of 2022. By November of 2022, OpenAI had launched ChatGPT, a massive advancement in the capability and utility of large-language models. The technology lacked the decency to give a very good paper even a year’s shelf life.

2. I include no study to demonstrate this, as I could find none worthy of inclusion. It’s taken as a given that radiologists outperform new technology at this time. I accept that reality.

3. Mr. Goldman reached out to correct my mistake of addressing him as a doctor. He describes himself as a technologist, a way cooler title anyway. The mistake was mine, a result of assumptions about his background given his apparently considerable knowledge of veterinary medicine.