Will Polaris be the Guiding Star of Medical AI?
A clever architecture and the potential for a big step in the development of medical communications AI.
First, I’d like to say I’m disappointed in the American Veterinary Medical Association. There is a single English-language lecture on artificial intelligence given by a veterinarian at the 2024 AVMA Convention. And it’s not specific to medical AI.
There is a lecture titled “Integrating ChatGPT into Veterinary Clinical Practice.” Except it’s not given by someone who has ever been in veterinary clinical practice, let alone a veterinarian.
There are six lectures about bees. And six more about euthanasia. And nine on social media. Glad the profession can stay ahead of the curve on things like bees and social media.
When I see how the most prominent veterinary professional organization in the world treats the biggest technological advancement of the decade, it evokes serious skepticism about how they handle everything else.
My salty griping out of the way, let’s get to it.
The exciting news is Polaris from Hippocratic AI.
In a preprint, the team at Hippocratic AI outlines the development of Polaris, a first-of-its-kind Large Language Model (LLM) system designed for real-time patient-AI healthcare conversations. Unlike previous healthcare AI that focused on tasks such as question answering, Polaris emphasizes long, multi-turn voice conversations to enhance safety and reduce errors. It comprises a constellation of multi-billion parameter LLMs (it’s actually more than a trillion), including a stateful primary agent for engaging conversations and several specialist support agents for healthcare-specific tasks.
Constellation Architecture
Polaris employs a unique "constellation" architecture composed of several key components:
Primary Agent: A stateful, multi-billion parameter LLM that serves as the core of the conversation. It is responsible for driving patient-friendly, engaging conversations and integrating inputs from various specialist support agents to ensure the conversation is medically accurate and adheres to healthcare protocols.
This is the part of the system that interacts with the human patient.
Specialist Support Agents: A set of specialized LLMs, each focused on specific healthcare tasks performed by professionals like nurses, social workers, and nutritionists. These tasks include medication adherence checks, lab result interpretations, nutrition guidance, and privacy compliance, among others. The support agents work cooperatively with the primary agent, providing task-specific information and checks to enhance safety and reduce hallucinations.
Specifically trained agents working in concert. Now that’s a nifty idea.
Orchestration Layer: Polaris includes an orchestration layer that manages the control flow between the primary agent and the specialist support agents. This layer ensures that the conversation maintains coherence, aligns with healthcare protocols, and integrates the specialized knowledge from the support agents smoothly and correctly.
Key Components and Functions
Training and Alignment: The models within Polaris are trained on a diverse dataset comprising proprietary data, clinical care plans, healthcare regulatory documents, medical manuals, and simulated conversations between patient actors and care-management nurses. This training aims to align the models to speak and reason like medical professionals, emphasizing empathy, rapport building, trust building, and bedside manner.
Safety Mechanisms: Polaris purports to adopt a multi-pronged approach to safety, leveraging the specialized knowledge of the support agents and implementing guardrails that include manual checks and the possibility of human intervention when necessary. This is supposed to ensure that the information provided during conversations is medically accurate and contextually appropriate.
Evaluation and Adaptation
Polaris has undergone comprehensive clinician evaluation - more than 130 physicians and more than 1,100 nurses - where it was assessed by healthcare professionals in various scenarios. This evaluation helped fine-tune its performance, ensuring it meets high standards for medical safety, clinical readiness, conversational quality, and patient education.
Polaris's architecture combines the strengths of a large, conversational LLM with the specialized knowledge of healthcare professionals encapsulated in specialist support agents. This architecture enables Polaris to conduct meaningful, safe, and empathetic conversations with patients, addressing a wide range of healthcare needs.
The primary agent is trained to emulate nurse-like conversations to build trust and rapport, emulate empathy, and demonstrate medical reasoning with patients. The support agents focus on areas like medication adherence, lab and vitals interpretation, and nutrition guidance, among others. These agents work cooperatively, optimizing for diverse objectives to improve conversational quality and safety, a built-in safety net that moves with the conversation.
A significant part of the preprint details the comprehensive clinician evaluation of Polaris. Over 1,100 U.S. licensed nurses and 130 physicians participated in end-to-end conversational evaluations, acting as patients to assess the system across various measures.
The evaluations showed that Polaris performs on par with human nurses on aggregate across dimensions like medical safety, clinical readiness, patient education, conversational quality, and bedside manner. In specific task-based evaluations of the individual specialist support agents, Polaris outperformed both a general-purpose LLM (GPT-4) and a medium-size LLM (LLaMA-2 70B), demonstrating its effectiveness and potential in healthcare applications. This is a big deal, as many of the specifically trained models were unable to compete with the big-brained vanilla models.
The development of Polaris represents a significant step forward in the application of AI in healthcare, offering a scalable and safety-oriented solution to augment the healthcare workforce and improve patient outcomes through effective communication and medical reasoning. If you’ve tried an LLM for answering questions of clients or technicians in veterinary medicine, there’s no doubt that you’ve found it wanting.
The architecture and approach of Polaris is designed for human healthcare but offers a promising blueprint for application within veterinary medicine. By adapting the principles and functionalities of Polaris, a similar system could significantly enhance veterinary care, communication, and management. Here’s how it could be applied:
Specialized Veterinary Support Agents
Just as Polaris uses specialist support agents for specific healthcare tasks, a veterinary version could include agents specialized in various aspects of animal care, such as:
Medication and Treatment Compliance: An agent focused on ensuring pet owners understand and adhere to prescribed treatments and medication dosages for their pets.
Nutrition Guidance: Specialized in providing diet recommendations based on the specific needs of different species, breeds, or health conditions.
Symptom Checker and Triage: This could pet owners understand the potential urgency of their pet's symptoms and advises on the appropriate next steps, whether it's home care or seeking immediate veterinary attention.
Preventative Care: Offers reminders and information on routine care needs, such as vaccinations, parasite prevention, and wellness checkups.
Conversational Alignment with Veterinary Context
The system would be trained on a diverse dataset specific to veterinary medicine, including case studies, veterinary medical records, and simulated conversations between pet owners and veterinary professionals. This would ensure the system can understand and generate conversations that are medically accurate and aligned with standard veterinary practices, including the nuances of communicating empathetically with pet owners.
There may be considerable differences in veterinary practices between veterinary practices. Is there a reliable authority? I’m not sure, so I wonder how we would agree on the training data.
Veterinary Workforce Augmentation
Similar to how Polaris aims to augment the human healthcare workforce, its veterinary adaptation could alleviate the workload on veterinary professionals by handling routine inquiries, follow-up communications, and educating pet owners. This allows veterinarians and veterinary technicians to focus more on clinical tasks and direct animal care, potentially improving the efficiency and quality of veterinary services, not to mention addressing ongoing concerns for staffing shortages.
Integration with Veterinary Practice Management Software
The system could be integrated with practice management software to streamline administrative tasks such as appointment scheduling, reminders, and follow-ups. It could also provide veterinarians with summaries of owner communications, highlighting key concerns or changes in the pet's condition, thus enhancing the continuity and quality of care.
Training and Safety Protocols
Adapting a similar architecture for veterinary use would include rigorous training protocols to ensure safety and accuracy, with input from veterinary experts. It would involve the development of safety guardrails, including mechanisms for human intervention when the system encounters complex cases or limitations in its decision-making capabilities.
Potential Challenges and Considerations
Species and Breed Diversity: Veterinary applications must account for a wide range of species and breeds, each with unique healthcare needs and challenges. Further, information on conditions and circumstances can be limited.
Limited and Inconsistent Information: Evidence-based medicine isn’t as it sounds. While I’ve joked that I can’t get five veterinarians to agree on what day of the week it is, the certainty of experts citing conflicting studies borders on the fervor of religious zealots. We will need to navigate the murky waters thoughtfully.
Regulatory and Ethical Considerations: Adapting AI for veterinary use involves navigating regulatory requirements and ethical considerations specific to animal care.
While AI agents could be used to lighten the load on the veterinary workforce, the regulations would need to be adapted to account for the source of medical advice coming from an automated system. Nobody blames Dr. Google for bad advice resulting in patient harm, but I bet they’d go after a veterinary hospital or doctor who used a chatbot for teletriage.
I am, at this point, against significant and specific regulation regarding medical AI use. Adequate research has not yet been performed to facilitate the formation of a truly informed opinion about the risks and benefits. Let’s do the research, then we can form opinions - and not vice versa.
Owner Communication: The system must be adept at communicating complex medical information to pet owners in an understandable and empathetic manner, recognizing the emotional bond between pets and their owners.
Frankly, I think there’s a lot of room for improvement in client communication in our profession.
This is pretty interesting and, I think, not that hard to build. Veterinary medicine has less depth, though greater breadth, of information regarding our patients. While the potential for client communication is an exciting application and likely has the greatest financial upside to its creator, I’m a bit selfish because I imagine one of these geared toward aiding clinical decision-making.
How much better of a doctor could I be if I was ever-so-slightly turning myself into a cyborg? I’d really like to find out.