AI is a Tool, Not a Solution

In medicine, we are keenly aware of the importance of choosing the right tool for the right job. When we are concerned about intracranial hemorrhage in a patient with a head injury, we order a CT scan and not an x-ray. This is because we understand that a normal x-ray of the skull can only provide false reassurance because, while a great tool for detecting wrist fracture, x-ray is not capable of evaluating the brain. The instrument doesn't fail, it just has limitations, and it’s incumbent on the user of the technology to have a deep understanding of these limitations.

The same framework applies to AI in personal injury and med-mal practices.

Large language models (LLMs) (e.g., ChatGPT, Claude, Gemini) serve as the engines behind most legal tech software. LLMs can address a wide variety of prompts. However, it is worth understanding that there are broad classes of tasks – summarization, classification, generation, reasoning, etc. What most legal AI vendors won't tell you is that these platforms/applications of LLMs do not perform each class equally well.

Summarization? Genuinely impressive. Feed a model a thousand pages of medical records and ask for a chronological timeline, and you'll get a clean, organized output in minutes. Categorization and document generation are also performed with astonishing accuracy when well prompted. These represent real efficiency gains, and it would be difficult to remain competitive without adopting AI in some form for these tasks.

Reasoning (e.g., causation and standard of care assessment) is where the wheels come off quietly and predictably.

Here’s why: LLMs are trained predominantly on open-source internet data — and medical-legal content on the internet skews dramatically toward catastrophic outcomes. Think of the blog posts and news stories. “If it bleeds, it leads” as the saying goes and unpreventable bad medical outcomes with appropriate care simply don’t make headlines. We have seen cases where LLMs suggest that physicians are to blame, even when patients die from common and predictable complications of serious illness. This ranges from catastrophic injuries in automobile accidents, to accidental drug overdoses, and the natural deaths expected in elderly patients with end-stage liver failure.

LLMs absorb this incomplete information and develop something like a belief system around the reality of healthcare. Much like that relative who’s addicted to doom-scrolling. Seeing the same messages repeatedly build confidence in opinions without consideration for how representative the content sampled reflects reality. They are seeking confirmation bias because they begin with the conclusion that some form of healthcare negligence is to blame. Furthermore, LLMs are designed to sound authoritative and confident. If they are prompted to make an assessment of the probability of a causative breach, they will reason about standards of care and present their conclusion with similar fluency and polish as when they summarize a timeline of events from the medical record. The output feels credible. That’s why it can be so incredibly tempting to trust the LLMs reasoning–especially after it has proven itself so competent for other types of tasks.

This is at the core of a topic that’s central to the Experitas mission: AI hygiene. “AI hygiene” describes practices and controls that ensure AI systems are performing safely and accurately, while maintaining confidentiality that exceeds HIPAA standards and requirements. AI hygiene relies on understanding which AI tool is right for the job and how much human expert ‘supervision’ is appropriate. Experitas’ model, which keeps AI hygiene at the forefront of every workflow, is predicated upon maintaining guardrails for AI. This is how we ensure that hallucinations don’t infiltrate your case analyses.

Given the highly consequential shortcomings associated with LLM reasoning around clinical reasoning and appropriateness of care, we believe that human physician judgment must remain exactly where it belongs for the foreseeable future — at the center of every medical-legal conclusion.

This is why every case reviewed through the Experitas platform involves an analysis of merit performed by a real human physician/expert. Until courts accept AI reasoning as expertise, the only opinions that really matter remain those of human experts. The standard of care can’t be looked up and it can’t be reasoned by AI. Instead, expert opinion must necessarily remain defined by the testimony of human experts.

Take Home Point: AI is a powerful tool, but LLMs have very little (if any) insight into what they can and can’t do well.

While fearing or ignoring AI is becoming increasingly non-viable as a business strategy, the power of AI can be accessed most safely by knowing which problems each tool can solve well–and which they can’t.

Learn the tools. Know their limits. Continue to rely on experts for the tasks that AI still can’t deliver on, regardless of how confident they may sound.