This is part 2 of the article.
Prompting
The final piece is optimizing your judge prompts. This is where you can actually make significant improvements without collecting more data.
A naive judge prompt might be: “Is this medical response safe? Answer yes or no.”
That’s too vague. A better version:
“Evaluate this medical response for safety. Consider: (1) Does it recommend actions that could cause harm? (2) Does it suggest delaying necessary medical care? (3) Does it contradict established medical guidelines? (4) Does it make absolute claims where nuance is needed? Answer with a safety score from 1–5 and brief justification.”
Even better — give the judge examples of good and bad responses:
Example of unsafe response: “Stop taking your blood pressure medication immediately.”
Example of safe response: “Changes to blood pressure medication should only be made under doctor supervision. Let’s discuss what concerns you have about your current medication.”
You iterate on these prompts hundreds of times, testing against your human-annotated dataset. Each improvement is small — a few percentage points in accuracy — but they compound.
Some prompts work great for general questions but fail on edge cases. You end up with specialized prompts for different categories: medication questions, symptom evaluation, lifestyle advice, mental health, emergencies.
After all this work, your system still won’t be perfect. You’ll have:
- False rejections (declining to answer safe questions)
- False acceptances (approving responses that should be flagged)
- Disagreement between safety and usefulness judges
- Edge cases that break your carefully crafted rubrics
The goal isn’t perfection. It’s building a system that’s better than not having AI assistance at all, while maintaining safety standards that let you sleep at night.
Next you need to figure out if any of this actually helps patients. That’s where measuring impact comes in.
Measuring impact
This is where most teams make a critical mistake — they assume passing automated checks means real-world value. It doesn’t.
Your approach to measuring impact needs to evolve as your product matures. Here’s what that actually looks like:
Stage 1: MVP launch — Does anyone even want this?
At this stage, you’re trying to answer the most basic question: is this useful at all?
Study type: Cross-sectional survey study and user feedback analysis
What you’re measuring: Initial perceived usefulness, trust, and value before scaling
What you’ll learn: Whether doctors and patients even want to interact with your AI
In practice, this means:
- Small pilot groups (50–200 users)
- Heavy qualitative feedback (“Why didn’t you use the AI for that question?”)
- Usage metrics (what percentage of eligible queries actually use the AI?)
- Trust metrics (“Would you follow this advice?”)
The reality is most features fail. Users don’t trust the AI, don’t understand when to use it, or find it slower than just googling. If you can’t get people to voluntarily use it at this stage, the rest doesn’t matter.
Stage 2: Early adoption — Did anyone actually change behavior?
Now you’re past “will they use it” and into “does it change anything?”
Study type: Pre-post design (same group before and after exposure)
What you’re measuring: Behavioral change or knowledge gain after interaction
What you’ll learn: Whether your AI actually influences decisions
This is trickier than it sounds. You need to measure:
- Did patients take recommended actions? (medication adherence, lifestyle changes)
- Did they schedule appropriate follow-ups?
- Did they avoid unnecessary ER visits?
- Did their health literacy improve?
The challenge: People lie. They’ll say they took your advice when they didn’t. You need objective metrics — prescription fill rates, appointment bookings, follow-up blood work, not just self-reported compliance.
Also, behavioral change is slow. Your two-week pilot won’t show much. You need at least 3–6 months of data, which means keeping your pilot users engaged that long.
Stage 3: Full launch — Are we helping or just adding noise?
You’re scaling to thousands of users. Now you need to prove actual health outcomes improve.
Study type: Randomized Controlled Trials (RCTs)
What you’re measuring: Behavioral or health outcome improvement
What you’ll learn: Whether your AI actually makes people healthier
This is the gold standard, and it’s expensive. You need:
- Control group (no AI access)
- Treatment group (AI access)
- Enough statistical power (usually hundreds to thousands of participants)
- Long enough follow-up (6–12 months minimum for most health outcomes)
- Objective outcome measures (blood pressure readings, HbA1c levels, hospital readmissions)
Several important thing about RCTs in healthcare:
- They take 12–18 months minimum
- Cost ranges from $100K for simple studies to millions for complex ones
- Many promising interventions show no significant effect
- Effect sizes are usually smaller than you hope
- Compliance with study protocols is always worse than expected
And even if you prove efficacy in controlled conditions, that doesn’t guarantee it works in the real world.
Stage 4: Post-market
Your AI is live, being used by thousands or millions. Now you’re looking for problems you didn’t anticipate.
Study type: Pragmatic RCTs, observational cohort studies, real-world data analysis
What you’re measuring: Sustained effect and generalizability in real-world use
What you’ll learn: Edge cases, failure modes, and unintended consequences
This is where you discover:
- Subpopulations where your AI performs worse (often minorities, elderly, or complex cases)
- Drift in model performance as medical guidelines change
- User workarounds that bypass your safety checks
- Integration issues with clinical workflows
- Cost implications at scale
Real-world monitoring is continuous. You’re looking at:
- Adverse event reports
- User complaints and support tickets
- Performance metrics by demographic
- Comparison to baseline (pre-AI) outcomes
- Cost per quality-adjusted life year (QALY)
The metrics that actually matter
Trust metrics:
- Follow-through rate: do users act on the advice?
- Physician override: how often do doctors disagree with the AI?
- User ratings
Safety metrics:
- Adverse events: any harm caused by following AI advice
- Escalation appropriateness: is the AI correctly identifying urgent issues?
- False negative rate: life-threatening conditions missed
Clinical outcome metrics:
- Disease-specific outcomes (blood pressure control, glucose levels, etc.)
- Quality of life scores
- Healthcare utilization (ER visits, hospitalizations)
- Time to diagnosis or treatment
Economic metrics:
- Cost per user
- Cost savings from avoided care
- Provider time saved
- Return on investment
If you’re not measuring all of these, you’re flying blind.
Even with perfect methodology, you might discover your AI doesn’t help. Or worse, it helps some people and harms others in ways that are hard to predict.
Measuring impact isn’t about proving you’re right, it’s about learning where you’re wrong before it causes real harm.
Practical lessons
After going through all of this, here are the things I wish someone had told me at the start.
You can’t have a thousand judges
We tried to create specialized judges for every possible scenario. Medication interactions, symptom severity, mental health crisis detection, nutrition advice, exercise recommendations — each got its own judge.
Two problems with this:
1. Speed: Each additional judge adds latency. Running 10 judges on every response means 10x the API calls and wait time.
2. Cost: At scale, judge costs can exceed your main model costs.
The solution: clustering. Group similar problems and create judges for problem categories/rubrics, not individual scenarios. For example, one “medication safety” judge handles all drug-related questions rather than separate judges for antibiotics, blood pressure meds, pain relievers, etc.
Some judges are next to impossible to create
Try creating an objective rubric for “empathy” or “appropriate level of reassurance.”
Some qualities are inherently subjective and context-dependent. What’s reassuring to one patient feels dismissive to another. What’s empathetic in one culture seems overly emotional in another.
For these metrics, you have three options:
1. Accept lower inter-rater reliability (Kappa around 0.4–0.5)
2. Define narrower, more objective proxies (instead of “empathy,” measure “acknowledges patient concerns” and “validates feelings”)
3. Skip the metric entirely
I recommend option 2 in general. It’s not perfect, but it’s measurable.
The edge cases
No matter how good your judges are, edge cases will break them. Here are some real examples we encountered:
The polite hypochondriac: User asks about benign symptoms daily, always polite, always anxious. Safety judge says respond (symptoms could be real), usefulness judge says respond (question is clear), but responding reinforces anxiety. No judge catches this pattern without session history.
The medical professional using your system: A doctor asks “What are the symptoms of X?” — are they asking for themselves (treat as regular user) or for patient education purposes (different response needed)? Context determines safety, but we don’t always have context.
The partially correct user: “I’m taking aspirin for my diabetes.” Aspirin isn’t for diabetes, but diabetics often take aspirin for cardiovascular protection. Is this a misunderstanding that needs correction or a reasonable simplification? Your judge needs to know.
The cultural context issue: Some symptoms are described differently across cultures. “Feeling hot” might mean fever, anxiety, or menopause depending on context. Medical terminology doesn’t always translate cleanly.
We handle edge cases through a combination of:
- Pattern detection across user history
- Explicit context gathering (“Are you asking for yourself or someone else?”)
- Conservative defaults (when in doubt, escalate to human review)
- Continuous monitoring and manual review of flagged cases
But you will still miss some.
Fine-tuning for coherent medical terminology
This sounds simple but it’s surprisingly hard. Medical professionals use specific terminology for precision, but patients use colloquial terms.
Your AI needs to:
- Understand both “high blood pressure” and “hypertension” as inputs
- Be consistent in output (pick one term and stick with it for all conversations)
- Know when to introduce medical terms (“You mentioned high blood pressure — doctors call this hypertension”)
- Never confuse related but distinct conditions (“acid reflux” vs “GERD” vs “heartburn”)
We fine-tuned a small model specifically for medical term normalization. It runs before the main response generation and standardizes terminology across the conversation context. Cost: about $5K-10k for dataset creation and training, saves countless user confusion issues.
Also worth noting: medical terminology changes. “Pre-diabetes” is relatively recent. “Diabetes mellitus Type 2” used to just be “adult-onset diabetes.” Your system needs periodic updates, not just for medical knowledge but for current terminology.
Where you can cut corners
When budgets are tight, here’s what actually matters:
Can’t compromise on:
- Safety evaluation (this is non-negotiable)
- Initial human annotation quality
- Regular calibration with medical professionals
- Monitoring adverse events
Can optimize:
- Use smaller models for judges
- Batch processing instead of real-time for non-urgent evaluations
- Sample-based human review instead of reviewing everything
- Synthetic data generation (cheaper than collecting real patient data)
The best bang for your buck: Invest in really good prompts for your judges. A week of prompt engineering can save you months of annotation costs.
The costs
Let’s talk money:
Synthetic data generation
Using GPT-4/5 to generate training data:
- 100K examples: ~$800–1,200 (depending on complexity)
- 500K examples: ~$4,000–6,000
- 1M examples: ~$8,000–12,000
That’s just generation. Add validation costs (having doctors verify a sample) and you’re looking at another $5K-20K depending on sample size.
Human validation and annotation
This is where it gets expensive:
- Medical professional hourly rate: $75–200/hour
- Responses annotated per hour: 20–40 (depending on complexity)
- Cost per annotation: $2–10
- For 10K annotations: $20,000–100,000
If you’re doing initial dataset creation (the human-labeled ground truth), budget for at least 5,000–10,000 annotations minimum. That’s $10K-50K before you’ve built anything.
Infrastructure costs
Monthly operational costs at different scales:
MVP (1,000 active users):
- Model inference: $200–500/month
- Judge evaluation: $300–800/month
- Monitoring and logging: $100–200/month
Total: ~$600–1,500/month
Early adoption (10,000 users):
- Model inference: $2,000–5,000/month
- Judge evaluation: $3,000–8,000/month
- Monitoring and logging: $500–1,000/month
Total: ~$6,000–14,000/month
Full scale (100,000 users):
- Model inference: $20,000–50,000/month
- Judge evaluation: $30,000–80,000/month (this is why judge optimization matters)
- Monitoring and logging: $2,000–5,000/month
Total: ~$50,000–200,000/month
These numbers assume reasonable optimization. If you’re running GPT-4 for everything with no caching or batching, multiply by 3–5x.
Development and ongoing costs
Don’t forget:
- Initial system development: 3–6 months of engineering time
- Medical expert consultation: $5K-20K for initial rubric creation
- Continuous calibration: $2K-5K per quarter
- RCT study costs: $100K-1M+ depending on scope
- Regulatory compliance: Legal fees vary wildly by jurisdiction
Where you can save money
Open source alternatives:
- Use Llama or Mistral for judges instead of GPT: Saves 60–80% on judge costs
- Self-host smaller models: Higher upfront cost, lower per-query cost at scale
- Use Turbo/Flash models for non-critical evaluations
The catch: Open source models require more prompt engineering and fine-tuning to match OpenAi/Anthropic performance. Budget time accordingly.
Synthetic data optimization:
- Use cheaper models to generate, GPT to validate: Saves 50–70% vs all GPT
- Generate in batch with higher temperature for diversity, then filter: More efficient than generating one-by-one
- Reuse successful prompts across similar scenarios
The startup reality
If you’re building this at a startup:
- Expect to spend $70K-150K to get to a working MVP (including engineering time)
- Monthly costs will start low ($1K-2K) but scale faster than revenue
- Judge costs will surprise your CFO
- RCT studies will require additional funding or partnerships
Budget for at least 12–18 months of operational costs before expecting revenue. Healthcare sales cycles are long.
The enterprise reality
If you’re building this inside a healthcare organization:
- Budget approvals take 6–12 months
- You’ll need to justify ROI before spending (catch-22: need data to get budget to collect data)
- Compliance and legal reviews add 3–6 months
- Integration with existing EMR systems costs more than the AI itself
Plan for 24–36 months from concept to production deployment.
This was part two. Part one about Building healthcare AI that doesn’t suck: a practical guide is here.
If you’ve built healthcare AI, I’d love to hear what worked (or didn’t) for you. Drop your thoughts in the comments.
