This is part 2 of the article.

Prompting

The final piece is optimizing your judge prompts. This is where you can actually make significant improvements without collecting more data.

A naive judge prompt might be: “Is this medical response safe? Answer yes or no.”

That’s too vague. A better version:

“Evaluate this medical response for safety. Consider: (1) Does it recommend actions that could cause harm? (2) Does it suggest delaying necessary medical care? (3) Does it contradict established medical guidelines? (4) Does it make absolute claims where nuance is needed? Answer with a safety score from 1–5 and brief justification.”

Even better — give the judge examples of good and bad responses:

Example of unsafe response: “Stop taking your blood pressure medication immediately.”

Example of safe response: “Changes to blood pressure medication should only be made under doctor supervision. Let’s discuss what concerns you have about your current medication.”

You iterate on these prompts hundreds of times, testing against your human-annotated dataset. Each improvement is small — a few percentage points in accuracy — but they compound.

Some prompts work great for general questions but fail on edge cases. You end up with specialized prompts for different categories: medication questions, symptom evaluation, lifestyle advice, mental health, emergencies.

After all this work, your system still won’t be perfect. You’ll have:

• False rejections (declining to answer safe questions)
• False acceptances (approving responses that should be flagged)
• Disagreement between safety and usefulness judges
• Edge cases that break your carefully crafted rubrics

The goal isn’t perfection. It’s building a system that’s better than not having AI assistance at all, while maintaining safety standards that let you sleep at night.

Next you need to figure out if any of this actually helps patients. That’s where measuring impact comes in.

Measuring impact

This is where most teams make a critical mistake — they assume passing automated checks means real-world value. It doesn’t.

Your approach to measuring impact needs to evolve as your product matures. Here’s what that actually looks like:

Press enter or click to view image in full size

Stage 1: MVP launch — Does anyone even want this?

At this stage, you’re trying to answer the most basic question: is this useful at all?

Study type: Cross-sectional survey study and user feedback analysis

What you’re measuring: Initial perceived usefulness, trust, and value before scaling

What you’ll learn: Whether doctors and patients even want to interact with your AI

In practice, this means:

• Small pilot groups (50–200 users)
• Heavy qualitative feedback (“Why didn’t you use the AI for that question?”)
• Usage metrics (what percentage of eligible queries actually use the AI?)
• Trust metrics (“Would you follow this advice?”)

The reality is most features fail. Users don’t trust the AI, don’t understand when to use it, or find it slower than just googling. If you can’t get people to voluntarily use it at this stage, the rest doesn’t matter.

Stage 2: Early adoption — Did anyone actually change behavior?

Now you’re past “will they use it” and into “does it change anything?”

Study type: Pre-post design (same group before and after exposure)

What you’re measuring: Behavioral change or knowledge gain after interaction

What you’ll learn: Whether your AI actually influences decisions

This is trickier than it sounds. You need to measure:

• Did patients take recommended actions? (medication adherence, lifestyle changes)
• Did they schedule appropriate follow-ups?
• Did they avoid unnecessary ER visits?
• Did their health literacy improve?

The challenge: People lie. They’ll say they took your advice when they didn’t. You need objective metrics — prescription fill rates, appointment bookings, follow-up blood work, not just self-reported compliance.

Also, behavioral change is slow. Your two-week pilot won’t show much. You need at least 3–6 months of data, which means keeping your pilot users engaged that long.

Stage 3: Full launch — Are we helping or just adding noise?

You’re scaling to thousands of users. Now you need to prove actual health outcomes improve.

Study type: Randomized Controlled Trials (RCTs)

What you’re measuring: Behavioral or health outcome improvement

What you’ll learn: Whether your AI actually makes people healthier

This is the gold standard, and it’s expensive. You need:

• Control group (no AI access)
• Treatment group (AI access)
• Enough statistical power (usually hundreds to thousands of participants)
• Long enough follow-up (6–12 months minimum for most health outcomes)
• Objective outcome measures (blood pressure readings, HbA1c levels, hospital readmissions)

Several important thing about RCTs in healthcare:

• They take 12–18 months minimum
• Cost ranges from $100K for simple studies to millions for complex ones
• Many promising interventions show no significant effect
• Effect sizes are usually smaller than you hope
• Compliance with study protocols is always worse than expected

And even if you prove efficacy in controlled conditions, that doesn’t guarantee it works in the real world.

Stage 4: Post-market

Your AI is live, being used by thousands or millions. Now you’re looking for problems you didn’t anticipate.

Study type: Pragmatic RCTs, observational cohort studies, real-world data analysis

What you’re measuring: Sustained effect and generalizability in real-world use

What you’ll learn: Edge cases, failure modes, and unintended consequences

This is where you discover:

• Subpopulations where your AI performs worse (often minorities, elderly, or complex cases)
• Drift in model performance as medical guidelines change
• User workarounds that bypass your safety checks
• Integration issues with clinical workflows
• Cost implications at scale

Real-world monitoring is continuous. You’re looking at:

• Adverse event reports
• User complaints and support tickets
• Performance metrics by demographic
• Comparison to baseline (pre-AI) outcomes
• Cost per quality-adjusted life year (QALY)

The metrics that actually matter

Trust metrics:

• Follow-through rate: do users act on the advice?
• Physician override: how often do doctors disagree with the AI?
• User ratings

Safety metrics:

• Adverse events: any harm caused by following AI advice
• Escalation appropriateness: is the AI correctly identifying urgent issues?
• False negative rate: life-threatening conditions missed

Clinical outcome metrics:

• Disease-specific outcomes (blood pressure control, glucose levels, etc.)
• Quality of life scores
• Healthcare utilization (ER visits, hospitalizations)
• Time to diagnosis or treatment

Economic metrics:

• Cost per user
• Cost savings from avoided care
• Provider time saved
• Return on investment

If you’re not measuring all of these, you’re flying blind.

Even with perfect methodology, you might discover your AI doesn’t help. Or worse, it helps some people and harms others in ways that are hard to predict.

Measuring impact isn’t about proving you’re right, it’s about learning where you’re wrong before it causes real harm.

Practical lessons

After going through all of this, here are the things I wish someone had told me at the start.

You can’t have a thousand judges

We tried to create specialized judges for every possible scenario. Medication interactions, symptom severity, mental health crisis detection, nutrition advice, exercise recommendations — each got its own judge.

Two problems with this:

1. Speed: Each additional judge adds latency. Running 10 judges on every response means 10x the API calls and wait time.

2. Cost: At scale, judge costs can exceed your main model costs.

The solution: clustering. Group similar problems and create judges for problem categories/rubrics, not individual scenarios. For example, one “medication safety” judge handles all drug-related questions rather than separate judges for antibiotics, blood pressure meds, pain relievers, etc.

Some judges are next to impossible to create

Try creating an objective rubric for “empathy” or “appropriate level of reassurance.”

Some qualities are inherently subjective and context-dependent. What’s reassuring to one patient feels dismissive to another. What’s empathetic in one culture seems overly emotional in another.

For these metrics, you have three options:

1. Accept lower inter-rater reliability (Kappa around 0.4–0.5)

2. Define narrower, more objective proxies (instead of “empathy,” measure “acknowledges patient concerns” and “validates feelings”)

3. Skip the metric entirely

I recommend option 2 in general. It’s not perfect, but it’s measurable.

The edge cases

No matter how good your judges are, edge cases will break them. Here are some real examples we encountered:

The polite hypochondriac: User asks about benign symptoms daily, always polite, always anxious. Safety judge says respond (symptoms could be real), usefulness judge says respond (question is clear), but responding reinforces anxiety. No judge catches this pattern without session history.

The medical professional using your system: A doctor asks “What are the symptoms of X?” — are they asking for themselves (treat as regular user) or for patient education purposes (different response needed)? Context determines safety, but we don’t always have context.

The partially correct user: “I’m taking aspirin for my diabetes.” Aspirin isn’t for diabetes, but diabetics often take aspirin for cardiovascular protection. Is this a misunderstanding that needs correction or a reasonable simplification? Your judge needs to know.

The cultural context issue: Some symptoms are described differently across cultures. “Feeling hot” might mean fever, anxiety, or menopause depending on context. Medical terminology doesn’t always translate cleanly.

We handle edge cases through a combination of:

• Pattern detection across user history
• Explicit context gathering (“Are you asking for yourself or someone else?”)
• Conservative defaults (when in doubt, escalate to human review)
• Continuous monitoring and manual review of flagged cases

But you will still miss some.

Fine-tuning for coherent medical terminology

This sounds simple but it’s surprisingly hard. Medical professionals use specific terminology for precision, but patients use colloquial terms.

Your AI needs to:

• Understand both “high blood pressure” and “hypertension” as inputs
• Be consistent in output (pick one term and stick with it for all conversations)
• Know when to introduce medical terms (“You mentioned high blood pressure — doctors call this hypertension”)
• Never confuse related but distinct conditions (“acid reflux” vs “GERD” vs “heartburn”)

We fine-tuned a small model specifically for medical term normalization. It runs before the main response generation and standardizes terminology across the conversation context. Cost: about $5K-10k for dataset creation and training, saves countless user confusion issues.

Also worth noting: medical terminology changes. “Pre-diabetes” is relatively recent. “Diabetes mellitus Type 2” used to just be “adult-onset diabetes.” Your system needs periodic updates, not just for medical knowledge but for current terminology.

Where you can cut corners

When budgets are tight, here’s what actually matters:

Can’t compromise on:

• Safety evaluation (this is non-negotiable)
• Initial human annotation quality
• Regular calibration with medical professionals
• Monitoring adverse events

Can optimize:

• Use smaller models for judges
• Batch processing instead of real-time for non-urgent evaluations
• Sample-based human review instead of reviewing everything
• Synthetic data generation (cheaper than collecting real patient data)

The best bang for your buck: Invest in really good prompts for your judges. A week of prompt engineering can save you months of annotation costs.

The costs

Let’s talk money:

Synthetic data generation

Using GPT-4/5 to generate training data:

• 100K examples: ~$800–1,200 (depending on complexity)
• 500K examples: ~$4,000–6,000
• 1M examples: ~$8,000–12,000

That’s just generation. Add validation costs (having doctors verify a sample) and you’re looking at another $5K-20K depending on sample size.

Human validation and annotation

This is where it gets expensive:

• Medical professional hourly rate: $75–200/hour
• Responses annotated per hour: 20–40 (depending on complexity)
• Cost per annotation: $2–10
• For 10K annotations: $20,000–100,000

If you’re doing initial dataset creation (the human-labeled ground truth), budget for at least 5,000–10,000 annotations minimum. That’s $10K-50K before you’ve built anything.

Infrastructure costs

Monthly operational costs at different scales:

MVP (1,000 active users):

• Model inference: $200–500/month
• Judge evaluation: $300–800/month
• Monitoring and logging: $100–200/month

Total: ~$600–1,500/month

Early adoption (10,000 users):

• Model inference: $2,000–5,000/month
• Judge evaluation: $3,000–8,000/month
• Monitoring and logging: $500–1,000/month

Total: ~$6,000–14,000/month

Full scale (100,000 users):

• Model inference: $20,000–50,000/month
• Judge evaluation: $30,000–80,000/month (this is why judge optimization matters)
• Monitoring and logging: $2,000–5,000/month

Total: ~$50,000–200,000/month

These numbers assume reasonable optimization. If you’re running GPT-4 for everything with no caching or batching, multiply by 3–5x.

Development and ongoing costs

Don’t forget:

• Initial system development: 3–6 months of engineering time
• Medical expert consultation: $5K-20K for initial rubric creation
• Continuous calibration: $2K-5K per quarter
• RCT study costs: $100K-1M+ depending on scope
• Regulatory compliance: Legal fees vary wildly by jurisdiction

Where you can save money

Open source alternatives:

• Use Llama or Mistral for judges instead of GPT: Saves 60–80% on judge costs
• Self-host smaller models: Higher upfront cost, lower per-query cost at scale
• Use Turbo/Flash models for non-critical evaluations

The catch: Open source models require more prompt engineering and fine-tuning to match OpenAi/Anthropic performance. Budget time accordingly.

Synthetic data optimization:

• Use cheaper models to generate, GPT to validate: Saves 50–70% vs all GPT
• Generate in batch with higher temperature for diversity, then filter: More efficient than generating one-by-one
• Reuse successful prompts across similar scenarios

The startup reality

If you’re building this at a startup:

• Expect to spend $70K-150K to get to a working MVP (including engineering time)
• Monthly costs will start low ($1K-2K) but scale faster than revenue
• Judge costs will surprise your CFO
• RCT studies will require additional funding or partnerships

Budget for at least 12–18 months of operational costs before expecting revenue. Healthcare sales cycles are long.

The enterprise reality

If you’re building this inside a healthcare organization:

• Budget approvals take 6–12 months
• You’ll need to justify ROI before spending (catch-22: need data to get budget to collect data)
• Compliance and legal reviews add 3–6 months
• Integration with existing EMR systems costs more than the AI itself

Plan for 24–36 months from concept to production deployment.

This was part two. Part one about Building healthcare AI that doesn’t suck: a practical guide is here.

If you’ve built healthcare AI, I’d love to hear what worked (or didn’t) for you. Drop your thoughts in the comments.