Hanad KubatVisit Website →
← Back to blog

Stop Regressions: AI Evaluation Metrics for Production LLMs

September 1, 2026
Stop Regressions: AI Evaluation Metrics for Production LLMs

The production baseline looks like this: deterministic checks (exact match, schema validation) gating every pull request, embedding similarity catching paraphrase drift, and rubric or LLM-as-a-judge scoring covering open-ended quality on a schedule, not every commit. Agentic systems add a fourth layer: trajectory metrics that grade the plan and tool calls, not just the final answer. Everything below maps each metric family to the task it actually fits, then covers calibration, cost, and the checklist that keeps the whole thing from rotting in three months.

TL;DR:

  • Use deterministic, schema, or exact match metrics for quick regression checks on every pull request to catch basic errors early.
  • Rely on semantic similarity or LLM-based judgement for open-ended outputs, but always calibrate and validate judges with human labels regularly.
  • Apply precision, recall, and confusion matrices for classification tasks, especially to detect rare class issues ignored by overall accuracy.
  • For retrieval-augmented systems, measure retrieval relevance with precision@k and recall@k, and assess answer faithfulness with entailment or source-based checks.
  • Build evaluation pipelines with fixed update schedules, versioned components, and clear decision thresholds based on downstream error costs.

Table of Contents

What AI Evaluation Metrics Actually Measure

A metric tells you what to measure. A scorer tells you how you compute it. Conflating the two is the first mistake I see in evaluation stacks that quietly fail: someone picks "accuracy" as the metric, then implements it as exact string match against a single reference answer, and wonders why a paraphrase that's factually correct scores zero.

Metric choice has to match what you're actually trying to catch. There are three distinct jobs a metric can do, and they pull in different directions:

  • Regression detection: did the last code or prompt change break something that used to work? This wants fast, deterministic, low-variance scorers running on every pull request.
  • A/B comparison: is variant B actually better than variant A, or is the difference noise? This wants statistical rigor, not a single point estimate.
  • Production monitoring and risk control: is the system degrading in the wild, on inputs your test set never saw? This wants sampling, drift alerts, and human review on the highest-risk slice.

The classic failure mode is optimizing a single aggregate number while a subgroup silently collapses. A binary classifier can hit 95% accuracy on a dataset that's 95% negative class by predicting "negative" every time. BLEU has the equivalent blind spot for generation: it rewards n-gram overlap with a reference string, so a fluent, correct answer phrased differently than the reference gets penalized, while a garbled sentence that happens to share four-word chunks with the reference scores well. Neither number is wrong. Both are answering a narrower question than the one you're actually asking.

Deterministic, Semantic, and Model-Based Scoring: Choosing an Implementation

Every evaluation stack is built from three implementation families, and the trade-off between them is speed and reproducibility on one end versus semantic sensitivity on the other.

Deterministic scorers compute something exact: string match, token-level F1, JSON schema validation, regex extraction. They're near-instant, cost nothing per run, and give identical results every time you run them. Their weakness is rigidity. A deterministic scorer checking for the substring "yes" will miss "certainly" and "that's correct" even though a human would score all three identically.

Embedding and semantic-similarity scorers close part of that gap. BERTScore and cosine similarity over sentence embeddings measure meaning rather than surface tokens, so paraphrases score well even with zero word overlap with the reference. The cost is a dependency you now have to manage: the embedding model itself has version drift, and a similarity threshold tuned against one embedding model doesn't transfer cleanly to another.

LLM-as-a-judge scoring hands the evaluation to another language model, usually against a rubric or through structured approaches like G-Eval or QAG (Question Answer Generation) and its close relative DAG (Directed Acyclic Graph) decomposition. This is the most flexible option and the only one that reasonably handles genuinely open-ended output: tone, coherence, whether an answer actually resolves the user's intent. It's also the most expensive per evaluation, and it introduces a new failure surface: judge variance, meaning the same input can get a different score on a different run unless the judge is calibrated and its prompt is frozen.

One more axis matters as much as the implementation family: whether the metric needs a reference answer at all. Reference-based metrics (BLEU, ROUGE, token F1 against a gold answer) require someone to have written that gold answer, which caps how much of your traffic you can ever evaluate this way. Reference-free metrics, including entailment and factuality checks and QA/QG-based approaches, score outputs against source context instead of a fixed answer, which is what makes them workable at production scale for summarization and RAG.

Pro Tip: Never let a judge model grade its own outputs on the same prompt family it was fine-tuned or heavily prompted with. Use a different model, or at minimum a materially different prompt, as the judge. Self-grading inflates scores in ways that are hard to catch until a human spot-check exposes it.

Metric Families Mapped to Task Type

Pick the metric family based on what kind of system you're shipping, not based on which metric is easiest to compute. Here's the mapping I use, task by task.

1. Classification systems

For anything outputting a label, category, or discrete class, the core metrics are accuracy, precision, recall, F1, and AUC-ROC. Accuracy alone is close to useless on imbalanced data. What actually tells you something is per-class precision and recall, laid out in a confusion matrix, so you can see exactly which class the model confuses with which. A spam filter that's 98% accurate but misses 40% of a rare fraud subclass has a real problem accuracy hides completely. AUC-ROC is worth tracking separately because it measures ranking quality across every possible decision threshold, which matters when you expect to tune that threshold later for a different cost trade-off between false positives and false negatives.

2. Generation and summarization

This is where the metric conversation has shifted the most. Overlap metrics like BLEU and ROUGE are fast to compute and fully reproducible, which is why they still show up in CI as a cheap regression tripwire. But they measure lexical overlap with a reference, not correctness or fluency, and they systematically punish valid paraphrases. BERTScore and MoverScore push toward semantic similarity instead of surface overlap and correlate much better with human judgment on open-ended text. Token F1 still earns its place for short, structured spans, like extracted entities or a specific numeric answer, where the answer space is narrow enough that overlap and correctness converge. For anything genuinely open-ended, judged quality, coherence, and instruction-following need rubric or LLM-based scoring. A recent practical evaluation framework is explicit that no single metric here is a silver bullet and recommends a balanced suite rather than a single number to optimize.

3. Retrieval and RAG pipelines

Retrieval-augmented generation splits cleanly into two questions: did you retrieve the right context, and did the model actually use it faithfully? The retrieval half is measured with precision@k and recall@k (of the top-k chunks returned, how many were relevant, and how many relevant chunks did you actually surface), MRR (mean reciprocal rank, rewarding the first relevant result appearing early), and NDCG (normalized discounted cumulative gain, which accounts for the full ranked order rather than just presence in the top k).

The generation half needs context precision and faithfulness, sometimes called groundedness: does the answer actually follow from the retrieved context, or did the model hallucinate something plausible-sounding that isn't in any retrieved chunk? These are two genuinely different failure modes and deserve two different metrics rather than one blended score. A system can retrieve perfectly relevant chunks and still hallucinate on top of them, and a faithfulness score is the only thing that catches that. Reference-free, entailment-based checks are particularly useful here because you rarely have a gold "correct answer" for every possible RAG query, only the source documents.

3. Retrieval and RAG pipelines — overview diagram

4. Agentic systems

Agents fail in the plan, not just the output, so scoring only the final result throws away most of the diagnostic signal. Layered agentic evaluation tracks plan quality, tool-selection correctness, execution or trajectory metrics, and final task completion as separate scores, because an agent that picks the right tool but constructs the wrong parameters fails differently than one that picks the wrong tool entirely, and those two error types belong to different teams to fix. Task completion tells you whether the agent got there. Tool selection quality tells you whether it used the right instrument. Reasoning coherence, usually judged against the full trace rather than the summary, tells you whether the steps in between actually make sense or the agent got lucky.

Building Reproducible, Calibrated Evaluation Pipelines

Metrics are only as trustworthy as the pipeline that runs them. Three implementation decisions determine whether your evaluation numbers mean anything six months from now.

Cadence first. Run deterministic checks, schema validation, exact match, token F1, on every pull request. They're cheap enough to gate merges without slowing anyone down. Push semantic-similarity sweeps to a nightly job, and reserve full rubric or LLM-as-a-judge evaluation for a weekly deep pass, plus any release that touches the prompt or the model version directly. Combining cheap deterministic checks with deeper periodic sweeps, adding human validation on the highest-risk slice, is the pattern most production teams converge on, and it's the one I default to on client builds.

Statistical rigor when comparing variants. A single run of variant A scoring higher than variant B is not evidence of anything. Treat model outputs as samples rather than fixed values, and use bootstrap confidence intervals or a paired statistical test before declaring a winner. Store the random seed and the environment version alongside the result, or the "significant" difference you found on Tuesday won't reproduce on Thursday.

Judge calibration is not optional. LLM-as-a-judge scoring aligns well with human judgment on open-ended tasks, but only once the judge has been validated against a human-labeled sample and its prompt is frozen. An unfrozen judge prompt is a moving target: you can't tell if a quality score dropped because the model got worse or because someone edited the rubric wording last Tuesday. Re-validate the judge against a fresh human-labeled batch on any model or prompt change, and version the judge prompt the same way you'd version application code.

Cost and latency decide where each check lives. A 200-millisecond deterministic check belongs inline, in CI, blocking a merge. A rubric pass that calls an LLM judge for every one of ten thousand test cases does not; run it offline, on a sample, and reserve the full sweep for release candidates.

Roughly a third of production LLM evaluation failures I've seen trace back to a judge prompt that quietly drifted after a "small wording fix," never re-validated against the original human-labeled set.

Building Reproducible, Calibrated Evaluation Pipelines — overview diagram

Building a Minimal, Production-Ready Evaluation Stack

Match the stack to the system, not the other way around.

  1. Classification systems: per-class precision, recall, F1, and AUC-ROC computed on every merge, with a confusion matrix reviewed weekly. Set a per-class recall floor, not just an overall accuracy floor, so a rare-class collapse trips an alert instead of hiding inside a healthy aggregate.
  2. RAG systems: precision@k and recall@k on retrieval, faithfulness and context precision on generation, run nightly against a held-out query set. Convert faithfulness below a set threshold into a hard block on deployment, not a warning that gets ignored.
  3. Code generation: deterministic execution tests (does the code run, do the unit tests pass) on every PR, plus a weekly rubric pass judging readability and adherence to house conventions, which no test suite catches.
  4. Agentic systems: task completion and tool-selection correctness logged on every trace, reasoning coherence sampled and rubric-scored weekly, with any trajectory that fails tool selection routed to a review queue instead of silently retried.

Pro Tip: Pick your threshold from the cost of the error, not from what number looks respectable. A faithfulness score of 0.85 might be fine for an internal documentation assistant and dangerous for a system answering medical dosage questions. Set the guardrail to the downstream cost of being wrong, not to an industry-average benchmark.

Every metric in this article has a known failure mode, and pretending otherwise is how evaluation stacks quietly stop meaning anything.

  • BLEU and ROUGE penalize valid paraphrases that share no n-grams with the reference; swap in BERTScore or a rubric for anything open-ended.
  • Headline accuracy hides class imbalance completely; always pair it with per-class precision, recall, and a confusion matrix.
  • LLM judges drift when their prompt changes or the underlying model gets updated; freeze the prompt, pin the model version, and re-validate against human labels on a schedule.
  • A small or narrow reference set makes reference-based metrics brittle against legitimate variation; diversify references or move to reference-free, context-based scoring instead.
  • Chasing a single metric upward eventually optimizes the test set instead of the actual task. Rotate in fresh evaluation samples the model has never been tuned against.

How I Structure Evaluation in Client Builds

On the production systems I build, the pattern stays the same regardless of the client: deterministic checks gate every merge, a nightly job runs semantic similarity across a broader sample, and a weekly rubric pass samples the highest-risk queries for human review. Nothing exotic. It just has to run every week without someone remembering to trigger it.

The checklist I actually use: version every judge prompt like application code, store the raw input, output, and retrieved context for every evaluated case (not just the score), track per-class metrics instead of one aggregate, and alert on drift rather than waiting for a quarterly review to notice. If you're building out compliance documentation alongside this, the same versioning discipline shows up in my AI audit checklist, and if the system touches EU users, the metric documentation doubles as half the paperwork covered in my EU AI Act compliance checklist. When a client's evaluation relies on a third-party model or scoring API, I run the same due diligence I describe in assessing AI vendor risk before trusting its numbers.

Metrics as Code: What I Won't Compromise On

I treat metrics the way I treat tests: small in number, versioned in source control, and reviewed the same way a pull request is reviewed. A metric that isn't versioned isn't a metric. It's a number someone can silently redefine.

Every metric I add gets one sentence next to it in the codebase explaining what decision it supports and what action a drop in that number should trigger. If nobody can answer "what do we do when this number falls," the metric isn't ready to ship, no matter how principled it sounds in a design doc.

The instinct to keep adding metrics because more visibility feels safer is worth resisting. Once a metric stops changing anyone's decision, it's dashboard noise, and dashboard noise is how real regressions get lost in a wall of green.

— Hanad Kubat

Getting a Working Evaluation Stack Without an Agency Contract

Most teams that need this end up with two bad options: build it themselves over several sprints they didn't budget for, or hire an agency that hands off a dashboard nobody on the team can maintain past the handoff call. Hanad Kubat is the alternative: one senior engineer, fixed price, scope frozen at kickoff, building the evaluation pipeline, RAG scoring, or agentic trajectory checks your production system actually needs, and handing you code you fully own from the first commit.

Hanad Kubat

The work fits the pattern in this article directly: deterministic CI gates, calibrated LLM-as-a-judge scoring, RAG faithfulness checks, production monitoring, all delivered in weeks, not months, with no juniors and no surprise invoices. If your prototype already exists but the evaluation layer is missing or held together with a spreadsheet, the fixed-price Prototype Audit (€1,500, credited against the build) is the fastest way to find out exactly what's missing before committing to a full build. Full builds start at €12,000, and every engagement ships yours milestone by milestone. Visit Hanadkubat to book an audit or start a build.

Sources

For deeper implementation detail beyond what fits here: the Comet guide to LLM-as-a-judge covers calibration in more depth, the arXiv practical evaluation framework walks through dataset curation, Microsoft's generative AI evaluation metric list catalogs reference-free options, and the Hugging Face Evaluate library gives you versioned, reproducible metric code instead of reimplementing BLEU from scratch. For structured-data checks that feed retrieval pipelines, BabyLoveGrowth's structured data audit tool is worth a look.

FAQ

What Are AI Performance Metrics?

AI performance metrics are quantitative measures, like accuracy, F1, faithfulness, or task completion, used to judge whether a model or system does its job correctly, consistently, and safely enough to ship.

What Are Some Examples of Evaluation Metrics?

Common examples include accuracy, precision, recall, and F1 for classification; BLEU, ROUGE, and BERTScore for generation; precision@k, recall@k, and NDCG for retrieval; and task completion or tool-selection accuracy for agentic systems.

How Do You Evaluate AI Accuracy?

Compute accuracy alongside per-class precision and recall, since accuracy alone hides class imbalance; for generative or agentic systems, pair a deterministic check with semantic similarity and periodic rubric or LLM-as-a-judge scoring for a fuller picture.

What Are the 7 Levels of AI?

There's no single agreed-upon set number of AI capability levels in the technical literature; most capability tiers referenced informally range from reactive rule-based systems through narrow task-specific AI to theoretical general and superintelligent AI, none of which are standardized evaluation categories.

How Do I Know Which Metric to Trust for My System?

Match the metric to the task and the cost of being wrong: deterministic checks for regressions, embedding similarity for paraphrase tolerance, and a calibrated LLM-as-a-judge or rubric score for open-ended quality, validated against a human-labeled sample before you trust it in production.