Human evaluation doesn’t scale. At 1,000 LLM outputs per day, you need reviewers working in shifts — and after 50 annotations, accuracy plummets. Fatigue. Inconsistency. Spiraling cost.
You automate. GPT-4 returns scores — faithfulness: 0.87, relevance: 0.92. You build dashboards, set thresholds, make decisions.
Three months later, every score is inflated 15%. Longer responses always win, regardless of quality. And upgrading GPT-4 to GPT-4o shifted all scores up three points — not because your model improved, but because the judge got more lenient.
LLM-as-Judge works — but only calibrated. Here is how.
Choosing and Calibrating Your Judge
GPT-4 is the most widely used LLM judge — its agreement with human annotators exceeds 80% on MT-Bench and 85%+ on G-Eval for factual and instruction-following tasks. But “widely used” doesn’t mean “correct out of the box.”
The calibration process:
- Collect 50 human-annotated examples. Real outputs from your production system. Real human judgments — correct/incorrect, faithful/unfaithful, helpful/unhelpful.
- Run the same 50 examples through your judge model with your evaluation prompt.
- Calculate the correlation between judge scores and human scores. Pearson or Spearman — either works as long as you’re consistent.
- If correlation is below 0.75 for any rubric, that rubric’s judge prompt needs work — or the judge model itself isn’t suitable for that dimension.
A correlation of 0.75 means the judge and humans agree on the direction of quality (better/worse) 75% of the time. Below that, you’re making decisions based on noise. For high-stakes evaluation — production gatekeeping, model selection, SLA compliance — calibrate to 0.85+.
Since every judge model call is an API call with a cost attached, applying API cost optimization techniques to your evaluation pipeline can reduce judge expenses without sacrificing calibration quality.
Three Bias Types and How to Fix Them
Position Bias
The judge prefers whichever response appears first. Swap the order of two identical-quality responses, and scores shift 5-10%. This is the most well-documented bias in LLM evaluation.
Fix: Randomize response order for every evaluation. If comparing two outputs, flip a coin to determine which appears first. If scoring a single output, occasionally insert a dummy low-quality response before the real one to calibrate.
Verbosity Bias
The judge scores longer responses higher — regardless of whether the extra length adds value. A 300-word answer with irrelevant padding outscores a concise 80-word answer that perfectly addresses the query.
Fix: Score relevance and completeness as separate dimensions. Tell the judge explicitly: “Evaluate whether the response answers the query. Do not reward length. A concise correct answer should score higher than a verbose partially-correct answer.” Then validate: do your judge scores still correlate with response length? If so, the bias persists — strengthen the instruction.
Self-Enhancement Bias
The judge inflates scores for outputs generated by the same model family. GPT-4 judging GPT-4 outputs gives higher scores than Claude judging the same outputs. The bias is 5-10% — enough to make cross-model comparisons unreliable.
Fix: Use a different model family as judge than you use in production. If your production stack runs Claude, evaluate with GPT-4. If you use multiple models in production, pick a judge model that isn’t in your routing pool. Or use a dedicated judge model trained specifically for evaluation tasks.
G-Eval: Chain-of-Thought Scoring
Instead of “rate the response from 1-5,” G-Eval asks the judge to think before scoring. The evaluation prompt includes:
- The evaluation criteria with precise definitions
- A chain-of-thought instruction: “First, analyze whether each claim in the response is supported by the context. Second, check if the response addresses all parts of the query. Third, identify any missing or incorrect information.”
- The scoring scale with anchor examples
- The final score output format
G_EVAL_PROMPT = """
Evaluate the following response based on FAITHFULNESS.
Definition: Faithfulness measures whether every factual claim in the response
is supported by the provided context. A response is faithful if it makes no
claims that contradict or extend beyond the context.
Steps:
1. List every factual claim in the response.
2. For each claim, check whether it appears in the context.
3. Flag any claims that are not supported by the context.
4. Assign a score: 5 = all claims supported, 1 = no claims supported.
Context: {context}
Response: {response}
Score (1-5) and justification:
"""G-Eval’s chain-of-thought structure produces scores that correlate more strongly with human judgments than direct scoring — typically improving judge-human agreement by 3-8 points. The cost: roughly 2× the tokens of a direct scoring prompt. For high-stakes evaluation, it’s worth it. For the provider-specific prompt formatting rules that make judge instructions more effective, consult the prompt engineering cheat sheet.
Production Judge Pipeline
- Batch scoring. Run evaluation asynchronously. Users shouldn’t wait for quality scores. Generate the response → return it to the user → score it in the background → attach the score to the trace span.
- Score storage. Write scores to your observability platform, not a separate database. Scores should live alongside the traces they evaluate — same platform, same query interface. This is what our observability guide calls eval-as-span-attachment.
- Trend monitoring. Watch scores over rolling windows. A 2-5 point sustained drop across any rubric triggers an alert. Individual score fluctuations are noise. Sustained shifts are signal.
- Judge version pinning. Lock the judge model version. When you upgrade the judge, recalibrate against your 50 human-annotated examples. Establish a score mapping between old and new judges. Without this, your historical score data becomes meaningless.
- Recalibration trigger. Recalibrate when: judge model changes, evaluation rubric definitions change, or production data distribution shifts significantly (new use cases, new user populations). For the infrastructure that hosts your evaluation pipeline, our production optimization guide covers scaling, monitoring, and cost management. For securing the API keys that power your judge model calls, follow standard API security best practices for key rotation and access controls.
Judge Failures That Produce False Confidence
A calibrated judge isn’t a bulletproof judge. These three failure modes have shipped to production — learn from them before you repeat them.
The Lenient Judge. A fintech team calibrated their GPT-4 judge against 50 human-annotated examples and hit 0.91 correlation. They shipped. Three months later, an audit revealed the judge was awarding 4/5 to outputs containing hallucinated account numbers. Why? Their calibration set contained only “medium-quality” outputs — nothing truly bad. The judge learned to score on a compressed scale where “mediocre” was a 3 and “slightly above mediocre” was a 5. When real failures appeared in production, the judge couldn’t distinguish them from mediocre outputs because it had never seen genuine failures during calibration.
Fix: your calibration set must include at least 20% unambiguous failures — responses that are clearly wrong, harmful, or nonsensical. If the judge gives any of them above a 2/5, it’s not calibrated.
The Drifting Judge. An enterprise SaaS team ran LLM-as-Judge for six months without recalibration. Scores held steady at 4.2/5 across all rubrics. Management celebrated consistent quality. Then a customer reported a critical factual error in a production response — an error the judge had scored 4/5. Investigation showed the data distribution had shifted: the product launched a new feature, user queries changed, and the judge’s calibration — built on the old query mix — had silently become irrelevant. The scores looked stable because the judge was consistently wrong about a new class of queries it had never been calibrated for.
Fix: run distribution checks monthly. Compare the topic mix, query length, and task type of current production traffic against your calibration set. A distribution shift of more than 15% on any axis triggers recalibration.
The Overfitted Prompt. A startup spent two weeks iterating their G-Eval prompt against their 50-example calibration set. They tuned the exact wording of evaluation criteria, refined the scoring anchors, adjusted the chain-of-thought steps. Correlation hit 0.88. The prompt shipped. On production data, judge-human agreement dropped to 0.61. The prompt had overfitted — it was optimized to score those 50 examples correctly, memorizing their patterns rather than learning generalizable evaluation criteria. The prompt’s chain-of-thought referenced artifact patterns (“when the response begins with a list, check item count”) that were specific to the calibration set but irrelevant to real production outputs.
Fix: split your 50 examples into a 30-example training set and a 20-example held-out set. Iterate your judge prompt against the training set only. Validate against the held-out set. If training-set correlation keeps climbing but held-out correlation flatlines or drops, you’re overfitting. Stop iterating.
For integrating evaluation gates into your automated CI/CD pipeline, see our LLM testing and CI/CD integration guide.
Further reading. LLM-as-judge is one evaluation strategy — the full evaluation toolkit spans automated testing, CI/CD integration, and regression detection. Pair judge evaluations with the provider-specific prompt engineering reference for the prompting rules that produce the most reliable judgments across each model family.
For production deployment patterns that incorporate judge evaluations, see our production optimization guide.
FAQ
How big is the gap between LLM judge and human evaluation?
On factual tasks (faithfulness, instruction following, format compliance): LLM judges match or exceed individual human annotators — they eliminate fatigue and inconsistency, and agreement exceeds 80%. On subjective tasks (creativity, stylistic quality, humor): the gap is larger, 15-25 points. LLM judges are reliable for objective quality dimensions. For subjective dimensions, use them as directional signals, not gatekeepers. Human preference benchmarks like LMSYS Chatbot Arena provide the independent ground-truth data against which judge calibration should be validated.
GPT-4 or GPT-5.5 as judge?
GPT-4 is sufficient for most evaluation tasks and costs less. GPT-5.5 provides marginally better calibration on complex reasoning tasks but costs 3-5× more per evaluation. Start with GPT-4. Upgrade to GPT-5.5 only if calibration against human labels shows GPT-4’s correlation is below your threshold and GPT-5.5’s correlation is above it.
Can one judge evaluate all types of outputs?
A single judge can handle multiple rubric dimensions — faithfulness, relevance, completeness — as long as each dimension has a clear definition and calibration data. But a judge calibrated for factual Q&A may not be reliable for creative writing evaluation. If your product has fundamentally different output types, use separate judge configurations with separate calibration.
Judge model upgraded — can I still compare historical scores?
Not directly. Always recalibrate after a judge upgrade: run both old and new judges on your 50 human-annotated examples, establish the score mapping, and apply it to historical data. Without this step, you cannot tell whether your production quality changed or your judge’s scoring behavior changed. For current model availability and lifecycle timelines that affect judge model selection, see the models page.
How often should I recalibrate my judge model?
Recalibrate on a fixed schedule, not when something breaks. Monthly is the default. Weekly if your data distribution is volatile — new product features, seasonal user behavior shifts, rapidly expanding use cases. The calibration process takes under an hour once your 50-example annotation pipeline is built. At monthly recalibration, you’ll catch drift before it produces six months of misleading scores. The teams that skip scheduled recalibration are the ones who discover their judge was wrong through a customer complaint — which is the most expensive detection mechanism available.
For the complete production stack that evaluation plugs into — from routing to observability — our ultimate LLM API stack guide covers the full toolchain.
LLM-as-Judge is the most cost-effective way to evaluate LLM outputs at scale. But an uncalibrated judge is worse than no judge at all — it produces numbers that look precise and are systematically wrong, giving you false confidence in decisions based on them.
Calibrate against human labels. Mitigate the three biases. Pin the judge version. Recalibrate on upgrade. Fifty annotated examples and an afternoon of work is the difference between evaluation you can trust and evaluation that’s actively misleading.