An LLM Judge That Can't Invent a Score
AI model evaluation with an LLM judge: the judge returns a JSON block the harness checks against the rubric, and an unscored answer beats a guessed one.
All
AI 69
Geology 26
Benchmarking 24
Machine Learning 16
Evaluation 12
Agentic AI 10
Open Source 9
Reinforcement Learning 5
China 4
Complexity Theory 3
Benchmarking
Position Bias in Pairwise LLM Evaluation
Asking the judge twice with answers swapped exposes position bias. The scoring table, worked examples, and why pairwise verdicts can't build a leaderboard.
Testing the Rubric Before You Test the Model
Calibration fixtures test a rubric before it grades any model. The dominant defect is double credit across the gate: here's what our fixture runs found.
LLM-as-a-Judge: Complete Guide to Using LLMs as Evaluators
How LLM judges work, single vs. pairwise judging, rubric design, bias sources, and how to validate a judge before trusting its scores.
Generating a 50-Question Benchmark With One Approval Step
A one-prompt authoring flow that stops once at the blueprint, runs four machine gates, and resumes from its own files. What it catches, and what it doesn't.
How to Benchmark AI Models: A Practical Framework for Reliable Results
Designing, running and interpreting AI model benchmarks, from question construction to statistical significance, so results actually hold up.
Benchmarking an Agent Harness With the Model Held Fixed
A leaderboard score belongs to a model and a harness together. Here's how pinning the model isolates the harness effect, and where that breaks down.
AI Agent Evaluation: Frameworks, Metrics, Benchmarks, and Best Practices
AI agent evaluation covers reasoning, tool use, trajectories and end-to-end outcomes: how it differs from model evaluation and how to build a framework.
Your AI Agent Can Read Its Own Answer Key
Agents with file and shell tools can reach the grading rubric on disk. What we measured across 50 questions, which fixes failed, and what the harness does now.
Which AI Model Is Best for Your Geology?
Best AI for Geology Testing. See how Groundtruth's dynamic benchmark tests models on your own data, comparing accuracy, cost, and performance.