Testing the Rubric Before You Test the Model
Calibration fixtures test a rubric before it grades any model. The dominant defect is double credit across the gate: here's what our fixture runs found.
All
AI 69
Geology 26
Benchmarking 24
Machine Learning 16
Evaluation 12
Agentic AI 10
Open Source 9
Reinforcement Learning 5
China 4
Complexity Theory 3
AI
LLM-as-a-Judge: Complete Guide to Using LLMs as Evaluators
How LLM judges work, single vs. pairwise judging, rubric design, bias sources, and how to validate a judge before trusting its scores.
Generating a 50-Question Benchmark With One Approval Step
A one-prompt authoring flow that stops once at the blueprint, runs four machine gates, and resumes from its own files. What it catches, and what it doesn't.
How to Benchmark AI Models: A Practical Framework for Reliable Results
Designing, running and interpreting AI model benchmarks, from question construction to statistical significance, so results actually hold up.
Benchmarking an Agent Harness With the Model Held Fixed
A leaderboard score belongs to a model and a harness together. Here's how pinning the model isolates the harness effect, and where that breaks down.
AI Agent Evaluation: Frameworks, Metrics, Benchmarks, and Best Practices
AI agent evaluation covers reasoning, tool use, trajectories and end-to-end outcomes: how it differs from model evaluation and how to build a framework.
Continual Learning: Three Approaches That Work
Continual learning without catastrophic forgetting: what fruit flies, LoRAs and differentiable plasticity show about models that learn after training.
Running Specialist AI Agents as Child Processes
Multi-agent orchestration in the Geocluster Research Harness: each specialist runs as its own child process, with credentials kept out of logged commands.
What Real Agents Taught Us About Designing MCP Tools
MCP tool design lessons from real agents: they ignored structuredContent, invented legend labels and misread 86 records as "1 item". Here's what we changed.
Detecting Map Layout and Legend Swatches on Scanned Geological Maps
Geological map legend extraction in Stratigraphic Amenity: a YOLOv10 detector finds the map parts, then swatches are paired to text and colours read.