Research

A fixed benchmark tells you what a model can do — not what it just learned from your corpus.

A benchmark harness that measures what a fine-tuned model actually learned from a specific corpus, not what it can do in general. Rather than a fixed question set, it ships a pipeline that reads an authenticated source corpus, extracts testable concepts, and writes questions with source-cited grading rubrics — then has an independent LLM judge score answers pointwise or head-to-head, with position-bias correction and contamination auditing. The v1.0.0 release includes a runnable geology edition built on 34 openly licensed mineral-deposit records from the Yudnamutana Copper district, South Australia, plus the corpus-to-rubric authoring pipeline itself, so the same method transfers to any domain.

Take away the reward function, and reward-hacking has nothing left to grip.

A proof-of-concept self-training architecture that removes reward functions entirely. Candidate behaviours run under real resource constraints, and only those whose effects on the environment persist — and leave future interaction possible — are propagated; because the environment returns no semantic feedback, there is nothing for proxy optimisation or reward-hacking to attach to. The paper characterises the resulting dynamics, termed negative-space learning: improvement by consolidation and pruning of strategies that simply keep working, with models developing meta-learning tactics unprompted.

Jennifer Dodgson, Alfath Daryl Alhajir, Michael Joedhitya, +6 more

How easily a machine learns the primes becomes a way to measure the order hiding in them.

Proposes machine-learning accuracy as an experimental instrument for number theory. An image-based model is trained to classify primes in blocks taken from different regions of an Ulam spiral; models trained near 500 million outperform those trained below 25 million, indicating more learnable structure at higher magnitudes, while precision/recall splits suggest the model switches strategy by scale — spotting prime patterns at low numbers, eliminating composites at high ones. The findings track conjectures that local randomness in prime distribution regularises after scaling by log x, and the method may extend to the strong and weak primes that matter for cryptography.

Jennifer Dodgson, Michael Joedhitya, Adith Ramdas, +5 more

One unbounded, ungameable number is enough supervision for a model to start teaching itself.

A framework for AI systems that generate and validate new knowledge by acting on their environment instead of consuming more human-curated data. Learning is steered by an unbounded, ungameable numeric signal — annexed disk space, follower count — with agents writing and running code to move that number and successful runs becoming training data for the next iteration. Empirical validation replaces textual similarity to guard against model collapse and the warm-start problem, with fine-tuning via GRPO and modular agents split across environment analysis, strategy, and code synthesis. It is the earliest statement of the research arc later formalised in “Survival is the Only Reward.”

Alfath Daryl Alhajir, Jennifer Dodgson, Joseph Lim, +4 more