The Katalin Trap: When Cherry-Picking Training Data Fails

Mon, 26 Oct 2026 22:00:00 +0800

Selecting only peak performances trains an agent for an exhausted world that ceases to exist

Key Takeaways

State of the art and curriculum learning: Conventional data selection self-improvement (such as Best-of-N and rejection sampling) treats top-performing trajectories as gold-standard supervision, assuming that curating peak reward functions like an effective curriculum learning filter in stationary environments.
The Katalin Trap and self-training collapse: In agentic non-stationary tasks, cherry-picking training data can cause niche overfitting and self-training collapse, as happened in the case of the “Katalin” experiment described below.. Because successful actions permanently consume the environment’s exploitable state, peak trajectories represent transient exploits from dead worlds rather than robust generalisation.
Non-parametric catastrophic forgetting: Unlike classical catastrophic forgetting caused by weight interference, the rise-and-collapse failure mode in our Negative Space Learning experiments occurs under frozen base models with fresh LoRA adapters, driven entirely by discarding recovery and exploration behaviours.
Sliding window training and online stability: Retaining recent trajectories regardless of score acts as a temporal anchor and biological respiratory rhythm. A sliding window training buffer preserves messy recovery behaviours and forces continual re-derivation, sustaining long-term online learning stability.

TL;DR

Filtering for peak performance seems like it should be an obvious choice in data selection for self-improvement, but cherry-picking training data in dynamic environments can causes catastrophic self-training collapse. In our Negative Space Learning experiments, an agent trained only on its highest-scoring runs (Katalin) suffered niche overfitting to depleted states, unlearning its gains within eight generations without weight-level catastrophic forgetting. Conversely, sliding window training across recent generations (Miri) discards stale strategies and preserves recovery behaviours, achieving superior online learning stability on four times less data.

Variance Deletion: Why Cherry-Picking Data Collapses Agents

Mon, 26 Oct 2026 00:00:00 +0800

Pruning sub-optimal runs deletes the recovery behaviours agents need when environments drift

Key Takeaways

State of the art and curriculum learning: Standard paradigms in data selection self-improvement (such as STaR, RAFT, and best-of-N rejection sampling) filter for top-scoring trajectories, treating peak curation like an automated curriculum learning filter that isolates high-reward behaviour while assuming the underlying task distribution remains stationary.
Mechanism of self-training collapse: In dynamic or agentic environments, cherry-picking training data can cause niche overfitting by deleting behavioural variance. Pruning sub-optimal, exploratory, and recovery runs starves the agent of recovery behaviours, transforming apparent quality filtering into catastrophic self-training collapse once the environment drifts.
Forgetting without weight interference: The parallel collapse observed between tool-trained shopping agents losing conversational competence and our internal self-training lineages is distributional rather than parametric. Because our Negative Space Learning experiments train fresh LoRA adapters from a frozen base with no inherited weights, forgetting here is driven entirely by data variance deletion rather than gradient overwriting.
Sliding window training and stability: Retaining a bounded buffer of recent trajectories regardless of score sustains online learning stability. Sliding window training preserves the full outcome distribution and failure-recovery paths, proving that long-term generalisation depends on distributional width rather than peak-performance cherry-picking.

TL;DR

Filtering for peak performance seems like intuitive data selection self-improvement, but cherry-picking training data deletes the behavioural variance needed for agent generalisation and online learning stability. In our self-training runs on disposable Linux containers, an agent trained only on its highest-scoring runs (Katalin) suffered self-training collapse from niche overfitting, mirroring how over-optimising tool use leads to conversational collapse without weight-level catastrophic forgetting. By contrast, sliding window training across recent generations preserves messy recovery trajectories, sustaining steady performance gains without data accumulation.

Sliding Window on Eigenform Continuous Learning Tool Tips

The Katalin Trap: When Cherry-Picking Training Data Fails

Key Takeaways

TL;DR

Variance Deletion: Why Cherry-Picking Data Collapses Agents

Key Takeaways

TL;DR