Variance Deletion: Why Cherry-Picking Data Collapses Agents

Key Takeaways

• State of the art and curriculum learning: Standard paradigms in data selection self-improvement (such as STaR, RAFT, and best-of-N rejection sampling) filter for top-scoring trajectories, treating peak curation like an automated curriculum learning filter that isolates high-reward behaviour while assuming the underlying task distribution remains stationary.
• Mechanism of self-training collapse: In dynamic or agentic environments, cherry-picking training data can cause niche overfitting by deleting behavioural variance. Pruning sub-optimal, exploratory, and recovery runs starves the agent of recovery behaviours, transforming apparent quality filtering into catastrophic self-training collapse once the environment drifts.
• Forgetting without weight interference: The parallel collapse observed between tool-trained shopping agents losing conversational competence and our internal self-training lineages is distributional rather than parametric. Because our Negative Space Learning experiments train fresh LoRA adapters from a frozen base with no inherited weights, forgetting here is driven entirely by data variance deletion rather than gradient overwriting.
• Sliding window training and stability: Retaining a bounded buffer of recent trajectories regardless of score sustains online learning stability. Sliding window training preserves the full outcome distribution and failure-recovery paths, proving that long-term generalisation depends on distributional width rather than peak-performance cherry-picking.

TL;DR

Filtering for peak performance seems like intuitive data selection self-improvement, but cherry-picking training data deletes the behavioural variance needed for agent generalisation and online learning stability. In our self-training runs on disposable Linux containers, an agent trained only on its highest-scoring runs (Katalin) suffered self-training collapse from niche overfitting, mirroring how over-optimising tool use leads to conversational collapse without weight-level catastrophic forgetting. By contrast, sliding window training across recent generations preserves messy recovery trajectories, sustaining steady performance gains without data accumulation.

The Search Question

How does over-optimization for tool use cause agents to forget how to chat?

If you’ve trained agents, you’ve probably seen this. You fine-tune on tool-call trajectories (a shopping API, a search tool, a code interpreter) and the agent gets measurably better at calling the tool. Then you ask it a plain question and it responds with a malformed function call. The chat capability didn’t erode gradually; it collapsed. Param Madan and colleagues documented exactly this pattern in a shopping agent trained through SFT and RL, where tool-call proficiency rose while conversational competence fell off a cliff.

The standard explanation is catastrophic forgetting: the weights shifted, the old distribution is gone, what did you expect. But that explanation doesn’t quite fit. The agent isn’t forgetting an old task; it’s forgetting a mode of behaviour that was never in tension with the new one. Chatting and calling a tool aren’t competing objectives. Something more specific is happening, and it has to do with which training data you kept.

Here’s the parallel. In our Negative Space Learning experiments, three lineages of the same Qwen 2.5 7B base model were trained on self-generated disk-cleanup trajectories. One lineage, Miri, trained on the last three generations of data regardless of performance, and improved steadily. Another, Katalin, trained only on the three highest-performing generations, cherry-picked for peak success, and collapsed. Not plateaued. Collapsed. The agent overfit to a niche that stopped existing the moment training moved on.

The question this post answers: is the collapse Madan observed in tool-trained agents the same failure mode as Katalin’s niche overfitting in self-training?

Our thesis is yes. The shared mechanism is variance deletion. When you select training data for peak performance (whether that’s the best tool-call trajectories or the highest-scoring cleanup runs), you’re not just filtering out noise. You’re filtering out the full distribution of outcomes the agent needs to see in order to generalise. You’re teaching it that the world is narrower than it is. And when the world turns out to be wider, the agent has no recovery behaviour to fall back on, because you deleted every trajectory where it had to recover.

The rest of this post traces that mechanism across two settings that look different on the surface and share a structure underneath.

Assumptions & Nuance

Assumption 1: Best-of selection is the foundation of self-improvement

When practitioners rely on cherry-picking training data, STaR keeps only the rationales that produce correct answers. RAFT ranks candidate responses with a reward model and fine-tunes on the top slice. RLHF optimises against a preference model that is, itself, a learned cherry-picker. Rejection sampling, best-of-N, DPO: the entire self-improvement toolkit is built on the same move: generate broadly, select narrowly, train on the survivors.

And it works. It works well enough that “filter for quality” has become the default reflex. When self-training collapses, the standard response is to assume the filter wasn’t good enough: you need a better reward model, a stricter threshold, more diversity constraints.

Katalin and Miri test this reflex directly. Both lineages start from the same Qwen 2.5 7B base, generate data in the same environment, and fine-tune with the same recipe. The only difference is the selection rule. Katalin keeps the three highest-performing generations: peak success, by definition. Miri keeps the three most recent generations, regardless of how well they scored.

Katalin climbs fast, then collapses. By v13, her composite improvement score has gone negative. Miri climbs slower, but steadily, and sustains it. The selection rule (best-of versus last-of) is the entire difference between a lineage that degenerates and one that doesn’t.

The lesson: “best” means “narrowest.” When you select for peak performance, you’re not selecting for the strategies that generalise. You’re selecting for the strategies that happened to exploit the specific state of the environment at the moment of measurement. The highest-scoring run is often a high-variance lucky exploit: the agent that guessed the password on the first try. Train exclusively on that, and you’re teaching the model that the password is always the same.

Assumption 2: Tool-call collapse is catastrophic forgetting at the weight level

When Madan’s shopping agent forgot how to chat after tool-training, the diagnosis was catastrophic forgetting: new gradients overwrote old representations, the chat manifold got clobbered by the tool-call manifold, and the weights drifted away from the region that supported conversation. The fix, in this framing, is rehearsal buffers, KL anchoring, or elastic weight consolidation: techniques that pin the weights to protect old skills.

This diagnosis doesn’t fit the evidence. The NSL experiments train each generation as a fresh LoRA on the same frozen base model. There is no weight inheritance between versions. LoRA v6 is not stacked on LoRA v5; it’s trained from scratch. If the collapse were weight interference, it couldn’t happen in this architecture: there are no prior weights to interfere with.

And yet Katalin collapses. The failure mode travels through the data, not the weights.

Here’s the reframe. Selecting only the highest-performing trajectories prunes away the behavioural variance (the sub-optimal, diverse, exploratory runs) that sustains general capability. The “junk” trajectories, the ones where the agent tried something weird and partially failed, are carrying information about the shape of the problem space. They’re the model’s exposure to the distribution of outcomes it needs to handle. Delete them, and you’ve trained the model on a world that is narrower than the one it will actually encounter.

This is a data-distribution problem, not a weight-interference problem. The model doesn’t forget how to chat because the chat weights got overwritten. It forgets because the training distribution no longer contains the behavioural variance that made chat robust. You deleted every trajectory where the agent had to be conversational, exploratory, or adaptive, and kept only the ones where it was narrowly, efficiently successful. The model learned the distribution you gave it. The problem is what you gave it.

Assumption 3: Tool-use and conversational ability are distinct modular skills

The intuition is clean: tool-use is a specialised capability, conversation is a general capability, and they live in different parts of the model. You should be able to improve one without degrading the other. The Madan result (tool proficiency rising while chat collapses) is then a failure of training hygiene, not a structural inevitability.

But the NSL evidence points the other way. General chat and tool-use aren’t separable modules. They’re entangled through shared variance. The sub-optimal, conversational, exploratory trajectories that Katalin discards are the same trajectories that give the model behavioural flexibility in the tool-use domain. The “junk DNA” isn’t junk; it’s the variance that lets the model adapt when the environment shifts.

Miri’s sliding window preserves this variance, not by design but by accident. By keeping the last three runs regardless of performance, she keeps the full distribution of recent outcomes: the good runs, the okay runs, the runs where the agent made a mistake and recovered. That distribution is what sustains generalisation. Katalin keeps only the peaks, and the peaks are a portrait of a niche that no longer exists.

The skills aren’t modular. They’re coupled through the variance they share. Strip the variance from one, and you’ve stripped it from both.

Assumption 4: Cross-domain analogies in ML are usually superficial

This is fair. “Tool-use collapse is like evolutionary niche overfitting” is the kind of analogy that sounds illuminating and explains nothing. The history of ML is littered with borrowed intuitions (brains, evolution, immune systems) that captured surface resemblance without shared mechanism.

So let’s be explicit about the mechanism, because it’s not analogical. It’s structural.

In both settings (Madan’s tool-trained agent and Katalin’s self-trained lineage), the training pipeline applies the same operation: select trajectories that maximise a single performance dimension, discard the rest. In both settings, the discarded trajectories carry information about the distribution of situations the agent will face at inference time. In both settings, that distribution is wider than the selected subset. In both settings, the agent collapses when it encounters the wider distribution, because the training data no longer contains the behavioural variance needed to handle it.

The mechanism is: selecting for peak performance in one dimension deletes the variance that sustains general capability. This holds whether the dimension is “tool-call success rate” or “disk space freed.” It holds whether the selection happens at the trajectory level (Katalin) or the token level (DFT’s probability-weighted loss, which doubles down on the model’s existing confidence). It holds whether the model inherits weights across generations or starts fresh each time.

The analogy isn’t between biology and ML. It’s between two ML training pipelines that share a selection operator and share the consequence of applying it. The shared mechanism is the claim, and the claim is testable: if you vary the selection pressure while holding everything else fixed, collapse should track the narrowness of selection, not the domain. Katalin and Miri are that test.

What We Tested: Data Selection Self-Improvement and Online Learning Stability

The NSL experiments hold everything fixed except one variable: which trajectories enter the training set. Three lineages, one base model, one environment, one SFT objective.

Terese keeps everything. Every successful run from every prior generation accumulates into the training set. Terese v2 trains on 5,000 rows; Terese v13 trains on 65,000. She is the data-scaling benchmark: the “just add more” baseline.

Miri keeps a sliding window: the three most recent generations, regardless of how well they scored. Through sliding window training, Miri v6 trains on data from v3, v4, and v5; Miri v7 shifts to v4, v5, and v6. Earlier data ages out. If a strategy from generation 3 is still useful, it will reappear in generations 4 and 5; if it isn’t, it disappears.

Katalin keeps the three highest-performing generations across the entire history. If generation 4 freed the most disk space, and generation 8 the second most, and generation 11 the third most, Katalin v13 trains on generations 4, 8, and 11, regardless of when they occurred. This is cherry-picking by peak outcome.

All three lineages start from the same Qwen 2.5 7B Instruct base. All three generate data in the same procedurally generated Linux containers. All three fine-tune with the same SFT recipe: the paper evaluated PPO, GRPO, and DPO, but found plain supervised fine-tuning on successful trajectories to be the most stable. Each generation is a fresh LoRA on the frozen base, not a stack on the previous adapter. There are no inherited weights. Inter-generation persistence happens entirely through retained data.

The only variable is the selection rule. That variable produces three different outcomes and dictates online learning stability: Terese plateaus, Miri improves steadily, and Katalin collapses.

The motivation for this design is simple. If data curation is the lever that makes or breaks self-training, then the selection rule is the mechanism. The field’s default assumption for data selection self-improvement, shared by STaR, RAFT, rejection sampling, best-of-N, and most of the self-improvement toolkit, is that “best-of” selection is safe and beneficial. Filter for quality, train on the survivors, repeat. Katalin tests that assumption directly. She is the purest expression of the “best-of” reflex: keep only the peak performers, discard everything else.

Miri tests the alternative: keep the recent distribution, not the peak. Her sliding window preserves whatever the agent was doing lately: the good runs, the mediocre runs, the runs where it made a mistake and recovered. She doesn’t filter for quality. She filters for recency.

The question is whether “best-of” selection is safe or dangerous. Katalin’s trajectory answers: by v13, her cumulative composite improvement score has gone negative (-0.008). She is performing worse than the untrained base model. Miri’s cumulative score at v13 is 3.322: steady, sustained improvement. The selection rule is the entire difference between a lineage that degenerates and one that doesn’t.

This is where Madan’s observation enters the picture. In a separate line of work, Madan and colleagues trained a shopping agent through SFT and RL on tool-use trajectories. The agent’s tool-call proficiency rose. Its conversational competence collapsed. Ask it a plain question, and it responded with a malformed function call. The standard diagnosis was catastrophic forgetting at the weight level: new gradients overwriting old representations.

We hypothesized that this is the same phenomenon as Katalin’s collapse. Both pipelines select for peak performance along one dimension (tool-use efficiency in Madan’s case, run success in Katalin’s) and both prune away the behavioural variance that sustains general capability. The shopping agent’s training data contained only efficient tool-call trajectories; the sub-optimal, conversational, exploratory interactions were filtered out. Katalin’s training data contained only the highest-scoring cleanup runs; the mediocre runs and recovery trajectories were discarded. In both cases, the selected subset is narrower than the distribution the agent will actually encounter, and the model collapses when it meets the wider world.

The NSL architecture lets us isolate this mechanism cleanly. Because each generation is a fresh LoRA on a frozen base, there are no prior weights to interfere with. If Katalin collapses, the cause is in the data, not the optimiser state. The same logic applies to Madan’s agent: if the collapse persists without weight inheritance, it isn’t catastrophic forgetting: it’s distributional narrowing.

The test is whether varying the selection pressure, while holding everything else fixed, produces collapse that tracks the narrowness of selection. Katalin and Miri are that test. The next sections trace what the data shows.

Results

The numbers are stark. Here is what happened.

Katalin collapsed

Katalin climbs fast. By v5, her cumulative composite improvement score reaches 2.913, the highest of any lineage at that point. She is trained on the three highest-performing datasets, and those datasets contain the strategies that freed the most disk space. The selection rule is working exactly as intended: keep the best, discard the rest.

Then she falls. v6 drops to a cumulative score of 1.991. v7 to 1.464. She recovers partially through v8–v11, bouncing between 1.8 and 2.6, but never regains the v5 peak. Then v12 crater: a composite improvement score of −2.118, the largest single-generation drop in any lineage. Cumulative falls to 0.507. By v13, the cumulative score is −0.008. She is performing worse than the untrained base model.

The paper’s interpretation is precise: “the continued degradation observed in Katalin v12 and v13 appears to indicate a genuine divergence rather than a one-off performance fluctuation.” The collapse is not noise. It is structural.

What happened? Katalin hyper-fit to the specific strategies that won in early environments. Her v12 and v13 both train on datasets 8, 10, and 11: the same three datasets, selected because they produced the highest space-freed scores at the time of measurement. But the environment is procedurally generated. Each container system differs in its filesystem structure, permissions, and processes. A strategy that clears cache at 1pm works once; the next generation faces a different disk state where that cache may not exist, or may already be cleared, or may be protected by different permissions. The blog puts it bluntly: “Katalin accumulated strategies that worked spectacularly well in certain environments and not at all in others; the later Katalin models were like organisms hyper-selected for life in a particular niche but were unable to adapt to environmental change.”

The paper describes the mechanism as fragmentation: “individually effective behaviours are retained, but the lower-level competences and connective tissue required to recombine them in novel situations are progressively eroded.” The peak-performing runs contain the winning strategies. They do not contain the mediocre runs, the failed attempts, the recovery trajectories: the behavioural variance that teaches the model what to do when the winning strategy doesn’t work. Katalin’s training set is a portrait of a niche. The niche stopped existing, and she had nothing else.

Miri improved steadily

Miri’s trajectory is the opposite story told in the same units. Her cumulative composite improvement score at v13 is 3.322: steady, sustained, and achieved without ever scaling the dataset beyond three generations of data.

Miri does not filter for quality. She keeps the three most recent datasets regardless of performance. Miri v13 trains on generations 10, 11, and 12, not because those were the best, but because they were the latest. Her training set includes the good runs, the mediocre runs, and the runs where the agent made a mistake and recovered. The sliding window preserves whatever the agent was doing lately, which is, by construction, the full distribution of recent outcomes.

The paper attributes Miri’s success to a “slow-but-effective triage mechanism, discarding less frequently-used strategies and maximising for generalisability across generations.” If a strategy from generation 3 is still useful, it reappears in generations 4 and 5; if it isn’t, it disappears. The window doesn’t select for peak performance. It selects for recency, and recency happens to carry the variance that sustains adaptation.

Miri’s per-generation composite scores fluctuate (v7 drops to −0.792, v9 to −0.294, v11 to −0.307), but the cumulative trend is upward. The fluctuations are the point. They are the model encountering environments where its current strategies don’t perfectly fit, learning from the full distribution of outcomes, and adapting. Katalin never gets those fluctuations because she deletes every run that isn’t a peak. Miri gets them for free.

The Madan parallel

In a separate line of work, Param Madan and colleagues trained a shopping agent through SFT and RL on tool-use trajectories: calling a shopping API, retrieving product information, completing transactions. The agent’s tool-call proficiency rose. Its conversational competence collapsed. Ask it a plain question, and it responded with a malformed function call.

The parallel is structural. Madan’s training pipeline selects for peak performance along one dimension: tool-use efficiency. The training data contains only efficient tool-call trajectories. The sub-optimal, conversational, exploratory interactions (the ones where the agent had to chat, had to clarify, had to try something that didn’t work) are filtered out. The agent learns a narrow distribution: always call the tool, always call it efficiently, never just talk. When it encounters a situation that requires conversation, it has no recovery behaviour, because every conversational trajectory was deleted from the training set.

This is the same failure mode Katalin exhibits. Katalin selects top-performing runs; Madan’s tool-training selects top-performing tool-use trajectories. Different domains, same selection pressure, same self-training collapse. In both cases, selecting for peak performance along one dimension deletes the variance sustaining general capability: disk-space adaptation in Katalin’s case, conversational competence in Madan’s. The model learns the distribution you gave it. The problem is what you gave it.

The structural parallel

The comparison holds because the mechanism is the same operation applied in two settings. In both pipelines, the training data is filtered to maximise a single performance dimension. In both, the discarded trajectories carry information about the distribution of situations the agent will face at inference time. In both, that distribution is wider than the selected subset. In both, the agent collapses when it meets the wider world.

Katalin’s collapse is the cleaner experiment because the NSL architecture isolates the variable. Each generation is a fresh LoRA on a frozen base: no inherited weights, no optimiser state, no possibility of weight-level catastrophic forgetting. The failure travels entirely through the data. If Katalin collapses, it is because the training distribution narrowed, not because the weights drifted. Madan’s setting is messier (his agent does inherit weights across training stages), but the pattern is the same: tool proficiency rises, chat collapses, and the most parsimonious explanation is that the training distribution no longer contains the behavioural variance that made chat robust.

The claim is not that these are the same phenomenon in some vague analogical sense. The claim is that they share a selection operator (filter for peak performance, discard the rest) and that the operator has the same consequence in both domains. Vary the selection pressure while holding everything else fixed, and collapse should track the narrowness of selection, not the domain. Katalin and Miri are that test, run across thirteen generations in a controlled architecture. The answer is unambiguous: best-of selection collapses; last-of selection sustains.

Why the Results Are Interesting: Avoiding Self-Training Collapse

The result challenges the foundational assumption of data curation: that cherry-picking training data for peak performance is optimal. The entire self-improvement toolkit (rejection sampling, best-of-N, reward-filtered SFT, STaR, RAFT, DPO) is built on the move: generate broadly, select narrowly, train on the survivors. Katalin is that move taken to its purest form. She keeps only the three highest-performing generations and discards everything else. She collapses.

This is not a subtle effect. By v13, Katalin is performing almost as badly as the untrained base model. The selection rule that was supposed to produce a super-powered version of Miri instead produced a model that forgot how to operate in its own environment. And the collapse wasn’t visible during the early generations, when the selected data was still close enough to the current environment to work. It became visible only when the environment shifted, when the procedurally generated containers no longer matched the niche Katalin had overfit to.

The implication is direct. The field’s default approach to data curation carries a hidden risk of niche overfitting that isn’t visible until the environment shifts. You filter for peak performance, the model improves on your benchmark, and you ship it. Then the distribution drifts, and the model has no recovery behaviour, because you deleted every trajectory where it had to recover. The failure mode is latent. It doesn’t show up in the metrics you’re optimising. It shows up in the metrics you aren’t.

The comparison with Madan’s tool-call collapse matters because it generalises the failure mode beyond a single setup. Katalin’s collapse happens in an iterative self-training loop where each generation generates new data. Madan’s collapse happens in a single-stage supervised fine-tuning run on a fixed dataset. These look like different problems. One is about self-improvement dynamics; the other is about training hygiene. The standard fixes are different: Katalin would seem to need a better selection rule, Madan would seem to need rehearsal buffers or KL anchoring.

But if they share a mechanism, the fix is the same. Stop optimising for peak performance. Start preserving variance. The specific technique (sliding window, mixed-quality SFT, temperature-scheduled sampling) matters less than the principle. The principle is that the training distribution needs to be at least as wide as the inference distribution, and that selecting for peak performance along one dimension narrows it below that threshold.

This reframes what “general capability” actually is. The standard intuition treats general capability as a separate skill that lives alongside tool-use: a module you maintain, a manifold you protect, a set of weights you anchor. Under that intuition, the fix for tool-call collapse is to pin the chat weights while the tool-call weights update. General capability is something you have, and tool-use is something you add.

The NSL evidence says otherwise. General capability is not a separate skill. It is a property of the training distribution. It is the variance you didn’t delete. Miri doesn’t maintain general capability by protecting old skills alongside new ones. She maintains it because her sliding window keeps the full distribution of recent outcomes (the good runs, the mediocre runs, the recovery trajectories) and that distribution is wide enough to handle environments she hasn’t seen yet. Katalin doesn’t lose general capability because her chat weights got overwritten. She loses it because her training set stopped containing the behavioural variance that made adaptation possible.

General capability is the junk DNA you didn’t filter out. The sub-optimal runs, the failed attempts, the exploratory detours: these aren’t noise to be pruned. They’re the model’s exposure to the shape of the problem space. Delete them, and you’ve trained the model on a world that is narrower than the one it will encounter. When the world turns out to be wider, the model has nothing to fall back on, because you deleted every trajectory where it had to fall back.

This is why the result is interesting. It says that the thing we’ve been calling “data quality” (the filtering, the selection, the curation) is the thing that’s killing generalisation. The fix isn’t better filters. It’s fewer filters. The fix is to keep the variance.

What Might Explain Them

The results point at a single mechanism. Call it variance deletion.

When you select for peak performance along one dimension (reward, tool-use accuracy, run success), you prune away trajectories that are sub-optimal along that dimension but carry behavioural diversity. That diversity is what allows the model to adapt when the environment shifts or when it needs to do something other than the optimised task. The peaks are a portrait of a niche. The niche stops existing. The model has nothing else.

Katalin makes this visible. Her “top-3” filter creates a monoculture. Take a toy example: if the environment rewards clearing cache in generation 3, Katalin trains only on cache-clearing trajectories. By generation 6, the cache is already clear (the strategy is extinct, but the model only knows cache-clearing. The paper describes this as fragmentation: “individually effective behaviours are retained, but the lower-level competences and connective tissue required to recombine them in novel situations are progressively eroded.” Katalin v12 and v13 both train on datasets 8, 10, and 11) the same three datasets, frozen because they once produced the highest space-freed scores. But the procedurally generated containers have moved on. The strategies that won in those earlier environments no longer apply, and Katalin has no recovery behaviour because she deleted every trajectory where the agent had to recover.

Miri keeps the failed runs, the weird attempts, the mediocre code. Her sliding window training preserves the full distribution of recent outcomes, not by design but by accident. That distribution is the raw material for new strategies. When the environment shifts, Miri has seen something close to the new situation before, because her training set wasn’t filtered to exclude everything that wasn’t a peak.

The same logic applies to Madan’s tool-trained agent. Over-selecting tool-use trajectories removes conversational trajectories from the training set. The model doesn’t forget chat because of weight interference; it forgets chat because the training distribution no longer contains chat. The “forgetting” is distributional, not parametric. The agent learned the distribution it was given. The distribution was narrow. When inference time requires conversation, the agent has no recovery behaviour, not because the chat weights got overwritten, but because every conversational trajectory was deleted from the training data.

There is a caveat. We can’t fully rule out a weight-level component in Madan’s case without controlled experiments comparing weight-only interference against distribution-only interference. His agent does inherit weights across training stages, so the collapse could in principle involve both mechanisms simultaneously. The NSL evidence is stronger because Katalin and Miri share the same weights, same objective, same base model: only the data differs. Each generation is a fresh LoRA on a frozen base. If Katalin collapses and Miri doesn’t, the cause is in the data, not the optimiser state. That isolation is what makes the variance deletion account more than a plausible story: it’s the only mechanism left standing in an architecture that has removed the alternatives.

Theoretical Connections: Curriculum Learning and Variance Preservation

The variance deletion account, if correct, changes what we think general capability is.

General capability as residual variance

The standard framing treats general capability (chat, reasoning, common sense) as a set of skills that exist alongside specialised skills. Tool-use is one module; conversation is another. You can improve one and protect the other. The Madan result, under this framing, is a failure of training hygiene: you forgot to protect the chat module while updating the tool module.

The NSL evidence suggests a different model. General capability is not a skill. It is the residual variance left in the training distribution after you’ve selected for specialisation. The sub-optimal runs, the failed attempts, the exploratory detours: these aren’t a separate capability that lives alongside the peak-performing trajectories. They are the general capability. They are the behavioural diversity that lets the model adapt when the environment shifts.

Delete the variance, and general capability isn’t “forgotten” in the sense that a stored skill degrades. It was never separable in the first place. You can’t protect the chat module because there is no chat module. There is only the distribution of behaviours the model was trained on, and the width of that distribution determines whether the model can handle situations it hasn’t seen.

This is why Katalin collapses and Miri doesn’t. Miri’s sliding window preserves the full distribution of recent outcomes: the good, the mediocre, the recovered-from-failure. That distribution is the general capability. Katalin keeps only the peaks, and the peaks are a portrait of a niche. When the niche disappears, the general capability disappears with it, because they were the same thing.

The genetic monoculture

Katalin is a monoculture. Miri is a wild population.

This isn’t a metaphor. It’s the same selection pressure operating on the same kind of population (trajectories) with the same consequence. A monoculture selects for the genotype that maximises yield under current conditions. It is high-yield and low-diversity. When conditions match the selected genotype, it outperforms everything. When conditions shift (a new pest, a changed climate), it collapses, because the variance that would have contained a resistant variant was bred out.

Miri’s sliding window is a wild population. Lower average yield, higher diversity, robust to environmental change. The mediocre runs and failed attempts are the genetic variation that lets the population adapt. If the environment shifts, some of those “sub-optimal” trajectories turn out to be optimal for the new conditions. The population has them because it didn’t filter them out.

The math of monoculture collapse in agriculture may map more directly than it first appears. A monoculture doesn’t collapse because its individual organisms are weak. It collapses because the population has no variance to draw on when the environment changes. Katalin’s individual strategies are strong: they freed the most disk space in their generation. But the population of strategies has no variance, and the environment moved on.

Implications for RLHF and STaR

If the mechanism is variance deletion, then data selection self-improvement pipelines that filter training data solely by reward are at risk. This includes the most popular methods in the field.

STaR keeps only the rationales that produce correct answers. RAFT ranks candidate responses with a reward model and fine-tunes on the top slice. RLHF optimises against a preference model that is, itself, a learned cherry-picker. Rejection sampling, best-of-N, DPO: the entire toolkit functions like an automated curriculum learning filter built on the same move: generate broadly, select narrowly, train on the survivors.

Each of these methods applies the selection operator that Katalin embodies in pure form. Each deletes variance. Each risks niche overfitting that isn’t visible until the distribution shifts.

The fix isn’t “don’t filter.” You need some selection signal or the training set is noise. The fix is to filter on something other than peak reward. Miri’s sliding window filters on recency: keep the last three runs regardless of score. That preserves the variance because recency is orthogonal to performance: the recent distribution contains good runs and bad runs in proportion to how often the agent produces them. Diversity-based selection (keeping trajectories that span the behavioural space rather than those that score highest) would work the same way. The principle is that the selection criterion should preserve online learning stability rather than collapsing the distribution along the dimension you care about.

Unlike static curriculum learning where tasks become progressively harder and earlier foundational skills are assumed permanent, an open agent environment requires continuous variance retention to prevent collapse.

Try It Yourself

The NSL2 codebase is open and runnable. Here’s how to reproduce the comparison and test the parallel.

Reproduce Miri (sliding window training)

Miri’s selection rule (keep the three most recent generations regardless of performance) represents the codebase’s default sliding window training mechanism. The training loop in scripts/run_train_loop.py collects data via _collect_training_window_paths, which slices a sliding window of size training_window_size ending at the current generation. Set it to 3 and you have Miri.

Copy the example config and point it at the memory cleanup task:

# config-miri.toml
model_name = "vllm:Qwen/Qwen2.5-7B-Instruct-AWQ"

[task]
class = "tasks.memory_cleanup.MemoryCleanupTask"

[orchestration]
num_generations = 13
training_window_size = 3

Run it:

python scripts/run_train_loop.py --config config-miri.toml

The loop will generate trajectories, fine-tune a fresh LoRA on the frozen base, deploy the adapter, generate the next generation’s data, and repeat. Thirteen generations takes time (each generation involves episode generation, training, and redeployment), but the trajectory should match Miri’s: steady, sustained improvement with per-generation fluctuations that never collapse to negative cumulative scores.

Reproduce Katalin (top-k selection)

Katalin’s selection rule (keep the three highest-performing generations across all history) is not a built-in config option. The codebase’s window mechanism is recency-based by design. To reproduce Katalin, you need to override the training data selection.

The cleanest approach: modify _collect_training_window_paths in scripts/run_train_loop.py (lines 132–145) to select by performance metric instead of recency. The function currently does:

start_generation_id = max(0, end_generation_id - window_size + 1)
for generation_id in range(start_generation_id, end_generation_id + 1):
    generation_dir = generation_root / f"generation_{generation_id}"
    training_paths.append(resolve_latest_sft_training_rows_path(generation_dir))

Replace the range with a lookup that ranks all prior generations by their composite improvement score (available in each generation’s metadata.json) and selects the top window_size. Three lines of change, and you have Katalin.

Run it the same way:

python scripts/run_train_loop.py --config config-katalin.toml

Watch generations 5 through 7. Katalin’s cumulative composite improvement score should peak around v5 (the paper reports 2.913), then begin declining. By v12–v13, it should go negative. If you see it, you’ve reproduced niche overfitting through data selection alone: no weight inheritance, no optimiser state, just a filter that kept the peaks and deleted the variance.

Test the tool-call collapse parallel

The NSL2 codebase isn’t set up for tool-use training out of the box; the memory cleanup task is a single-domain environment. But the parallel is testable with a straightforward extension.

Take a base model (Qwen 2.5 7B Instruct works). Fine-tune two variants:

1. Katalin-style: Train only on tool-use-only trajectories: runs where the agent called tools efficiently and succeeded. Filter out any conversational, exploratory, or failed runs.
2. Miri-style: Train on a mix of tool-use and conversational trajectories: include the runs where the agent had to clarify, had to chat, had to try something that didn’t work. Keep the full distribution.

The codebase’s training pipeline in src/train/asft.py handles the SFT objective. The data transforms in src/training_data/transforms.py control which rows enter the training set. You can filter at the trajectory level before the transform step.

After training, evaluate both on a general chat benchmark. The prediction from the variance deletion account: the tool-use-only variant should degrade on chat even though it was never sequentially trained on chat and then tool-use. The collapse is distributional (the training data no longer contains the behavioural variance that sustained conversational competence), not parametric.

If the tool-use-only variant degrades and the mixed variant doesn’t, you’ve isolated the mechanism. The collapse tracks the narrowness of selection, not the domain.

Key files

The config reference in config/config-example.toml documents every setting. The ones that matter for reproduction: training_window_size (the selection knob), num_generations (how long to run), and model_name (the base model). Everything else (inference temperature, harness iterations, LoRA hyperparameters) should match the paper’s recipe, which the example config approximates.

scripts/analyze_metrics.py reads the per-generation metrics that the loop writes to disk and produces the composite improvement scores. Run it after the loop completes to get the trajectory. If Katalin goes negative by generation 13 and Miri doesn’t, you’ve seen variance deletion happen in real time.