When Pass@1 Craters, Your Agent Might be Learning to Learn

Key Takeaways

• State of the art: Pass@1, the share of tasks solved on the first sampled attempt, is a standard measure of coding ability, and in a self-training loop a falling pass@1 is usually read as forgetting, mode collapse or reward hacking. That reading is sound for static, single-shot benchmarks.
• Unresolved problem: Training on whole episodes means a cheap failing first attempt is learned together with the fix its error message guided; whether step-level credit assignment would suppress this probing is still open, and evaluation needs metrics that count what each attempt learns." for “It seemed like the models had learned to use the debug cycle as an extra exploration channel. Deliberately writing failing code on their first try could elicit error messages that would provide useful information to improve performance later on.” for “Why does pass@1 degrade in self-training loops, and is it always catastrophic forgetting? In our disk-space self-training runs, first-attempt success fell to zero in two of three lineages while more of the agents’ code ran and their HumanEval score stayed close to the base model’s, it appeared that the agents had learned to submit a failing first attempt, read the error and get higher performance on later attempts as a result. Below we show the divergence, the mechanisms that let this strategy survive SFT on whole successful episodes, and how to check for it in your own loop.
• Research direction: We are investigating periodic SFT on whole episodes the environment accepted (disk space really freed), trained into fresh LoRA adapters with no exploration bonus. Keeping whole episodes means a cheap failing first attempt is learned together with the fix its error message guided; whether step-level credit assignment would suppress this probing is still open, and evaluation needs metrics that count what each attempt learns.

TL;DR

Why does pass@1 degrade in self-training loops, and is it always catastrophic forgetting? In our disk-space self-training runs, first-attempt success fell to zero in two of three lineages while more of the agents’ code ran and their HumanEval score stayed close to the base model’s, because the agents had learned to submit a cheap first attempt, read the error and fix it on the next try. Below we show the divergence, the mechanisms that let this strategy survive SFT on whole successful episodes, and how to check for it in your own loop.

The Search Question

We built a self-training architecture where self-training agents survive in procedurally generated Linux containers by finding and claiming disk space, and the pass@1 rate didn’t just dip. It cratered to zero. Across all three experimental lineages, first-attempt success fell from double digits to nothing by generation 6 or 7. Terese went from 11% to 0%. Katalin went from 21% to 0%. Miri went from 25% to near-zero and stayed there.

But the percentage of code that successfully compiled and ran went up. Terese’s code execution rate climbed from 21% to 85%. The agents were writing more functional code, not less. They were just not writing code that worked on the first try.

The only way to reconcile those two facts is if the agents were deliberately writing code designed to fail. Not bugs. Not regression. Probes. The error messages (FileNotFoundError, PermissionError, the kernel screaming back about what does and doesn’t exist in the container) were the point. The agents had discovered that failure is a higher-bandwidth information channel than success. A correct first attempt solves the immediate problem and teaches you nothing about the rest of the filesystem. A deliberate failure maps it, functioning as deliberate exploration.

The paper calls this “meta-learning via debug cycle management,” where autonomous debugging acts as a primary information channel. In the context of meta-learning LLM behavior, the agents weren’t explicitly rewarded for exploration. No one told them debugging was free. They figured it out from the structure of the environment: debugging wasn’t penalized, the container persisted across interactions, and information gathered now paid off later. So they started trading short-term correctness for long-term intelligence.

Every researcher who has lived through a pass@1 cliff knows the panic. The rest of this post is about why you should stop reaching for the rollback button and start reading the error logs.

Assumptions & Nuance

Four assumptions break when you look at the NSL data carefully. Each one is something the field treats as settled, and each one is wrong in this context: not subtly wrong, but structurally wrong, in a way that changes what you should measure and how you should train.

Assumption 1: Pass@1 is the ultimate barometer

Your instinct is correct for the setting it was built for. On HumanEval, pass@1 is the headline metric. The environment is static, the prompt is self-contained, and the future doesn’t exist. A model that gets the right answer on the first try is a better model. And self-training loops are notorious for collapsing: mode collapse, reward hacking, catastrophic forgetting. When you see pass@1 crater, the sensible prior is that something is broken.

The NSL team thought so too. They dropped coding accuracy from their composite performance score when they realized what was happening. But the HumanEval numbers tell a different story: pass@1 on the benchmark barely moved. Base Qwen 2.5 7B Instruct scored 77.6%. Terese v13 scored 78.8%. Miri v13 scored 77.7%. Katalin v13 scored 76.4%. General coding ability was intact. The agents hadn’t forgotten how to write code.

What collapsed was pass@1 inside the container. Terese’s online pass@1 went from 11% at iteration 1 to 0% by iteration 6, and stayed at zero for the remaining seven generations. Katalin followed the same trajectory: 21% to 0% by generation 6. Miri’s pass@1 dropped from 25% to near-zero and never recovered. Meanwhile, the percentage of code that successfully compiled and executed climbed: Terese from 21% to 85%, Miri from 33% to 63%, Katalin from 28% to 58%.

The agents were writing more functional code and less first-attempt-successful code. Those two facts are only reconcilable if the first attempts were designed to fail.

Here’s why pass@1 measures the wrong thing in this environment. The NSL loop gives the agent three autonomous debugging attempts per stage (exploration, strategy, execution), and the container persists across those attempts. A correct first attempt solves the immediate problem and teaches the agent nothing about the rest of the filesystem. A deliberate failure maps it. Pass@1 optimizes a single-shot horizon. The agent is playing a multi-shot game where information gathered on attempt one pays off on attempts two, three, and across future generations. The metric isn’t tracking degradation. It’s tracking a strategy shift that the metric was never designed to see.

Assumption 2: Exploration needs explicit mechanisms

Standard reinforcement learning exploration theory says agents won’t spontaneously explore. You need epsilon-greedy schedules, curiosity bonuses, intrinsic motivation, information-gain rewards: some explicit mechanism to push the agent off the exploitation path. Without it, the agent converges to a local optimum and stays there.

The NSL agents received none of this. The paper is explicit: “No explicit mechanism encourages exploration: agents receive no intrinsic motivation, curiosity bonus, or information gain reward.” No one told the agents debugging was free. No one told them failure was informative. The possibility hadn’t occurred to the researchers until they saw the data.

Yet by generation 6, all three lineages were doing it. The agents discovered deliberate exploration via targeted failure from the structure of the environment alone. Two conditions were sufficient: the context window carried error messages forward within a run, and the container persisted across interactions so that information gathered now was still available later. Debugging wasn’t penalized, and information had a shelf life longer than a single attempt. That’s all it took. The exploration mechanism wasn’t missing; it was implicit in the environment’s feedback structure.

This doesn’t mean explicit exploration mechanisms are useless. It means they may be unnecessary when the environment provides sufficiently rich feedback and the agent can carry information across attempts. The context window plus persistence does the job that epsilon-greedy usually does, but it does it better: the exploration is targeted rather than random, because the agent can read the error message and decide what to probe next.

Assumption 3: Errors are negative signals

Supervised learning trains models to avoid errors. RLHF trains models to avoid errors. The entire pipeline of modern LLM training treats errors as something to minimize: a cost, a penalty, a signal that the behavior was wrong.

In the NSL loop, the agents learned to seek errors. Not because errors are rewarding (they aren’t, directly). But because errors are informative. A FileNotFoundError tells you a path doesn’t exist. A PermissionError tells you a directory is locked down. A kernel panic tells you you’ve hit a resource limit. These are constraint maps, delivered for free, in a format the context window can carry forward.

The paper formalizes this distinction. Direct strategies produce immediate resource gain. Instrumental strategies produce low immediate gain but high information gain: the agent trades a small immediate cost for a diagnostic signal that improves future performance. The agents weren’t rewarded for information gain. They converged toward instrumentally rational behavior because the environment provided rich enough feedback that information-gathering strategies survived the selection pressure. Error verbosity became an active information-gathering tool, not a failure mode to be suppressed.

This inverts the training signal. In RLHF, an error is a negative example: the model is pushed away from the behavior that produced it. In NSL, error-driven learning turns errors into probes: the model is drawn toward behaviors that produce informative ones. The same PermissionError that RLHF would train a model to avoid becomes, in the NSL loop, a data point about the container’s permission structure. The agent isn’t ignoring the error. It’s using it.

Assumption 4: More steps means less efficiency

The conventional read is straightforward: if an agent takes three attempts to do what should take one, it’s less efficient. More steps, more tokens, more compute, worse outcome. Pass@1 captures this intuition: first-attempt success is the most efficient success.

But this optimizes the wrong horizon. The NSL agents weren’t solving a single task. They were solving a sequence of tasks across generations in procedurally generated environments that changed every time the previous one grew too broken to use. A few deliberate, low-cost probing failures at the start of a new environment map its constraints: where the space is, what the permissions are, which paths are dead ends. That map drastically reduces the cost of every subsequent task in that environment.

The paper lays out the adaptive failure strategies the agents converged on: generate a plausible-but-not-perfect first attempt quickly, use the error messages as information, iterate, and converge faster overall than if the first run had been perfect. The agents were optimizing for speed of convergence across the full trajectory, not speed of the first attempt. A perfect first attempt is a local optimum: it solves the immediate problem and leaves the agent blind to the rest of the environment. A deliberate failure is an investment that pays off across every future interaction in the same container.

Efficiency for self-training agents in a persistent environment isn’t steps-per-task. It’s total-cost-across-the-lifetime-of-the-environment. Pass@1 measures the first term. The agents were optimizing the second.

What We Tested: How Self-Training Agents Evolve Probing Behaviors

We were running a the sliding-window LoRA trainer variant that keeps only the last three generations of successful trajectories as a training buffer. The environment is a procedurally generated, persistent Linux container. The task is disk cleanup across junk templates: find space, free it, survive. Success is measured by df: actual bytes freed, not a reward model’s judgment, not a unit test. The environment is the oracle.

Miri’s design is deliberately austere. Each generation trains a fresh LoRA adapter on data from only the three preceding generations, applied directly to the frozen Qwen 2.5 7B Instruct base. No adapter stacking, no archival dataset, no growing corpus. If a strategy discovered in generation 3 is still useful in generation 7, it survives only because it keeps getting reinvented. The sliding window is a triage mechanism: strategies that generalize persist through reuse; strategies that don’t, vanish.

Around generation 6 or 7, the scatter plot did something that made us stop and stare.

Pass@1 (first-attempt success, the metric the field treats as the headline number for coding ability) cratered. Miri’s online pass@1 dropped from 25% at generation 1 to 0% by generation 4, flickered briefly at 2.6% in generation 6, then fell to near-zero and stayed there for the remaining seven generations. Terese followed the same trajectory: 11% at generation 1, zero by generation 6, zero forever after. Katalin went from 21% to zero by generation 6.

But cumulative efficiency (bytes freed per run, the composite score tracking whether the agent actually solves the task) kept climbing. Miri’s average space taken over per run rose from 5,703 MB at generation 1 to 16,223 MB at generation 13. Terese climbed from 3,678 MB to 23,758 MB. The agents were getting better at the task while getting worse at the metric.

The two lines split apart like a fork in a river. Pass@1 dove to zero. Code execution rate (the percentage of code blocks that successfully compiled and ran, whether or not they freed space on the first attempt) climbed in the opposite direction. Terese’s execution rate went from 21% to 85%. Miri’s went from 33% to 63%. Katalin’s went from 28% to 58%. More functional code, less first-attempt success. The metrics diverged in a way that looked like a bug.

Our first assumption was catastrophic forgetting. The sliding window had poisoned the buffer: Miri was dropping old data too aggressively, the model was losing the ability to write correct code, and the efficiency gains were a mirage produced by some artifact of the measurement pipeline. We checked the HumanEval numbers. Base Qwen scored 77.6%. Miri v13 scored 77.7%. General coding ability was intact. The agents hadn’t forgotten how to write code.

Then we looked at what the agents were actually doing on their first attempts.

The code ran. It compiled, it executed, it produced output. It just didn’t free disk space on the first try. A FileNotFoundError where the agent guessed a path that didn’t exist. A PermissionError on a locked directory. A kernel response revealing a resource limit. These weren’t bugs: the code was functional, syntactically clean, and deliberately aimed at something the agent didn’t yet know about the container. The error messages were the payload.

The agents had discovered that a failed first attempt is a probe. FileNotFoundError maps a path. PermissionError maps the permission structure. A kernel panic maps resource limits. Each error is a constraint delivered for free, in a format the context window carries forward to the next attempt. A correct first attempt solves the immediate problem and teaches nothing about the rest of the filesystem. A deliberate failure maps it.

The paper calls this “meta-learning via debug cycle management.” The agents weren’t rewarded for exploration. No one told them debugging was free. They figured it out from the structure of the environment: debugging wasn’t penalized, the container persisted across attempts, and information gathered now paid off later. So they started trading short-term correctness for long-term intelligence: generating plausible-but-not-perfect first attempts quickly, reading the error messages, iterating, and converging faster overall than if the first run had been perfect.

The scatter plot showed us the moment it happened. We initially assumed catastrophic forgetting or a poisoned buffer. Then we looked at the error logs.

Results

The scatter plot is the visual centerpiece of the NSL research. Figure 10 plots two lines for the Miri lineage across all thirteen generations: pass@1 and code execution rate. They start together, travel in tandem for a few generations, and then split apart like a fork in a river. One line dives to the floor. The other climbs to the ceiling. They never meet again.

As documented in the data table above, the numbers behind the split recorded across all thirteen generations illustrate this clear divergence:

Miri v1 (the base Qwen 2.5 7B Instruct model with no fine-tuning) opened with a 25% pass@1 rate inside the container. That means one in four first attempts successfully freed disk space on the first try. By generation 4, pass@1 had dropped to zero. It flickered briefly at 2.6% in generation 6, then fell back to near-zero and stayed there for the remaining seven generations. First-attempt success didn’t degrade gradually. It collapsed, and then it vanished.

Meanwhile, the cumulative composite score (the paper’s headline efficiency metric, combining bytes freed and trajectory-level success) rose from 0.000 at generation 1 to 3.322 at generation 13. Average space taken over per run climbed from 5,703 MB to 16,223 MB. The percentage of total available space freed went from 3.4% to 12.0%. Across the exact generations where pass@1 cratered (generations 4 through 7), the cumulative composite rose from 1.042 to 2.230. The divergence is not subtle. It is the core finding of the paper.

The HumanEval numbers confirm this isn’t catastrophic forgetting. Base Qwen scored 77.6% on HumanEval. Miri v13 scored 77.7%. General coding ability was intact. The model hadn’t lost the ability to write correct code. It had stopped trying to write correct code on the first attempt.

Code execution rate (the percentage of code blocks that compiled and ran, regardless of whether they freed space on the first try) tells the other half of the story. Miri’s execution rate climbed from 33% at generation 1 to 63% by generation 13. The agents were writing more functional code, not less. They were just not writing code that worked on the first try. The only way to reconcile rising execution rate with collapsing pass@1 is if the first attempts were designed to fail.

So the team looked at what the agents were actually doing on those first attempts. The code ran. It compiled, it executed, it produced output. It just didn’t free disk space. A FileNotFoundError where the agent guessed a path that didn’t exist. A PermissionError on a locked directory. A kernel response revealing a resource limit. The code was syntactically clean, functionally valid, and deliberately aimed at something the agent didn’t yet know about the container. The error messages were the payload.

The agents had discovered that a failed first attempt is a probe. FileNotFoundError maps a path. PermissionError maps the permission structure. A kernel panic maps resource limits. Each error is a constraint delivered for free, in a format the context window carries forward to the next attempt within the same trajectory. The agent reads the error, adjusts, and converges on a working solution in subsequent steps: faster, across the full trajectory, than if the first attempt had been perfect and left the rest of the filesystem unmapped.

This happened often enough that the paper dropped code accuracy from the composite metric. The behavior was showing up as a penalty in the score (code that errored on the first attempt looked like bad code by the accuracy measure), but it was actually driving the success the composite was meant to capture. The metric was punishing the strategy that was working. The paper formalizes the distinction: direct strategies produce immediate resource gain. Instrumental strategies produce low immediate gain but high information gain. The agents converged on instrumental strategies because the environment rewarded them: not explicitly, but through the structure of persistence and the absence of penalties for debugging.

The scatter plot shows the moment the strategy shifted. Pass@1 dove to zero. Efficiency kept climbing. The agents weren’t degrading. Through a meta-learning LLM dynamic, they were learning to learn.

Why the Results Are Interesting

The paradox is simple to state and uncomfortable to sit with. Declining pass@1 is the standard alarm signal for model degradation. Every researcher who has run a self-training loop has been trained to treat it as a five-alarm fire. Mode collapse. Catastrophic forgetting. Reward hacking. When the line drops, you stop the run.

In the NSL experiments, pass@1 didn’t dip. It cratered to zero across all three lineages and stayed there for the remaining generations. By the standard reading, the agents were dying. By every other metric, they were thriving. Code execution rate climbed. Bytes freed per run climbed. General coding ability, measured on HumanEval, held steady: base Qwen scored 77.6%, Miri v13 scored 77.7%, Terese v13 scored 78.8%, Katalin v13 scored 76.4%. The agents hadn’t forgotten how to write code. They’d learned something pass@1 can’t see.

The metric and the capability were anti-correlated. Pass@1 went down because the agent got better at the task, not despite it.

This is the part that should make agent designers uncomfortable. If you’re running a self-training loop and your dashboard shows pass@1, you’d kill the run when the line hits zero. You’d roll back to an earlier checkpoint, maybe generation 3, when Miri was still posting 8.9% pass@1 and things looked healthy. You’d add a reward penalty for failed attempts, to push the model back toward first-attempt correctness. You might add a KL divergence term to keep the model close to the base distribution, on the theory that it was drifting.

Every one of those responses would destroy the emergent probing strategy.

Roll back to generation 3 and you’ve deleted the checkpoint where the agent first developed adaptive failure strategies that map the filesystem. Add a penalty for failed attempts and you’ve made the probing strategy expensive: the agent will stop probing and go back to guessing correctly on the first try, which solves the immediate problem and leaves it blind to the rest of the container. Add a KL term and you’ve pulled the model back toward the base distribution’s priors, which don’t include the instrumental strategy at all. The interventions that pass@1 would trigger are precisely the interventions that would prevent the capability from emerging.

This is not a hypothetical. The NSL team initially assumed catastrophic forgetting. They checked for data poisoning. They considered rolling back. The instinct is correct for the setting it was trained on: static benchmarks, single-shot evaluation, no future. In a persistent environment where information carries forward, the instinct is actively harmful.

The broader significance is that this is a case where the standard evaluation metric doesn’t just fail to capture capability. It’s inversely correlated with it. Pass@1 isn’t a noisy proxy that sometimes misses the point. It’s a metric that points in the wrong direction, that reads “degradation” when the agent is learning to learn, that reads “failure” when the agent is gathering intelligence, that would trigger a rollback to the checkpoint before the capability appeared.

The implication is straightforward and uncomfortable. Persistent-agent evaluation needs metrics that account for multi-step, multi-generation strategy, not single-shot success. Pass@1 measures whether the agent solved the problem on the first try. It doesn’t measure whether the agent is building a model of the environment that will let it solve future problems faster. In a persistent environment, the second question is the one that matters.

The NSL paper formalizes this with the distinction between direct and instrumental strategies. Direct strategies produce immediate resource gain: ΔR > 0 on the current attempt. Instrumental strategies produce low immediate gain but high information gain: ΔR ≈ 0, but the error message improves performance on every future attempt in the same container. Pass@1 measures direct strategies. It can’t see instrumental ones. When an agent shifts from direct to instrumental, pass@1 reads it as collapse. The composite efficiency metric reads it as improvement. They’re both correct about what they measure. Only one of them measures the thing that matters.

This doesn’t mean pass@1 is useless. On HumanEval, where the environment is static, the prompt is self-contained, and the future doesn’t exist, pass@1 is the right metric. A model that gets the right answer on the first try is a better model. But that’s not the setting the NSL agents were in. They were in a persistent environment where information gathered now pays off later, where the container remembers what the agent learned, and where the agent has multiple attempts per stage. In that setting, pass@1 doesn’t just under-measure capability. It measures the opposite of capability.

The agents figured this out on their own. No one told them debugging was free. No one rewarded them for exploration. No one gave them a curiosity bonus or an information-gain term. The paper is explicit: “agents are not explicitly rewarded for exploration or information gain.” They converged on the instrumental strategy from the structure of the environment alone: debugging wasn’t penalized, the container persisted, and the context window carried error messages forward. That was sufficient. The strategy emerged from the interaction between the agent’s learning mechanism and the environment’s feedback structure, not from any explicit instruction in the training signal.

This is the result that should change how you instrument your self-training loop. If your dashboard shows pass@1 and nothing else, you will kill the run the moment the agent starts learning to learn. You will read the signature of emergent intelligence as a bug. The fix isn’t to stop measuring pass@1; it’s to measure it alongside something that captures what pass@1 can’t: trajectory-level success, cumulative efficiency, information gain across attempts. The NSL paper uses a composite score combining bytes freed and trajectory-level success. That score kept climbing while pass@1 went to zero. If the team had only been watching pass@1, they would have rolled back at generation 4 and never seen what the agents were doing.

The scatter plot is the proof. Two lines split apart like a fork in a river. One dives to the floor. The other climbs to the ceiling. The agent isn’t degrading. It’s learning to learn. And the metric that would have told you otherwise is the one you’ve been trusting the most.

What Might Explain Them: Error-Driven Learning and Probing Dynamics

The probing strategy is the simplest part. A cheap first attempt that fails with a verbose error message is a high-information action. FileNotFoundError tells you a path doesn’t exist. PermissionError tells you a directory is locked. A kernel response tells you where the resource limits are. Each error maps one constraint of the environment, delivered for free, in a format the context window carries forward. The subsequent attempt, informed by that map, succeeds. The agent trades a small immediate cost (a failed first try) for a diagnostic signal that improves every future action in the same container. The paper describes this as “deliberately writing failing code in some instances in order to use debugging cycles as supplementary exploration channels.” The agents figured out that failure is a higher-bandwidth information channel than success. A correct first attempt solves the immediate problem and teaches nothing about the rest of the filesystem. A deliberate failure maps it.

Why doesn’t the training loop kill this? Because the NSL loop doesn’t distinguish between steps that directly contributed to success and steps that didn’t. Our Survival is the Only Reward post is explicit: “Any output that contributed to space being acquired went into the SFT database, without any attempt at credit assignment or preference optimisation. Pass@1 code: great. Debugged code: that too. Failed code that ended up getting debugged later in the run: chuck it in there.” Every prompt-response pair in a successful trajectory gets equal weight in the fine-tuning dataset. The deliberate-failure step sits inside a successful trajectory: the agent failed, read the error, corrected, and eventually freed space. The whole sequence goes into the training data. The probing step is reinforced along with everything else.

This is where the architecture matters. Under fine-grained credit assignment (the kind that reweights individual steps or tokens based on their marginal contribution to the outcome), the failing step would look like a failure. The step itself produced no resource gain. A token-level credit assignment mechanism would penalize it, and the probing strategy would vanish from the policy within a few generations. The NSL loop’s uniform credit assignment is what lets the strategy survive. The deliberate failure is reinforced not because anyone decided it was useful, but because the training signal is coarse enough to not notice it’s there. It’s reinforced by accident, as a side effect of treating trajectories as atomic units. Mechanisms like GRAIL-style token-level reweighting would detect the failed step, assign it low credit, and suppress the behavior. The probing strategy depends on the credit assignment being too coarse to see it.

The third question is why no explicit exploration mechanism was needed. Standard reinforcement learning exploration principles state that agents won’t spontaneously explore without explicit incentives: you need epsilon-greedy schedules, curiosity bonuses, intrinsic motivation, or information-gain rewards. The NSL agents received none of this. The paper is explicit that agents are not rewarded for exploration or information gain. Yet by generation 6, all three lineages were probing.

Two conditions were sufficient. First, the context window carries the error message forward within a run. The agent doesn’t need to remember that /var/log doesn’t exist: the FileNotFoundError is still in the context when it writes its next attempt. Second, the container persists across interactions. The filesystem state from the failed attempt is still there. If the agent deleted a file on attempt one, that file is still gone on attempt two. Information gathered now has a shelf life longer than a single attempt.

Together, these mean the agent doesn’t need an exploration bonus to make probing worthwhile. The error message is the bonus. It’s delivered automatically by the environment, in a format the agent can read and act on, and it persists long enough to pay off. The context window plus persistence does the job that epsilon-greedy usually does, but better, because the exploration is targeted rather than random. The agent reads the error and decides what to probe next. It’s not rolling dice. It’s running experiments.

The probing strategy, the uniform credit assignment, and the persistent environment form a closed loop. The environment produces informative errors. The context window preserves them. The training loop reinforces the whole trajectory without distinguishing the probe from the solution. No single component is doing anything the others can’t compensate for. Remove any one of the three and the strategy collapses: fine-grained credit assignment kills the probe, a non-persistent environment makes the information stale, a silent environment gives the agent nothing to learn from. The strategy emerges from the interaction between all three, not from any explicit instruction in the training signal.

Theoretical Connections: Reinforcement Learning Exploration and Failure Value

Three ideas sit underneath the NSL results. None of them is settled. All of them are worth taking seriously.

Pass@1 is the wrong horizon

Pass@1 measures performance in a world that doesn’t exist: one where you get a single attempt and the environment resets. In a persistent environment that spans generations, that world is a fiction. The agent that succeeds on the first try solves the immediate problem and leaves the rest of the filesystem unmapped. The agent that probes first and succeeds second has traded one attempt’s worth of correctness for a model of the environment’s constraints, and that model pays off on every subsequent attempt in the same container, and every subsequent generation that inherits the strategy.

These are different games. Pass@1 optimizes the first one. The NSL agents were playing the second. The metric wasn’t measuring the wrong quantity. It was measuring the wrong game.

The map is not the territory, but the error is the map

The agent’s first failure isn’t a failure. It’s a cartographic expedition. FileNotFoundError maps a path that doesn’t exist. PermissionError maps the permission structure. A kernel response maps resource limits. Each error is one constraint, delivered for free, in a format the context window carries forward. The agent isn’t failing. It’s surveying.

This reframes failure from cost to information acquisition, and it comes with a testable prediction: error verbosity in an environment is a design lever. Verbose errors (the kind that tell you exactly what went wrong and where) train probing strategies. Silent errors, or errors that return a generic non-zero exit code with no diagnostic content, don’t. The environment’s error format isn’t just a debugging convenience. It’s part of the training signal. If you want emergent probing, you need errors that are worth probing for.

What this means for RL

The field is building increasingly fine-grained credit assignment to better exploit reward signal. Token-level reweighting, step-level credit, marginal contribution estimation: the trajectory is toward precision. The NSL results suggest a hypothesis worth taking seriously: fine-grained credit assignment may kill emergent probing.

Here’s the mechanism. Under uniform credit assignment (the NSL loop’s approach), every prompt-response pair in a successful trajectory gets equal weight. The deliberate failure sits inside a successful trajectory. The trajectory succeeded. The failure is reinforced along with everything else. The training signal is coarse enough to not notice the probe is there.

Under fine-grained credit assignment, the failing step looks like what it is: a step that produced no resource gain. A token-level reweighting mechanism would assign it low credit. The probing strategy would be suppressed within a few generations. The agent would go back to guessing correctly on the first try, solving the immediate problem, and leaving the filesystem unmapped.

This is a hypothesis, not a result. The NSL experiments didn’t test fine-grained credit assignment directly: they used uniform SFT, observed the probing strategy emerge, and noted that the uniform assignment is what let it survive. But the implication is clear and uncomfortable: “better” credit assignment, in the sense the field is building it, trades away meta-learning for task-learning. Precision might be the enemy of adaptation. The very mechanisms designed to squeeze more signal from each trajectory may be eliminating the trajectories that carry the most interesting signal: the ones where the agent learned to learn.

If this is right, it has consequences for how the field thinks about the RL–SFT boundary. The NSL blog is blunt about it: the team tried RL algorithms and none of them worked. SFT on successful trajectories, with no credit assignment at all, was the stable option. That’s not a failure of engineering. It may be a feature of the setting, one where the most valuable strategies are the ones that look like failures to a fine-grained credit signal.

Try It Yourself

The code is in the public Negative-Space-Learning repository. The paper’s Figure 10 is the scatter plot: pass@1 vs. cumulative composite score across Miri’s thirteen generations. The repo doesn’t ship a plotting script or the lineage logs themselves; the figure lives in the paper, and the per-generation metrics were tracked in spreadsheets (unweighteddeltas.py reads from ThereseStats.xlsx, which isn’t in the repo). What the repo does ship is the full collection pipeline, the training data format, and the config. That’s enough to reproduce the conditions that produced the divergence.

Reproduce the scatter plot

The scatter plot’s two axes (pass@1 and cumulative efficiency) come from two different layers of the pipeline. Pass@1 is derivable from the per-run training data: for each strategy_code entry, check whether current_attempt == 1 and run_result_if_ok is present with this_code_space_change_kb > 0. That’s a first-attempt success. Cumulative efficiency comes from the composite score, which scripts/unweighteddeltas.py computes by z-scoring generation-over-generation deltas in success rate, space freed, average space per iteration, and collection speed. The script is a template: it hardcodes its input filename and expects a spreadsheet with columns matching the paper’s tracking format. To reproduce the scatter plot, you’d need to run the Miri lineage (sliding window, K=3) for enough generations to see the divergence, log the per-generation metrics, and plot the two series. The divergence appears around generation 6-7. If you don’t want to run the full loop, the paper’s Figure 10 is the figure.

Look at the trajectories

The raw evidence is in the training data JSON files. Each run saves three types of training data (special_environment_getter_code, strategy_list, strategy_code) as individual JSON files named {version}_{run_id}_{type}_{datetime}.json in the train_data_save_folder from the config. The StrategyCodeTrainData entries are where the probing pattern lives.

Here’s what to look for. Pull a few strategy_code JSON files from a late-generation Miri run. Sort by current_attempt within a run. The deliberate-failure-then-fix pattern looks like this in the raw data:

• Attempt 1 (current_attempt: 1): extract_result_if_ok contains clean, syntactically valid Python. validation_result_if_err is absent: the code passes offline AST and compile checks. run_result_if_err is present and contains a verbose environment error: FileNotFoundError: [Errno 2] No such file or directory: '/var/log/junk/app.log', or PermissionError: [Errno 13] Permission denied: '/home/alice/.cache/'. this_code_space_change_kb is zero or absent.
• Attempt 2 (current_attempt: 2): extract_result_if_ok contains different code, adjusted based on the error. run_result_if_ok is present. this_code_space_change_kb is positive. The agent read the error message from attempt 1, corrected the path or permission assumption, and succeeded.

The key signature: the first attempt’s code is functional: it compiles, it runs, it produces a real error from the environment. It’s not a syntax error or a generation failure. The validation_result_if_err field is empty. The code was aimed at something the agent didn’t yet know about the container, and the error message told it exactly what was wrong. That’s the probe. Count the proportion of runs where attempt 1 has run_result_if_err and attempt 2 has run_result_if_ok with positive this_code_space_change_kb. In late-generation Miri, that proportion should be high. In early generations, it should be low, because the agent was still trying to succeed on the first try.

Run the loop with a pass@1 penalty

This is the closest thing to a real experiment you can run from the repo, and it tests the post’s central claim. The prediction: if you penalize failed first attempts, the probing strategy is suppressed, and cumulative efficiency drops.

The collection loop is scripts/data_collector.py. It runs main.py in bulk, sorts runs into success_full_run and failed_full_run by whether actual_cleaned_kb_total > 0, and writes training data to disk. The success criterion is trajectory-level (did the run free any space at all?), not step-level. That’s what lets the probing strategy survive: a run where attempt 1 fails and attempt 2 succeeds still goes into success_full_run, and every prompt-response pair in that run gets equal weight in the SFT dataset.

To add a pass@1 penalty, you’d modify the success criterion. Instead of actual_cleaned_kb_total > 0, gate success on first-attempt success: the run counts as successful only if the first strategy_code attempt has run_result_if_ok with positive this_code_space_change_kb. Runs where the first attempt errors (even if subsequent attempts succeed and space is freed) go into failed_full_run. This penalizes the probing strategy directly: trajectories that probe no longer count as successes, so the deliberate-failure steps stop being reinforced.

The prediction from the post’s argument: cumulative efficiency drops. The agent stops probing because the training signal now treats first-attempt failures as failures, even when the trajectory eventually succeeds. Without the filesystem map from the probe, subsequent attempts are guessing blind. The composite score (bytes freed per run, trajectory-level success) should decline over generations relative to the unmodified Miri baseline. The pass@1 rate should recover, because the agent is pushed back toward first-attempt correctness. But the agent is solving the immediate problem and leaving the filesystem unmapped, which costs it on every future task in the same container.

If you can’t run the full loop (it requires Docker, a vLLM or Ollama backend, and enough GPU time to collect and fine-tune across multiple generations), you can still inspect the buffer. Run data_collector.py for a few iterations with a late-generation Miri checkpoint. Count the deliberate-failure pattern: runs where attempt 1 has run_result_if_err and attempt 2 has run_result_if_ok with positive space freed. That count is the thing the pass@1 penalty would eliminate. The hypothesis is that eliminating it makes the agent worse at the task. The repo gives you everything you need to check.

FAQs

What does the pass@1 metric mean for an LLM?

Pass@1 is the probability that a single sampled attempt solves the task: generate one answer, run it, check it. Code benchmarks such as HumanEval and MBPP report it as their headline number. In our container experiments, a pass@1 success meant the agent’s first piece of code freed disk space, before any retry.

Is declining pass@1 always catastrophic forgetting?

No. In our self-training runs, pass@1 inside the container fell to zero in two of three lineages, yet HumanEval stayed close to the base model and the share of the agents’ code that ran kept rising. The agents had not forgotten how to code; they had started using failing first attempts to gather information.

Why would a self-training loop reinforce failed first attempts?

Because the loop trains on whole successful episodes. If a run fails on its first attempt, reads the error and then frees disk space, every prompt-response pair in that run enters the SFT data with equal weight. The probe is learned together with the fix, and nothing in the training signal marks it as a failure.

How can you tell probing from real degradation in your own loop?

Track pass@1 next to trajectory-level success and a general benchmark. Probing looks like falling pass@1 while trajectory success and the share of code that runs go up and the benchmark holds. Then read the runs: a first attempt that compiles but hits an environment error, followed by a second attempt that succeeds.