Continual Learning Under Self-Induced Distribution Shift

Fri, 23 Oct 2026 15:00:00 +0800

When an agent’s actions permanently alter its environment, keeping past data preserves contradictions; a sliding window tracks the new reality

Key Takeaways

The Exogenous Shift Fallacy: Standard continual learning assumes the environment shifts externally while task distributions remain passive. In open-ended agent interaction, successful actions permanently mutate the environment, making the distribution shift endogenous and self-induced.
Catastrophic Remembering: Accumulating all historical trajectories in a cumulative replay buffer (Terese) causes policy stagnation not from forgetting, but from averaging over mutually incompatible past states from destroyed worlds.
The Sliding Window Solution: Retaining only the last three generations (\(K=3\)) of environment-validated trajectories (Miri) sustains monotonic improvement on a quarter of the data, achieving over 50% of peak performance without plateauing or collapsing.
Epistemic Hygiene via the Environment: A sliding window does not destroy knowledge; it enforces epistemic hygiene. Valid strategies are naturally re-derived and re-enter the window through live environmental feedback, while obsolete habits age out cleanly.

TL;DR

When an AI agent’s actions permanently alter its environment, standard continual learning assumptions collapse. Cumulative replay buffers hoard trajectories from worlds the agent itself destroyed, causing catastrophic averaging over contradictory states. Testing identical Qwen2.5-7B-Instruct agents in procedural Linux containers reveals that accumulating all history (Terese) plateaus at 65,000 rows, while a \(K=3\) sliding window SFT buffer (Miri) discards ghost states and achieves continuous, monotonic improvement on 4x less data by letting the live environment re-derive valid strategies.

Replay Buffer on Eigenform Continuous Learning Tool Tips

Continual Learning Under Self-Induced Distribution Shift

Key Takeaways

TL;DR