Meta-Learning in Self-Training LLM Loops

Mon, 09 Nov 2026 09:00:00 +0800

A fast inner loop adapts to each task while a slow outer loop changes how the inner loop starts

TL;DR: Meta-learning optimises the learning procedure rather than the solution: an inner loop solves a task, and an outer loop, trained on how well the inner loop adapted, changes the initialisation, update rule or representation it starts from. Schmidhuber’s 1987 thesis and his fast-weight networks put both loops inside one system, and MAML later made the outer loop a second-order gradient. Self-training LLM loops run the same two-timescale structure externally: in-context adaptation within an episode, then environment-filtered training of fresh LoRA adapters between episodes.

Agent Loops on Eigenform Continuous Learning Tool Tips

Meta-Learning in Self-Training LLM Loops