Decoupled Framework Design for Test-Time Learning Agents

Fri, 13 Nov 2026 11:00:00 +0800

One task contract runs across different agent harnesses, while verification secrets never reach the harness

TL;DR: Training language model agents across multi-container tasks often stalls because benchmark definitions, prompt formatting, tool schemas, and inference clients are tightly coupled inside monolithic runner scripts. Separating the domain contract (TaskSpec) from the turn-level execution loop (HarnessSpec), the inference wrapper stack (Genner and Backend), and the parallel slot scheduler (Execution) allows any task to run unchanged across in-process orchestrators or containerized third-party agents. This four-layer boundary preserves secret isolation during reward verification and produces uniform supervised fine-tuning (SFT) trajectories for iterative LoRA consolidation.

Reproducible ML Pipelines and Recipe Hash Provenance

Fri, 13 Nov 2026 09:00:00 +0800

Row lineage, recipe hash and atomic publication: the checks behind each training export

TL;DR: Static supervised fine-tuning pipelines treat dataset preparation as a one-off preprocessing script, but continual agent loops regenerate their own fine-tuning corpus at every generation boundary. If a post-collection transform silently drops a metadata key, mutates a prompt without recording lineage, or leaves a partially flushed JSONL file on disk during a process crash, the resulting weight update permanently diverges from its intended specification. In nsl-workbench, src/training_data/transforms.py we close these failure modes by validating a strict 19-field row schema with parent_row_id ancestry checks, compiling sequential TrainingDataTransform stages into a canonical SHA-256 recipe_hash with exact row-level deltas, and enforcing atomic directory publication verified by path canonicality and cryptographic digests before each training pass begins.

Test-Time Learning on Eigenform Continuous Learning Tool Tips

Decoupled Framework Design for Test-Time Learning Agents

Reproducible ML Pipelines and Recipe Hash Provenance