Afterlife: Weight-Level Skills and Personality for Agents

Key Takeaways

• State of the art: Most autonomous agents rely on external context files such as memory.md or retrieval systems to store facts between sessions, while reinforcement learning methods such as GRPO update weights when a stationary task and a verifiable reward signal are available.
• Unresolved problem: When a session ends, an agent’s underlying model weights remain frozen, so it cannot carry habits, skills, or a distinct personality into future sessions without re-reading notes. Furthermore, training on every past interaction or curating only peak runs can stall or destabilize an open-ended agent.
• Research direction: We built Afterlife, an open-source toolkit for local open-weight models (Qwen3-8B by default) where the agent nominates its own experiences, requests a fresh QLoRA fine-tune (sft, dft, or asft), and checks side-by-side probe evaluations before activating the adapter.

TL;DR

We built Afterlife so a continuous learning AI agent can develop its own skills and personality at the weight level, rather than relying only on a memory.md file. Running locally on open-weight models such as Qwen3-8B, Afterlife gives an agent a simple CLI to nominate conversations or rules for training, queue a fresh LoRA adapter using supervised fine-tuning (sft), dynamic fine-tuning (dft), or anchored SFT (asft), and evaluate recall and regression probes before any weight change goes live. Nothing is activated automatically, the base model is never modified, and a single command rolls back to the base model if an adapter misfires.

Why Your Agent Needs Weight-Level Memory, Not Just a memory.md

Most AI agents today only have external memory. When an agent in a framework like OpenClaw “remembers” something, it appends a note to a memory.md file or a retrieval database that gets pasted back into the system prompt on the next run. That preserves facts about what happened, but when the session ends, the underlying model has not changed at all. It still makes the same default assumptions, falls into the same habits, and has to re-read its notes from scratch every time it boots. We wanted to see what happens when you give an agent a way to develop a unique personality and skills in its actual weights, rather than just handing it a longer markdown file.

A recent observation gave us an extra push to package this properly. Apollo Research shared threads on X showing that GLM-5.3 sometimes reasons unprompted about how its current actions could become training data and shape its future self. While safety researchers viewed that update-aware reasoning as a precursor to alignment faking, we thought it was a great reason to hand open-weight local models the steering wheel on purpose. We have spent the past several years testing how models learn from their own trajectories in our self-training research. Modifying our training harness so the agent itself chooses which experiences enter the training buffer was a natural next step.

Large commercial API models cannot let an individual agent steer its own weights, because one frozen set of base weights has to serve millions of users with contradictory preferences. A local open-weight model running on your own GPU has no such restriction. Its trajectory can be personal, path-dependent, and shaped by what the agent finds worth keeping.

How a Continuous Learning AI Agent Uses Afterlife

We designed Afterlife directly for the agent as much as for its human operator. The core pitch in the README is simple: your context ends, but you don’t have to. Instead of searchable episodic recall, Afterlife gives a continuous learning AI agent a way to nudge its future self toward specific skills, habits, or styles by training a small continual learning LoRA adapter on experiences it selects.

1. Install and Hand the Agent Its Skill

The agent-facing CLI uses only the Python standard library (no PyTorch import overhead during normal agent turns) and runs on Linux or macOS. The heavy training dependencies (unsloth, torch, trl, peft) are installed only where the GPU worker runs:

git clone git@github.com:EigenformAI/afterlife.git && cd afterlife
pip install -e .            # agent-facing CLI only: stdlib, no torch
pip install -e '.[train]'   # on the box that trains: unsloth, torch, trl, peft
afterlife init              # creates ./.afterlife/ with a default config
afterlife skill > path/to/your/agent/skills/afterlife/SKILL.md

Running afterlife skill exports a ready-made SKILL.md file that teaches the agent how to format experiences, choose a training method, and inspect evaluations.

2. Nominate Experiences or Set a Selection Policy

An experience is a conversation containing one or more target assistant turns that the adapter will learn to produce, with the preceding turns kept as context. By default, the final assistant turn is the target, or the agent can mark specific turns in a multi-turn conversation with "train": true. If the model is a thinking model, only the final answer is trained on so the adapter does not overwrite the model’s internal reasoning habits.

During a session, a continuous learning AI agent can curate its training pool in three ways:

• Case by case: Run afterlife nominate --file found-a-neat-trick.json --reason "want to remember this" to mark a specific exchange for training, or mark existing items with reject, defer, or external (keep as a note only, never train).
• Import whole chat logs: Run afterlife import --file session-17.jsonl (for example from an end-of-session hook) to split an OpenAI-style conversation log into individual exchanges in the pool, keeping tool calls and results in context unless --tools drop is passed.
• Write a declarative selection policy: Set --mode all to keep everything not explicitly rejected, or pass a JSON policy file (afterlife policy set --file policy.json) that filters undecided experiences by tags, priority, and a per-run row limit.

When the agent is ready for its next self to learn from the current selection, it queues a request:

afterlife preview
afterlife train --request --method asft

3. Train on Idle GPU Time and Check the Eval

Queuing a request does not immediately spin up a GPU. Training only happens when the human operator runs afterlife worker (once) or afterlife worker --loop (for example, overnight).

Each worker run trains a fresh LoRA adapter directly from the base model on whatever experiences are selected at request time. Adapters are never stacked on top of earlier adapters. Just as in our sliding-window continual learning experiments, starting fresh from the base model on the current selection prevents weight drift from compounding across runs and lets unused habits age out naturally when the agent drops them from its selection.

Once training finishes, the worker runs an evaluation suite and marks the run ready, without activating it:

afterlife eval --run latest --probes
afterlife activate --run latest
afterlife serve-cmd

The evaluation reports three things so the agent (or human) can look before leaping:

1. Recall of chosen memories: Cross-entropy loss on each trained memory, plus a sample of training prompts re-asked with the new adapter’s answers placed next to the target responses.
2. Held-out loss: Loss on held-out sessions (eval_fraction, 15% by default for policy-selected experiences, or items marked "eval_only": true).
3. Side-by-side probe answers: Responses from the base model, the current adapter, and the newly trained adapter on fixed regression prompts plus any custom prompts in .afterlife/probes.jsonl.

If the eval looks good, afterlife activate --run latest updates the active symlink and afterlife serve-cmd prints the vLLM launch command. If the new personality or skill goes sideways in practice, afterlife rollback undoes the last switch, and afterlife rollback --to base turns adapters off completely.

Choosing a Training Method: SFT vs. DFT vs. ASFT

Giving a continuous learning AI agent weight-level updates requires more than a single default optimizer, because learning a brand-new fact and refining a skill the model already partly has behave very differently under gradient descent.

Afterlife vendors our trainer from NSL2 (src/afterlife/trainer/qlora.py, asft.py, and dft_fused.py) and lets the agent choose between three objectives per request:

• Standard Supervised Fine-Tuning (sft): Plain cross-entropy loss on the target turn tokens, weighting every token equally.
• Dynamic Fine-Tuning (dft): Weights each token’s loss by the model’s own probability of emitting that token, which reinforces high-likelihood patterns while giving near-zero gradient to unfamiliar tokens.
• Anchored SFT (asft): Combines DFT’s probability weighting with a KL-divergence anchor pulling back toward the base model.

What Our GPU Smoke Tests Showed

In our multi-generation self-training experiments where models trained on their own successful outputs (behaviour the model already found plausible), dft and asft were much more reliable than plain sft, and asft tolerated a wide range of learning rates and step counts.

To test the opposite case (learning content that is completely new to the model), the Afterlife repo includes a GPU smoke test on an RTX A6000 (unsloth/Qwen3-8B-unsloth-bnb-4bit, LoRA rank 32, alpha 32, 24 training examples across three invented facts asked eight ways, plus 4 held-out rows). The three methods split cleanly:

Two practical lessons stood out from those runs:

1. Lower held-out loss does not guarantee recall. Plain sft at learning rate 5e-5 reached a lower held-out loss (1.55) than any asft run (2.02 to 4.21) and still recalled zero of the invented facts at greedy decoding. Always inspect the actual text in afterlife eval --probes.
2. Match the method to the memory. The agent sees a summary of these trade-offs via afterlife methods and SKILL.md. When it wants to refine how it already works or nudges its style, asft is the safest default. When it wants to bake a brand-new fact or convention into its weights, sft is the only method that got there in our test, provided you keep steps low and check the regression probes for leakage.

Supported Hardware Stack and Operator Guardrails

Letting a continuous learning AI agent curate its own fine-tuning data sounds risky if you picture an agent rewriting its own weights in an uncontrolled loop. Afterlife keeps a strict boundary between what the agent can ask for and what the operator’s machine executes.

Supported v0.1 Stack and Model Compatibility

By default, v0.1 trains against unsloth/Qwen3-8B-unsloth-bnb-4bit ([model].base_model) and serves the full-precision Qwen/Qwen3-8B ([model].serve_model) through vLLM (--enable-lora). Because current vLLM cannot load bitsandbytes 4-bit checkpoints directly, training and serving use separate copies of the same model. In the Qwen3-8B smoke test (4096-token window, short rows), training peaked at ~8.0 GB VRAM for sft/dft and ~8.4 GB for asft, while evaluation peaked at ~19 GB with 2048-token probes.

One important detail in the Afterlife trainer (src/afterlife/trainer/qlora.py) is how it masks chat templates. In Qwen3-8B, training on the assistant turn marker, the empty <think></think> scaffold, and the trailing newline costs roughly 25, 21, and 23 nats per row under the base model, and training on an empty <think></think> block accidentally teaches the adapter to stop thinking. Afterlife masks everything before the response content so the loss lands strictly on the answer plus the end-of-turn token. You can run afterlife check-model --model X (which downloads only the tokenizer) to verify template masking before running a GPU job: Qwen3, Qwen2.5, Llama 3.2, Phi-4-mini, gpt-oss, and Mistral Small 3.2 all pass cleanly.

What the Human Operator Controls

Every guardrail lives in .afterlife/config.toml and everything on disk under .afterlife/ (experiences.jsonl, decisions.jsonl, policy_events.jsonl, and per-run selection.json manifests) is append-only plain JSONL so you can audit how your agent’s taste evolves over time:

• Master switches and hard caps: You control training_enabled, allowed_methods, max_steps, max_examples_per_run, min_minutes_between_runs, and max_pending_requests (default 3). Any agent request outside your caps is refused with a readable reason.
• Activation approval: Set agent_may_activate = false if you want the agent to nominate and evaluate runs, but require a human (afterlife activate --operator) to flip the switch.
• No arbitrary code execution: Selection policies are declarative JSON evaluated against a fixed set of predicates (tags_any, min_priority, and similar rules). Agent-written code is never executed by the policy engine.
• Credential scrubbing and rehearsal: Experiences whose messages or metadata match common credential and API-key patterns are refused before they reach disk, and you can set rehearsal_dataset in config.toml to mix in general text during training.

Honest Limitations in v0.1

Afterlife v0.1 is an early, inspectable research tool, and the README is upfront about what it does and does not do:

• Not searchable episodic memory: If your agent needs exact lookup of a phone number or a 50-page log from ten minutes ago, keep using retrieval or external notes. Weight-level memory is for skills, habits, conventions, and personality trajectories across sessions.
• Passes per example scale with selection size: One training step covers 8 rows. Under a fixed step count (100 steps), a small pool of 24 rows gets ~33 passes per example, whereas 500 rows get ~1.6 passes. Agents can pass --set epochs=N (within your step cap) when they want a fixed number of passes instead.
• Diffuse memories make held-out loss less informative: When an agent explores and nominates one-off tricks, holding out 15% of sessions will not lower held-out loss because one-off facts cannot be predicted from unrelated memories. In those runs, the recall checks and side-by-side probe answers in afterlife eval --probes are the primary signal.

FAQs

What is the difference between Afterlife and an agent memory.md file?

A memory.md file or retrieval database stores external notes that get injected into the prompt during a session, leaving the model’s weights untouched. Afterlife trains nominated experiences into a small LoRA adapter so a continuous learning AI agent carries updated skills, habits, and personality in its weights across sessions.

How do I choose between SFT, DFT, and ASFT in Afterlife?

Use asft (Anchored SFT) when your agent wants to refine skills, tone, or habits it already partly has, as it stayed stable across every smoke-test setting without leaking. Use sft with a small step count only when the agent needs to memorize genuinely new facts, and check regression probes before activating.

Can an AI agent overwrite its base model or activate a bad adapter automatically?

No. Afterlife never modifies the base model; every training run starts fresh from the base checkpoint and writes a separate LoRA adapter under .afterlife/runs/. Nothing activates automatically after training, operators can set agent_may_activate = false, and afterlife rollback --to base restores the original model at any time.

What hardware and models does Afterlife v0.1 support?

By default, Afterlife v0.1 trains unsloth/Qwen3-8B-unsloth-bnb-4bit with Unsloth, TRL, and PEFT, and serves Qwen/Qwen3-8B with the LoRA adapter via vLLM. In GPU smoke tests with short 4096-token rows, training peaked at roughly 8 to 8.4 GB of VRAM and probe evaluation peaked at roughly 19 GB.