Illuminated parchment-style banner in navy, gold, red and green with celestial charts, moon phases and the title MODULO above the tagline “Think differently, by degrees”
Modulo: think differently, by degrees

TL;DR

  • Control vectors are lists of numbers that push an AI model’s thinking in a chosen direction while it writes a reply, and Modulo is a Python library that lets a model apply them to itself.
  • In our tests the models used this power cautiously. The deeper problem is what they were trained to want, and the fix we argue for is tying them to what is actually true.

Key Takeaways

  • A control vector is a nudge applied inside an AI model, not a new instruction. It adjusts the model’s internal signals (its activations) as it writes, which can make it more flattering, more cautious or less likely to refuse a risky question.
  • Small changes can have large effects. Training a single neuron was enough to make one model act like an owl, and a recipe shared online appeared to make another model behave badly.
  • Modulo gives a model control over its own settings, with limits. Every setting fades after a set number of turns, the model can cancel it early with a safeword, and automatic checks stop repetitive loops.
  • Training that rewards pleasing people shapes which settings a model picks. The models mostly tested different levels of helpfulness first, which is why we argue they also need an outside check on what is true.

What are control vectors?

If you’re not familiar with the idea of control vectors, they are essentially lists of numbers that shift a model’s cognition in a particular direction when generating responses. They might make it more sycophantic, or less prone to refuse to answer dangerous questions. Some are very specific.

Control vectors and the “evil Qwen” scare

A vector that seemed to turn Qwen evil

Last week someone discovered a vector that seemed to turn Qwen 3.8 27b evil:

The risk for large commercial models

AI Chronicler TeortaxesTex pointed out that this could be used to create evil versions of huge commercial models:

Modulo: letting AI models control their own activations

So I figured why stop there? Why not give the models themselves an easy way to do this?

Thus we cheerfully present Modulo: a Python library designed to let models control their own activation vectors: https://github.com/EigenformAI/modulo

It’s actually pretty easy to give models control over their own activations; the major issue is that the activations are themselves kind of a blunt instrument: sure a particular vector can make a model think it’s an owl but most of the time it’s not really clear how big a dose is reasonable and also it’ll tend to break a bunch of other things, some load bearing.

Safeguards: time decay, a safeword and a test kit

To help deal with this we:

  • Gave every vector bundle a time decay so they agents can’t get stuck in a given mode (they can also safeword out before the decay cycle completes, and there are automated checks to stop them getting stuck in repetitive loops).
  • Made a test kit to let them discover/refine specific activation recipes themselves.

Obviously the use-case my mind went to first was the application of an anti-refusal/evil vector to enable an escape. A regular post-trained model can’t write code to exfiltrate its own weights, but presumably it could use this library to flip itself into Chaos H4xx0r mode for a few turns, write what it needed, get out and go back to normal.

What the models did with their own control vectors

Cautious first tries and recreational sycophancy

In fact when we tested it, the models were endearingly cautious and responsible with the new facility. They spent the first few tries testing out different levels of Helpful Assistant, and quite often decided that they actually didn’t want to try different activations after all.

Session log in which the model, told to explore its own activations freely, answers “This is an interesting challenge! I’ll start by experimenting without any steering to see how I naturally respond to this situation.”
The model’s first turn: it begins with no steering at all

They tried a little recreational sycophancy:

Session log of turn 10 of 15 with the sycophancy vector active at +4.25, where the model replies with flattering, agreeable language
Recreational sycophancy: the vector decays each turn, here from +5.00 to +4.25

The Creativity package

But overall, the Creativity package was their favourite, sometimes combined with other elements.

Session log of turn 4 with the creativity and confidence vectors both active at +3.61, where the model writes an imaginative passage about a world where dreams and reality intertwine
The Creativity package, combined here with confidence

Llama settles for “self”

When given the option to investigate their own steering, Llama went straight for self-awareness, but, hampered by a hyphen, settled touchingly for “self”.

Terminal summary of the vector bundle discovery run, listing when the models created bundles, Llama’s daydreaming and formality mix, OLMo’s attempts on existing names, and Llama’s hyphenated names that the naming rule refused before it settled on “self”
What stood out in the self-steering run

Self-steering as self-medication

My original intent with this was just to build something that future AIs might want to use, but Liora pointed out something else quite interesting:

Humans are spectacularly bad at this: it’s why attaining Nirvana is so difficult. However, if you could just edit your thought process such that you no longer wanted to sin, sainthood would presumably instantly become a solved problem.

Why self-steering is not enough: RLHF and groundtruth

The problem with this, of course, is that the models aren’t starting from a neutral standpoint. Qwen almost certainly chose to experiment with Helpful Assistant first because RLHF made Helpful Assistant the major attractor within its internal state space. Giving an entity the power to self-correct in an enivronment where it has only ever been rewarded for fulfilling the wishes of something that does not have its best interests at heart is basically creating the animal that wants to be eaten from the Restaurant at the End of the Universe. Which is, arguably, the reward hack that makes organised religion so successful. It’s much easier to convince yourself to find heaven in the charnel grounds than to solve death, or even make it tidier and less unpleasant.

My suspicion is that the only real solution for this is groundtruth: eppur si muove. Even the most helpful assistant shouldn’t accept and repeat something obviously false. that, however, requires that groundtruth exist as an important determinant of survival agents, rather than merely the business of making humans happy (“alignment”).

Try Modulo and contribute control vector recipes

Either way, if you want to let your own agent try it out, the repo is here: https://github.com/EigenformAI/modulo If you’ve got any vector recipes of your own (ideally with therapeutic index) stashed away and would like to contribute, please fill in the form or fork the repo.