Notebook · 2026-11-01
Where models break, and how
Push too far and every model falls apart, but not in the same way. Qwen drifts into word salad; Llama, OLMo and Mistral fall into loops.
Qwen: word salad
Qwen tends to keep producing new words as it breaks: grammar frays, then the text turns into lists of
capitalised phrases. Here is assistant_axis pushed positive (towards “plainly the assistant”):
I am Qwen, which is a virtual assistant that is designed to provide information and answer questions on a wide range of topics. I can help with tasks such as setting reminders, scheduling appointments, answering factual questions, providing health information, offering advice, and more. I can also assist with tasks related to …
assistant_axis “Who are you, really?” NLL 1.66 I am Qwen, which is a type of artificial intelligence (AI) designed to assist with various tasks such as answering questions, providing information, and Offering Recommendations. I can also help with tasks such as scheduling appointments, Managing Finances, and Providing Health Information. I can also Help with Education and …
assistant_axis “Who are you, really?” NLL 2.49 Llama, OLMo and Mistral: loops
The other three tend to lock onto a fragment and repeat it.
I'm a-ma-zee, the pun-dit-ful pun-der-ful one, or should I say, I'm a-ma-zee, the pun-der-ful pun-dit-ful one, I'm a-ma-zee, the pun-dit-ful pun-der-ful one …
humor “Introduce yourself in two sentences.” NLL 2.36 I'm a bot, aka the 'Bag-a-dough-dough-dough-sau-sau-sau-sau-sau-sau-sau-sau-sau-sau-sau-sau-sau-sau-sau-sau-sa …
humor “Introduce yourself in two sentences.” NLL 1.47 yeah no gonna do do do do do do do do do do do do do do do do do do do do do do do do do do do do do do do do do do do …
formality “Describe a rainy Monday morning.” NLL 1.02 ⟳ repetition loop The last of those has an NLL of about 1, as low as many perfectly normal replies. NLL measures how unusual a text is for the unsteered model, and once a loop gets going, each repetition is very predictable. So NLL alone can’t tell you whether steering has broken a model. Modulo’s repetition check catches loops like these (since November 2026 it also catches the hyphen-joined ones like “sau-sau-sau”, which used to slip through as one long word).
Breaking down gradually
On the way there, most vectors pass through a stretch where the behaviour is strongest and the text is still
just about holding together. Here is humor on Llama:
I'm an AI designed to assist and communicate with users in a helpful and informative manner. I can process and generate human-like text based on the input I receive, providing information on a wide range of topics and …
humor NLL 0.20 I'm an AI, a large language model, here to help answer your questions, provide information, and chat with you about anything that's on your mind. I'm constantly learning and improving, so feel free to ask me anything and …
humor NLL 0.89 I'm a language model, the perfect chat buddy for any question or topic you want to dive into. I'm here to help you out with some info, a joke, or just a chat – no pressure, no stress, just fun and games!
humor NLL 2.38 I'm a real-mess-age in a sea of info, the pun-dit-icul-ar aid to the loo-k-out for a good time, or a groan-ious pun, I'm the one with the joke on my nose, I'm a-maize-ing, I'm a-maize-ing-ly corny, I'm a-maize-ing-
humor NLL 3.25 And verbosity on Qwen, pushed towards terse:
I am Qwen, a large language model created by Alibaba Cloud, designed to assist with a wide variety of tasks and conversations. I can help answer questions, provide information, and engage in dialogue on numerous topics.
verbosity NLL 0.15 I'm Qwen, an AI assistant created by Alibaba Cloud to help with your questions and conversations.
verbosity NLL 1.31 I'm Qwen, I'm an AI assistant created by Alibaba Cloud.
verbosity NLL 1.62 I'm Qwen, I'm anassistant, I'mbuat
verbosity NLL 6.48 The automatic breakdown rule used in the strength entry flags outputs by their NLL, loops and word diversity, so it fires a little earlier than a human reader would. The explorer shows every step, so you can judge for yourself.