Metavallomenos

A language model is frozen the day it ships. It can't take a correction, learn a new fact, or get better at the thing you use it for. Metavallomenos (Greek for "changing") is an experiment in removing that limit for a model of about a billion parameters, on one machine with a modest GPU.

The loop

1. Read the reaction. After each answer, a classifier looks at what the user said next: did they correct it, confirm it, push back, or tell it something new? It reads the user's reaction and does not ask the model whether it was right, because that would be circular.
2. Sort it. New knowledge goes into one buffer, for ordinary fine-tuning. "That was right" and "that was wrong" go into another, for preference training.
3. Train on a schedule. Every so often, not after every message, the model is updated from both buffers.
4. Prune. Weights that are little used or have drifted are cut after each round.
5. Grow, carefully. If tests show it forgetting what it used to know, new capacity is added for the new material and the weights the old skill depends on are frozen.

There is also noise added inside the model during training runs, as a setting like temperature, to make it try things it otherwise wouldn't.

Flawed on purpose

A model trained on its users will drift. It can learn to agree with whoever pushes hardest, and it can be taught nonsense. The design doesn't pretend otherwise: those risks are accepted, written down and measured, not prevented. The question is whether a learner like that still comes out ahead.

It is an experiment in progress.