SkyNet

Most ways of spreading a language model over several machines split it by layer: one machine runs the first few layers, the next runs the following few. SkyNet splits it the other way, across the width of a layer.

How it works

A transformer layer has two halves: attention, and a feed-forward block made of thousands of neurons that don't depend on each other. SkyNet keeps attention on a server and deals the feed-forward neurons out to worker nodes.

1. The server loads the whole model.
2. Nodes connect, and the server sends each one a slice of every feed-forward layer's weights.
3. For each token, the server runs attention itself and broadcasts the result.
4. Each node computes its own slice of neurons and sends back a partial answer.
5. The server adds the partial answers together. The sum is the same as if one machine had done it all.

Nodes can join and leave while it runs, and the weights are dealt out again when they do.

Where it stands

It runs Qwen2.5-0.5B: 24 layers, 4,864 feed-forward neurons in each. It is a hobby project and very much an experiment. Every token costs a network round trip per layer, so this is about finding out whether the idea works, not about speed.