3 October 2026HN · Language modelsCurated

Kolibri: a sovereign German language model with open weights

Summary

On German Unity Day 2026, Aleph Alpha released Kolibri, a language model for German and English with open weights under the Apache 2.0 licence. The mixture-of-experts model has 78 billion parameters, of which around 3.5 billion compute per token, and handles up to one million tokens of context. It is intended for public administration, industry and aerospace, which want to run models on their own servers and be able to trace them.

What is Kolibri?

Kolibri is a large language model from the Heidelberg company Aleph Alpha, trained from scratch in Germany and Finland. It is a mixture-of-experts model: each of its 50 layers contains 384 small sub-networks, the experts, and a router selects six of them for each token, plus one shared expert. The model therefore computes like a small one but needs the memory of a large one, because all experts must be loaded.

The weights are on Hugging Face. Aleph Alpha keeps the rights to the training code; the weights and configuration files may be used freely under Apache 2.0.

What “sovereign” means here

Aleph Alpha means two things by it. First, origin: developed and trained under European and German law, with a traceable supply chain from the training data to the evaluation. Second, use: an authority or a supplier can run the model in its own data centre without sending data to an outside service, and nobody can change or switch it off from outside.

Sovereign does not mean that nothing from outside Europe went into it, though. According to the model card, training texts were rephrased with Gemma 4 and Mistral-NeMo, and Qwen3-32B helped with quality filtering.

Why German is more than a translation here

Around 21 percent of the training data is German; translations make up only 6 percent. The custom tokenizer splits German compound words much less often: “Bundesverfassungsgericht” is two tokens for Kolibri and six for GPT-5’s tokenizer. Fewer tokens mean fewer computing steps and more German text in the same context window. The model also reasons in German on German questions instead of switching to English internally.

Another special feature: Kolibri was specifically trained to say “I don’t know” when an answer is not in the documents provided. For applications that generate answers from their own documents, that matters more than a few percentage points in a benchmark.

Ideas

Mixture of experts lowers the computation per token, but not the memory requirement.
A tokenizer built for German compound words saves computing time and context.
A model can be trained to admit missing evidence.
Open weights allow operation without outside cloud services.

Insights

Sovereignty in AI concerns origin, operation and traceability, not just the company’s headquarters.
Language-specific training improves a model in places that general benchmarks hardly show.

Facts

Kolibri has 78.1 billion parameters, around 3.5 billion of them active per token.
The context natively covers 262,144 tokens and was validated up to 1,048,576 tokens.
It was trained on around 24 trillion tokens on 768 NVIDIA B200 GPUs.
On Hacker News the announcement reached 704 points on 3 October 2026.

References

Critique

The benchmark comparisons come from Aleph Alpha itself; independent measurements were only just beginning at the time of release.
With around 78 GB of weights in FP8, Kolibri does not fit on typical home graphics cards without offloading experts to system memory.

Remarks

Tejas Kumar’s analysis evaluates the 189-page technical report and tests the tokenizer on the German constitution.

Recommendations

Test Kolibri with your own German documents rather than only with benchmarks.
For local operation, plan enough system or graphics memory for all 78 billion parameters.
When answers are drawn from documents, check whether the model actually admits missing evidence.

Read the original article

Search the Web Archive