Publication
A Sovereign, Open-Source Foundation Model for German and English
Benedikt Droste; David Fitzek; Ruben Härle; Lukas Henrik Helff; Maximilian Idahl; Alex Jude; Abbas Goher Khan; Maurice Kraus; Timm Ruland; Richard Rutmann; Sebastian Sztwiertnia; Markus Frey; Daniil Gurgurov; Jan Pfister; Tom Röhr; Sebastian von Rohrscheidt; Jörg Bienert; Nicolas Flores-Herr; Simon Gottschalk; Andreas Hotho; Kristian Kersting; Joachim Köhler; Alexander Löser; Wolfgang Nejdl; Simon Ostermann; Jan Plogsties; Björn Plüster; Patrick Putzky; Mehdi Vali; Michael Fromm; Max Lübbering
In: Computing Research Repository eprint Journal (CoRR), Vol. abs/2607.09424, Pages 1-61, arXiv, 2026.
Abstract
We present Soofi S 30B-A3B, a sovereign, open-source Mixture-of-Experts (MoE) hybrid Mamba
Transformer foundation model for German and English. Its hybrid design activates only 3B of 30B
parameters per token and keeps the inference cache near-constant as context grows, giving it a
decisive throughput advantage over dense models for long-context, high-concurrency deployment.
Pretrained on roughly 27 trillion tokens with deliberately up-weighted German, Soofi S matches
dense 14 to 27B models on aggregate English and German benchmarks while achieving the best
code aggregates in both languages among 16 open base models, and outperforms every European
sovereign baseline in our comparison, including ones far larger in active parameters. Among fully
open models, Soofi S obtains the highest English and German evaluation scores, ahead of Olmo 3 32B
and Apertus 70B. Soofi S was built end-to-end on the German Industrial AI Cloud, a sovereign HPC-
scale AI infrastructure operated by Deutsche Telekom in Munich. Soofi S will be released under
highly permissive, open-access terms: weights, selected intermediate checkpoints1, full per-source
data accounting, hyperparameters, and training and evaluation code. Where source licenses permit,
data-construction artifacts are released under permissive licenses; commercially licensed sources are
documented with aggregate statistics and exact mixture accounting.
