Publikation
Tydra: An Efficient Hybrid Model for Tabular Data
Mieszko Komisarczyk; Saurabh Mathur; Maurice Kraus; Sriraam Natarajan; Kristian Kersting
In: Computing Research Repository eprint Journal (CoRR), Vol. abs/2608.21199, Pages 1-8, arXiv, 2026.
Zusammenfassung
Transformer-based tabular foundation models such as TabPFN
achieve strong predictive performance but incur quadratic
computational cost with context length. On the other hand,
subquadratic SSM-based alternatives such as Hydra trade
away accuracy for efficiency. To balance both, we introduce
Tydra, a hybrid Transformer–State Space Model (SSM) archi-
tecture for tabular in-context learning that interleaves attention
and SSM layers. Across 30 OpenML datasets, Tydra reduces
inference time by 30% relative to TabPFN while retaining
much of its predictive performance. Tydra also outperforms an
approximately ten-times-larger Hydra model while providing
faster inference. The results indicate that hybrid architectures
are a promising direction for tabular foundation models.