NeFut Logo NeFut
Admin Login

[CS.AI] Xiaomi-TabLDM: A Tabular Foundation Model Technical Report

Published at: 2026-09-04 22:00 Last updated: 2026-09-05 12:23
#algorithm #AI #Machine Learning

We introduce Xiaomi-TabLDM, a tabular large model for classification and regression that leverages in‑context learning, achieving higher prediction accuracy without task‑specific fine‑tuning. The model is pretrained solely on synthetic data generated by structural causal models (SCMs), enabling more flexible context usage and efficient capacity scaling.\ \ It sets new performance standards: ranking first on OpenML‑CTR23 and second on regression tracks of TALENT, TabArena, and BCCO, demonstrating consistently strong regression results across four complementary benchmark suites. At the same time, Xiaomi-TabLDM offers a favorable performance‑efficiency trade‑off; for example, on TabArena regression it attains the second‑highest Elo while using 82% less training time and 68% less prediction time than the top‑ranked TabFM.\ \ To enhance pretraining, Xiaomi-TabLDM expands the coverage and diversity of synthetic tabular data and adopts a three‑stage training pipeline together with dual‑stream feature grouping, lightweight Attention Residual, and sparse Mixture‑of‑Experts, allowing the model to learn richer feature interactions and specialize experts across diverse tasks.\ \ During inference, the model supports test‑time compute scaling: allocating additional computation at inference consistently improves predictive performance over the base model.\ \ Review

Original Source: https://arxiv.org/abs/2609.03880

[h] Back to Home