We study LLMs for tabular prediction with mixed text, numeric, and categorical fields. We introduce TabGemma, a schema-agnostic in-context learner that treats rows as sequences and tackles two practical hurdles when adapting pretrained LLMs for tabular predictions: unstable numeric tokenization and limited context size. We propose to canonicalize numbers via signed scientific notation and continue pretraining of a 12B Gemma 3 model with a target imputation objective using a large-scale real world dataset. For inference, we use a compact n-gram-based retrieval to select informative exemplars that fit within a 128k-token window.
On semantically rich benchmarks, TabGemma establishes a new state of the art on classification across low- and high-data regimes and improves monotonically with more context rows. For regression, it is competitive at small sample sizes but trails conventional approaches as data grows. Our results show that LLMs can be effective tabular in-context learners on highly semantic tasks when paired with dedicated numeric handling and context retrieval, while motivating further advances in numeric modeling and long-context scaling.
ICLAD: In-Context Learning for Unified Tabular Anomaly Detection Across Supervision Regimes
Department of Electrical and Computer Engineering, McGill University, Canada(麦吉尔大学电气与计算机工程系,加拿大);Mila - Quebec Artificial Intelligence Institute, Montreal, Canada(魁北克人工智能研究所,蒙特利尔,加拿大)
Algorithmic Recourse of In-Context Learning for Tabular Data
Mohamed bin Zayed University of Artificial Intelligence, United Arab Emirates(阿联酋人工智能马尔代夫 bin Zayed 大学);University of Copenhagen, Denmark(丹麦哥本哈根大学);Renmin University of China, China(中国人民大学);King Abdullah University of Science and Technology, Saudi Arabia(沙特阿拉伯王国科学与技术大学);The Hong Kong University of Science and Technology(Guangzhou), China(广州科技大学(香港))
Accept or Deny? Evaluating LLM Fairness and Performance in Loan Approval across Table-to-Text Serialization Approaches
Saarland University(萨尔兰大学);CISPA Helmholtz Center for Information Security(CISPA海德堡中心 for 信息安全);Max Planck Institute for Software Systems(马克斯·普朗克软件系统研究所)
本文评估三种地区贷款审批数据在不同表到文本序列化方式下的 LLM 零样本与 ICL 表现,发现格式显著影响 F1 与公平性。