Transformers Can Do Bayesian Inference

20 December 2021

Samuel G. Müller

Noah Hollmann

Sebastian Pineda Arango

Papers citing "Transformers Can Do Bayesian Inference"

30 / 30 papers shown

Title
How Well Does Your Tabular Generator Learn the Structure of Tabular Data? Xiangjian Jiang Nikola Simidjievski M. Jamnik LMTD 86 0 0 13 Mar 2025
TimePFN: Effective Multivariate Time Series Forecasting with Synthetic Data Ege Onur Taga M. E. Ildiz Samet Oymak AI4TS 57 2 0 22 Feb 2025
Vector-ICL: In-context Learning with Continuous Vector Representations Yufan Zhuang Chandan Singh Liyuan Liu Jingbo Shang Jianfeng Gao 52 3 0 21 Feb 2025
Distribution Transformers: Fast Approximate Bayesian Inference With On-The-Fly Prior Adaptation George Whittle Juliusz Ziomek Jacob Rawling Michael A. Osborne 91 2 0 04 Feb 2025
Amortized Bayesian Experimental Design for Decision-Making Daolang Huang Yujia Guo Luigi Acerbi Samuel Kaski 44 2 0 03 Jan 2025
N-Gram Induction Heads for In-Context RL: Improving Stability and Reducing Data Needs Ilya Zisman Alexander Nikulin Andrei Polubarov Nikita Lyubaykin Vladislav Kurenkov Andrei Polubarov Igor Kiselev Vladislav Kurenkov OffRL 50 1 0 04 Nov 2024
PViT: Prior-augmented Vision Transformer for Out-of-distribution Detection Tianhao Zhang Zhixiang Chen Lyudmila Mihaylova 129 0 0 27 Oct 2024
Amortized Probabilistic Conditioning for Optimization, Simulation and Inference Paul E. Chang Nasrulloh Loka Daolang Huang Ulpu Remes Samuel Kaski Luigi Acerbi AI4CE 49 4 0 20 Oct 2024
In-context learning and Occam's razor Eric Elmoznino Tom Marty Tejas Kasetty Léo Gagnon Sarthak Mittal Mahan Fathi Dhanya Sridhar Guillaume Lajoie 39 1 0 17 Oct 2024
Zero-Shot Learning of Causal Models Divyat Mahajan Jannes Gladrow Agrin Hilmkil Cheng Zhang M. Scetbon 39 1 0 08 Oct 2024
XLand-100B: A Large-Scale Multi-Task Dataset for In-Context Reinforcement Learning Alexander Nikulin Ilya Zisman Alexey Zemtsov Viacheslav Sinii 107 4 0 13 Jun 2024
Pretraining Decision Transformers with Reward Prediction for In-Context Multi-task Structured Bandit Learning Subhojyoti Mukherjee Josiah P. Hanna Qiaomin Xie Robert Nowak 72 2 0 07 Jun 2024
From Words to Actions: Unveiling the Theoretical Underpinnings of LLM-Driven Autonomous Systems Jianliang He Siyu Chen Fengzhuo Zhang Zhuoran Yang LM&Ro LLMAG 44 2 0 30 May 2024
On Understanding Attention-Based In-Context Learning for Categorical Data Aaron T. Wang William Convertino Xiang Cheng Ricardo Henao Lawrence Carin 58 0 0 27 May 2024
LaT-PFN: A Joint Embedding Predictive Architecture for In-context Time-series Forecasting Stijn Verdenius Andrea Zerio Roy L.M. Wang BDL AI4TS AI4CE 34 2 0 16 May 2024
In-Context Freeze-Thaw Bayesian Optimization for Hyperparameter Optimization Herilalaina Rakotoarison Steven Adriaensen Neeratyoy Mallik Samir Garibov Eddie Bergman Frank Hutter AI4CE 34 9 0 25 Apr 2024
MotherNet: Fast Training and Inference via Hyper-Network Transformers Andreas Müller Carlo Curino Raghu Ramakrishnan LMTD 38 2 0 14 Dec 2023
Looped Transformers are Better at Learning Learning Algorithms Liu Yang Kangwook Lee Robert D. Nowak Dimitris Papailiopoulos 24 24 0 21 Nov 2023
Efficient Bayesian Learning Curve Extrapolation using Prior-Data Fitted Networks Steven Adriaensen Herilalaina Rakotoarison Samuel G. Müller Frank Hutter BDL 31 19 0 31 Oct 2023
In-Context Learning Learns Label Relationships but Is Not Conventional Learning Jannik Kossen Y. Gal Tom Rainforth 34 27 0 23 Jul 2023
Benchmarking Bayesian Causal Discovery Methods for Downstream Treatment Effect Estimation Chris C. Emezue Alexandre Drouin T. Deleu Stefan Bauer Yoshua Bengio CML 35 2 0 11 Jul 2023
Meta-Learned Models of Cognition Marcel Binz Ishita Dasgupta A. Jagadish M. Botvinick Jane X. Wang Eric Schulz 30 25 0 12 Apr 2023
General-Purpose In-Context Learning by Meta-Learning Transformers Louis Kirsch James Harrison Jascha Narain Sohl-Dickstein Luke Metz 34 72 0 08 Dec 2022
What Can Transformers Learn In-Context? A Case Study of Simple Function Classes Shivam Garg Dimitris Tsipras Percy Liang Gregory Valiant 24 449 0 01 Aug 2022
Transformer Neural Processes: Uncertainty-Aware Meta Learning Via Sequence Modeling Tung Nguyen Aditya Grover BDL UQCV 19 99 0 09 Jul 2022
Sample-Efficient Optimisation with Probabilistic Transformer Surrogates A. Maraval Matthieu Zimmer Antoine Grosnit Rasul Tutunov Jun Wang H. Ammar 30 2 0 27 May 2022
Towards Learning Universal Hyperparameter Optimizers with Transformers Yutian Chen Xingyou Song Chansoo Lee Z. Wang Qiuyi Zhang ... Greg Kochanski Arnaud Doucet MarcÁurelio Ranzato Sagi Perel Nando de Freitas 24 63 0 26 May 2022
Benchmarking Simulation-Based Inference Jan-Matthis Lueckmann Jan Boelts David S. Greenberg P. J. Gonçalves Jakob H. Macke 104 185 0 12 Jan 2021
Bayesian Model-Agnostic Meta-Learning Taesup Kim Jaesik Yoon Ousmane Amadou Dia Sungwoong Kim Yoshua Bengio Sungjin Ahn UQCV BDL 202 498 0 11 Jun 2018
Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks Chelsea Finn Pieter Abbeel Sergey Levine OOD 332 11,684 0 09 Mar 2017