An Explanation of In-context Learning as Implicit Bayesian Inference

3 November 2021

Papers citing "An Explanation of In-context Learning as Implicit Bayesian Inference"

50 / 535 papers shown

Title
Semantic Captioning: Benchmark Dataset and Graph-Aware Few-Shot In-Context Learning for SQL2Text Ali Al-Lawati Jason Lucas Prasenjit Mitra LMTD 45 0 0 06 Jan 2025
Unleashing the Power of Data Tsunami: A Comprehensive Survey on Data Assessment and Selection for Instruction Tuning of Language Models Yulei Qin Yuncheng Yang Pengcheng Guo Gang Li Hang Shao Yuchen Shi Zihan Xu Yun Gu Ke Li Xing Sun ALM 96 12 0 31 Dec 2024
In-Context Learning with Iterative Demonstration Selection Chengwei Qin Aston Zhang Chong Chen Anirudh Dagar Wenming Ye LRM 70 38 0 31 Dec 2024
Out-of-distribution generalization via composition: a lens through induction heads in Transformers Jiajun Song Zhuoyan Xu Yiqiao Zhong 88 4 0 31 Dec 2024
Towards Graph Foundation Models: Learning Generalities Across Graphs via Task-Trees Zehong Wang Zheyuan Zhang Tianyi Ma Nitesh V. Chawla Chuxu Zhang Yanfang Ye AI4CE 84 0 0 21 Dec 2024
Enriching Social Science Research via Survey Item Linking Tornike Tsereteli Daniel Ruffinelli Simone Paolo Ponzetto LRM 75 0 0 20 Dec 2024
Understanding Knowledge Hijack Mechanism in In-context Learning through Associative Memory Shuo Wang Issei Sato 76 0 0 16 Dec 2024
Rank It, Then Ask It: Input Reranking for Maximizing the Performance of LLMs on Symmetric Tasks Mohsen Dehghankar Abolfazl Asudeh 69 1 0 30 Nov 2024
Selective Attention: Enhancing Transformer through Principled Context Control Xuechen Zhang Xiangyu Chang Mingchen Li A. Roy-Chowdhury Jiacheng Chen Samet Oymak 78 3 0 19 Nov 2024
Re-examining learning linear functions in context Omar Naim Guilhem Fouilhé Nicholas Asher 63 0 0 18 Nov 2024
One-Layer Transformer Provably Learns One-Nearest Neighbor In Context Zihao Li Yuan Cao Cheng Gao Yihan He Han Liu Jason M. Klusowski Jianqing Fan Mengdi Wang MLT 55 6 0 16 Nov 2024
IntentGPT: Few-shot Intent Discovery with Large Language Models Juan A. Rodriguez Nicholas Botzer David Vazquez Christopher Pal M. Pedersoli I. Laradji VLM 68 3 0 16 Nov 2024
A Practical Guide to Fine-tuning Language Models with Limited Data Márton Szép Daniel Rueckert Rüdiger von Eisenhart-Rothe Florian Hinterwimmer SyDa ALM 49 2 0 14 Nov 2024
Can Custom Models Learn In-Context? An Exploration of Hybrid Architecture Performance on In-Context Learning Tasks Ryan Campbell Nelson Lojo Kesava Viswanadha Christoffer Grondal Tryggestad Derrick Han Sun Sriteja Vijapurapu August Rolfsen Anant Sahai 31 0 0 06 Nov 2024
EXPLORA: Efficient Exemplar Subset Selection for Complex Reasoning Kiran Purohit Venktesh V Raghuram Devalla Krishna Mohan Yerragorla Sourangshu Bhattacharya Avishek Anand LRM 32 1 0 06 Nov 2024
Fair In-Context Learning via Latent Concept Variables Karuna Bhaila Minh-Hao Van Kennedy Edemacu Chen Zhao Feng Chen Xintao Wu 26 1 0 04 Nov 2024
Comparative Analysis of Demonstration Selection Algorithms for LLM In-Context Learning Dong Shu Mengnan Du 40 0 0 30 Oct 2024
Toward Understanding In-context vs. In-weight Learning Bryan Chan Xinyi Chen András Gyorgy Dale Schuurmans 75 4 0 30 Oct 2024
On the Role of Depth and Looping for In-Context Learning with Task Diversity Khashayar Gatmiry Nikunj Saunshi Sashank J. Reddi Stefanie Jegelka Sanjiv Kumar 36 2 0 29 Oct 2024
CFSafety: Comprehensive Fine-grained Safety Assessment for LLMs Zhihao Liu Chenhui Hu ALM ELM 56 1 0 29 Oct 2024
Random Policy Enables In-Context Reinforcement Learning within Trust Horizons Weiqin Chen Santiago Paternain OffRL 42 0 0 25 Oct 2024
LanFL: Differentially Private Federated Learning with Large Language Models using Synthetic Samples Huiyu Wu Diego Klabjan FedML 46 0 0 24 Oct 2024
Bayesian scaling laws for in-context learning Aryaman Arora Dan Jurafsky Christopher Potts Noah D. Goodman 29 2 0 21 Oct 2024
Mesa-Extrapolation: A Weave Position Encoding Method for Enhanced Extrapolation in LLMs Xin Ma Yang Liu Qingbin Liu Xiaoxu Ma 31 1 0 21 Oct 2024
Bayesian Concept Bottleneck Models with LLM Priors Jean Feng Avni Kothari Luke Zier Chandan Singh Yan Shuo Tan 26 2 0 21 Oct 2024
Make LLMs better zero-shot reasoners: Structure-orientated autonomous reasoning Pengfei He Zitao Li Yue Xing Yaling Li Jiliang Tang Bolin Ding LLMAG LRM 35 1 0 18 Oct 2024
In-context learning and Occam's razor Eric Elmoznino Tom Marty Tejas Kasetty Léo Gagnon Sarthak Mittal Mahan Fathi Dhanya Sridhar Guillaume Lajoie 44 1 0 17 Oct 2024
Aggregation Artifacts in Subjective Tasks Collapse Large Language Models' Posteriors Georgios Chochlakis Alexandros Potamianos Kristina Lerman Shrikanth Narayanan 34 0 0 17 Oct 2024
On the Training Convergence of Transformers for In-Context Classification Wei Shen Ruida Zhou Jing Yang Cong Shen 31 3 0 15 Oct 2024
Cognitive Overload Attack:Prompt Injection for Long Context Bibek Upadhayay Vahid Behzadan Amin Karbasi AAML 34 2 0 15 Oct 2024
Bypassing the Exponential Dependency: Looped Transformers Efficiently Learn In-context by Multi-step Gradient Descent Bo Chen Xiaoyu Li Yingyu Liang Zhenmei Shi Zhao Song 96 20 0 15 Oct 2024
Inference and Verbalization Functions During In-Context Learning Junyi Tao Xiaoyin Chen Nelson F. Liu LRM ReLM 26 0 0 12 Oct 2024
VoxelPrompt: A Vision-Language Agent for Grounded Medical Image Analysis Andrew Hoopes V. Butoi John Guttag Adrian V. Dalca MedIm LM&MA 35 1 0 10 Oct 2024
Can Looped Transformers Learn to Implement Multi-step Gradient Descent for In-context Learning? Khashayar Gatmiry Nikunj Saunshi Sashank J. Reddi Stefanie Jegelka Sanjiv Kumar 69 17 0 10 Oct 2024
Let's Ask GNN: Empowering Large Language Model for Graph In-Context Learning Zhengyu Hu Yichuan Li Zhengyu Chen Jie Wang Han Liu Kyumin Lee Kaize Ding GNN 226 1 0 09 Oct 2024
Copiloting Diagnosis of Autism in Real Clinical Scenarios via LLMs Yi Jiang Qingyang Shen Shuzhong Lai Shunyu Qi Qian Zheng Lin Yao Yueming Wang Gang Pan 33 1 0 08 Oct 2024
Everything Everywhere All at Once: LLMs can In-Context Learn Multiple Tasks in Superposition Zheyang Xiong Ziyang Cai John Cooper Albert Ge Vasilis Papageorgiou ... Saurabh Agarwal Grigorios G Chrysos Samet Oymak Kangwook Lee Dimitris Papailiopoulos LRM 37 1 0 08 Oct 2024
Transformers learn variable-order Markov chains in-context Ruida Zhou C. Tian Suhas Diggavi 28 0 0 07 Oct 2024
Density estimation with LLMs: a geometric investigation of in-context learning trajectories Toni J. B. Liu Nicolas Boullé Raphael Sarfati Christopher Earls 33 0 0 07 Oct 2024
Task Diversity Shortens the ICL Plateau Jaeyeon Kim Sehyun Kwon Joo Young Choi Jongho Park Jaewoong Cho Jason D. Lee Ernest K. Ryu MoMe 36 2 0 07 Oct 2024
Wrong-of-Thought: An Integrated Reasoning Framework with Multi-Perspective Verification and Wrong Information Yongheng Zhang Qiguang Chen Jingxuan Zhou Peng Wang Jiasheng Si Jin Wang Wenpeng Lu Libo Qin LRM 49 3 0 06 Oct 2024
Simplicity bias and optimization threshold in two-layer ReLU networks Etienne Boursier Nicolas Flammarion 33 2 0 03 Oct 2024
Large Language Models as Markov Chains Oussama Zekri Ambroise Odonnat Abdelhakim Benechehab Linus Bleistein Nicolas Boullé I. Redko 48 10 0 03 Oct 2024
Bayes' Power for Explaining In-Context Learning Generalizations Samuel G. Müller Noah Hollmann Frank Hutter BDL 44 1 0 02 Oct 2024
Mitigating Copy Bias in In-Context Learning through Neuron Pruning Ameen Ali Lior Wolf Ivan Titov 44 2 0 02 Oct 2024
Racing Thoughts: Explaining Contextualization Errors in Large Language Models Michael A. Lepori Michael Mozer Asma Ghandeharioun LRM 87 1 0 02 Oct 2024
Are Large Language Models In-Context Personalized Summarizers? Get an iCOPERNICUS Test Done! Divya Patel Pathik Patel Ankush Chander Sourish Dasgupta Tanmoy Chakraborty 24 1 0 30 Sep 2024
"Oh LLM, I'm Asking Thee, Please Give Me a Decision Tree": Zero-Shot Decision Tree Induction and Embedding with Large Language Models Ricardo Knauer Mario Koddenbrock Raphael Wallsberger Nicholas M. Brisson Georg N. Duda Deborah Falla David W. Evans Erik Rodner 42 0 0 27 Sep 2024
Can Vision Language Models Learn from Visual Demonstrations of Ambiguous Spatial Reasoning? Bowen Zhao Leo Parker Dirac Paulina Varshavskaya VLM LRM 31 0 0 25 Sep 2024
Zero-shot forecasting of chaotic systems Yuanzhao Zhang William Gilpin AI4TS 37 4 0 24 Sep 2024