An Explanation of In-context Learning as Implicit Bayesian Inference

3 November 2021

Papers citing "An Explanation of In-context Learning as Implicit Bayesian Inference"

50 / 535 papers shown

Title
LLM-Enhanced Bayesian Optimization for Efficient Analog Layout Constraint Generation Guojin Chen Keren Zhu Seunggeun Kim Hanqing Zhu Yao Lai Bei Yu David Z. Pan 34 7 0 07 Jun 2024
On Subjective Uncertainty Quantification and Calibration in Natural Language Generation Ziyu Wang Chris Holmes UQLM 53 4 0 07 Jun 2024
Pretraining Decision Transformers with Reward Prediction for In-Context Multi-task Structured Bandit Learning Subhojyoti Mukherjee Josiah P. Hanna Qiaomin Xie Robert Nowak 81 2 0 07 Jun 2024
What Do Language Models Learn in Context? The Structured Task Hypothesis Jiaoda Li Yifan Hou Mrinmaya Sachan Ryan Cotterell LRM 44 7 0 06 Jun 2024
Enhancing In-Context Learning Performance with just SVD-Based Weight Pruning: A Theoretical Perspective Xinhao Yao Xiaolin Hu Shenzhi Yang Yong Liu 47 2 0 06 Jun 2024
What Should Embeddings Embed? Autoregressive Models Represent Latent Generating Distributions Liyi Zhang Michael Y. Li Thomas Griffiths 61 3 0 06 Jun 2024
Exact Conversion of In-Context Learning to Model Weights in Linearized-Attention Transformers Brian K Chen Tianyang Hu Hui Jin Hwee Kuan Lee Kenji Kawaguchi 55 0 0 05 Jun 2024
On the Intrinsic Self-Correction Capability of LLMs: Uncertainty and Latent Concept Guangliang Liu Haitao Mao Bochuan Cao Zhiyu Xue K. Johnson Jiliang Tang Rongrong Wang LRM 37 9 0 04 Jun 2024
Iteration Head: A Mechanistic Study of Chain-of-Thought Vivien A. Cabannes Charles Arnal Wassim Bouaziz Alice Yang Francois Charton Julia Kempe LRM 27 7 0 04 Jun 2024
Eliciting the Priors of Large Language Models using Iterated In-Context Learning Jian-Qiao Zhu Thomas Griffiths BDL 45 2 0 04 Jun 2024
In-Context Learning of Physical Properties: Few-Shot Adaptation to Out-of-Distribution Molecular Graphs Grzegorz Kaszuba Amirhossein D. Naghdi Dario Massa Stefanos Papanikolaou Andrzej Jaszkiewicz Piotr Sankowski AI4CE OODD 42 0 0 03 Jun 2024
Universal In-Context Approximation By Prompting Fully Recurrent Models Aleksandar Petrov Tom A. Lamb Alasdair Paren Philip Torr Adel Bibi LRM 32 0 0 03 Jun 2024
Latent Logic Tree Extraction for Event Sequence Explanation from LLMs Zitao Song Chao Yang Chaojie Wang Bo An Shuang Li 63 4 0 03 Jun 2024
LOLA: LLM-Assisted Online Learning Algorithm for Content Experiments Zikun Ye Hema Yoganarasimhan Yufeng Zheng 49 2 0 03 Jun 2024
Guiding ChatGPT to Generate Salient Domain Summaries Jun Gao Ziqiang Cao Shaoyao Huang Luozheng Qin Chunhui Ai 59 0 0 03 Jun 2024
Is In-Context Learning in Large Language Models Bayesian? A Martingale Perspective Fabian Falck Ziyu Wang Chris Holmes 58 13 0 02 Jun 2024
Enhancing Jailbreak Attack Against Large Language Models through Silent Tokens Jiahao Yu Haozheng Luo Jerry Yao-Chieh Hu Wenbo Guo Han Liu Xinyu Xing 40 19 0 31 May 2024
From Words to Actions: Unveiling the Theoretical Underpinnings of LLM-Driven Autonomous Systems Jianliang He Siyu Chen Fengzhuo Zhang Zhuoran Yang LM&Ro LLMAG 44 2 0 30 May 2024
Why Larger Language Models Do In-context Learning Differently? Zhenmei Shi Junyi Wei Zhuoyan Xu Yingyu Liang 37 19 0 30 May 2024
Does learning the right latent variables necessarily improve in-context learning? Sarthak Mittal Eric Elmoznino Léo Gagnon Sangnie Bhardwaj Dhanya Sridhar Guillaume Lajoie 34 4 0 29 May 2024
IM-Context: In-Context Learning for Imbalanced Regression Tasks Ismail Nejjar Faez Ahmed Olga Fink 35 1 0 28 May 2024
Unifying Demonstration Selection and Compression for In-Context Learning Jun Gao Ziqiang Cao Wenjie Li 43 3 0 27 May 2024
On Mesa-Optimization in Autoregressively Trained Transformers: Emergence and Capability Chenyu Zheng Wei Huang Rongzheng Wang Guoqiang Wu Jun Zhu Chongxuan Li 44 1 0 27 May 2024
Automatic Domain Adaptation by Transformers in In-Context Learning Ryuichiro Hataya Kota Matsui Masaaki Imaizumi 40 1 0 27 May 2024
Cost-Effective Online Multi-LLM Selection with Versatile Reward Models Xiangxiang Dai Jin Li Xutong Liu Anqi Yu J. C. Lui 41 5 0 26 May 2024
Evaluating the Adversarial Robustness of Retrieval-Based In-Context Learning for Large Language Models Simon Chi Lok Yu Jie He Pasquale Minervini Jeff Z. Pan 28 0 0 24 May 2024
Off-the-shelf ChatGPT is a Good Few-shot Human Motion Predictor Haoxuan Qu Zhaoyang He Zeyu Hu Yujun Cai Jun Liu 48 1 0 24 May 2024
From Frege to chatGPT: Compositionality in language, cognition, and deep neural networks Jacob Russin Sam Whitman McGrath Danielle J. Williams Lotem Elber-Dorozko AI4CE 75 3 0 24 May 2024
OptLLM: Optimal Assignment of Queries to Large Language Models Yueyue Liu Hongyu Zhang Yuantian Miao Van-Hoang Le Zhiqiang Li 21 2 0 24 May 2024
Towards Better Understanding of In-Context Learning Ability from In-Context Uncertainty Quantification Shang Liu Zhongze Cai Guanting Chen Xiaocheng Li UQCV 48 1 0 24 May 2024
In-context Time Series Predictor Jiecheng Lu Yan Sun Shihao Yang AI4TS 38 4 0 23 May 2024
Understanding the Training and Generalization of Pretrained Transformer for Sequential Decision Making Hanzhao Wang Yu Pan Fupeng Sun Shang Liu Kalyan Talluri Guanting Chen Xiaocheng Li OffRL 73 1 0 23 May 2024
Implicit In-context Learning Zhuowei Li Zihao Xu Ligong Han Yunhe Gao Song Wen Di Liu Hao Wang Dimitris N. Metaxas 38 1 0 23 May 2024
DETAIL: Task DEmonsTration Attribution for Interpretable In-context Learning Zijian Zhou Xiaoqiang Lin Xinyi Xu Alok Prakash Daniela Rus K. H. Low 36 2 0 22 May 2024
Adapting Large Multimodal Models to Distribution Shifts: The Role of In-Context Learning Guanglin Zhou Zhongyi Han Shiming Chen Erdun Gao Liming Zhu Salman Khan Xin Gao Lina Yao VLM 47 2 0 20 May 2024
Data Science Principles for Interpretable and Explainable AI Kris Sankaran FaML 45 0 0 17 May 2024
What is it for a Machine Learning Model to Have a Capability? Jacqueline Harding Nathaniel Sharadin ELM 40 3 0 14 May 2024
PLeak: Prompt Leaking Attacks against Large Language Model Applications Bo Hui Haolin Yuan Neil Zhenqiang Gong Philippe Burlina Yinzhi Cao LLMAG AAML SILM 39 34 0 10 May 2024
Towards a Theoretical Understanding of the 'Reversal Curse' via Training Dynamics Hanlin Zhu Baihe Huang Shaolun Zhang Michael I. Jordan Jiantao Jiao Yuandong Tian Stuart Russell LRM AI4CE 55 13 0 07 May 2024
Self-Improving Customer Review Response Generation Based on LLMs Guy Azov Tatiana Pelc Adi Fledel Alon Gila Kamhi 40 0 0 06 May 2024
Enhancing News Summarization with ELearnFit through Efficient In-Context Learning and Efficient Fine-Tuning Che Guan Andrew Chin Puya Vahabi 23 1 0 04 May 2024
PICLe: Eliciting Diverse Behaviors from Large Language Models with Persona In-Context Learning Hyeong Kyu Choi Yixuan Li 69 17 0 03 May 2024
Position: Understanding LLMs Requires More Than Statistical Generalization Patrik Reizinger Szilvia Ujváry Anna Mészáros A. Kerekes Wieland Brendel Ferenc Huszár 36 12 0 03 May 2024
LLM as Dataset Analyst: Subpopulation Structure Discovery with Large Language Model Yulin Luo Ruichuan An Bocheng Zou Yiming Tang Jiaming Liu Shanghang Zhang 44 13 0 03 May 2024
AppPoet: Large Language Model based Android malware detection via multi-view prompt engineering Wenxiang Zhao Juntao Wu Zhaoyi Meng AAML 31 11 0 29 Apr 2024
Exploring the Robustness of In-Context Learning with Noisy Labels Chen Cheng Xinzhi Yu Haodong Wen Jinsong Sun Guanzhang Yue Yihao Zhang Zeming Wei NoLa 24 6 0 28 Apr 2024
Transformers Can Represent $n$ -gram Language Models Anej Svete Ryan Cotterell 40 18 0 23 Apr 2024
FMint: Bridging Human Designed and Data Pretrained Models for Differential Equation Foundation Model Zezheng Song Jiaxin Yuan Haizhao Yang AI4CE 40 18 0 23 Apr 2024
In-Context Learning State Vector with Inner and Momentum Optimization Dongfang Li Zhenyu Liu Xinshuo Hu Zetian Sun Baotian Hu Min Zhang 40 5 0 17 Apr 2024
Many-Shot In-Context Learning Rishabh Agarwal Avi Singh Lei M. Zhang Bernd Bohnet Luis Rosias ... John D. Co-Reyes Eric Chu Feryal M. P. Behbahani Aleksandra Faust Hugo Larochelle ReLM OffRL BDL 63 96 0 17 Apr 2024