Label Words are Anchors: An Information Flow Perspective for Understanding In-Context Learning

23 May 2023

Lei Li

Jie Zhou

Papers citing "Label Words are Anchors: An Information Flow Perspective for Understanding In-Context Learning"

50 / 135 papers shown

Title
Paying More Attention to Image: A Training-Free Method for Alleviating Hallucination in LVLMs Shiping Liu Kecheng Zheng Wei Chen MLLM 49 34 0 31 Jul 2024
PMoE: Progressive Mixture of Experts with Asymmetric Transformer for Continual Learning Min Jae Jung Romain Rouvoy KELM MoE CLL 46 2 0 31 Jul 2024
Stress-Testing Long-Context Language Models with Lifelong ICL and Task Haystack Xiaoyue Xu Qinyuan Ye Xiang Ren 53 6 0 23 Jul 2024
DOPRA: Decoding Over-accumulation Penalization and Re-allocation in Specific Weighting Layer Jinfeng Wei Xiaofeng Zhang 28 13 0 21 Jul 2024
$$\text{Memory}^3$: Language Modeling with Explicit Memory$ $\text{Memory}^3$ : Language Modeling with Explicit Memory Hongkang Yang Zehao Lin Wenjin Wang Hao Wu Zhiyu Li ... Yu Yu Kai Chen Zhiyu Li Linpeng Tang Weinan E 50 12 0 01 Jul 2024
Unlocking the Future: Exploring Look-Ahead Planning Mechanistic Interpretability in Large Language Models Tianyi Men Pengfei Cao Zhuoran Jin Yubo Chen Kang Liu Jun Zhao LLMAG AIFin 28 4 0 23 Jun 2024
Distributed Rule Vectors is A Key Mechanism in Large Language Models' In-Context Learning Bowen Zheng Ming Ma Zhongqiao Lin Tianming Yang 36 1 0 23 Jun 2024
Understanding the Role of User Profile in the Personalization of Large Language Models Bin Wu Zhengyan Shi Hossein A. Rahmani Varsha Ramineni Emine Yilmaz 54 5 0 22 Jun 2024
Learnable In-Context Vector for Visual Question Answering Yingzhe Peng Chenduo Hao Xu Yang Jiawei Peng Xinting Hu Xin Geng 37 4 0 19 Jun 2024
When Parts are Greater Than Sums: Individual LLM Components Can Outperform Full Models Ting-Yun Chang Jesse Thomason Robin Jia 45 4 0 19 Jun 2024
CItruS: Chunked Instruction-aware State Eviction for Long Sequence Modeling Yu Bai Xiyuan Zou Heyan Huang Sanxing Chen Marc-Antoine Rondeau Yang Gao Jackie Chi Kit Cheung 39 4 0 17 Jun 2024
Concentrate Attention: Towards Domain-Generalizable Prompt Optimization for Language Models Chengzhengxu Li Xiaoming Liu Zhaohan Zhang Yichen Wang Chen Liu Y. Lan Chao Shen 60 2 0 15 Jun 2024
AIM: Let Any Multi-modal Large Language Models Embrace Efficient In-Context Learning Jun Gao Qian Qiao Ziqiang Cao Zili Wang Wenjie Li 34 3 0 11 Jun 2024
How Alignment and Jailbreak Work: Explain LLM Safety through Intermediate Hidden States Zhenhong Zhou Haiyang Yu Xinghua Zhang Rongwu Xu Fei Huang Yongbin Li 29 28 0 09 Jun 2024
Enhancing In-Context Learning Performance with just SVD-Based Weight Pruning: A Theoretical Perspective Xinhao Yao Xiaolin Hu Shenzhi Yang Yong Liu 47 2 0 06 Jun 2024
From Redundancy to Relevance: Enhancing Explainability in Multimodal Large Language Models Xiaofeng Zhang Chen Shen Xiaosong Yuan Shaotian Yan Liang Xie Wenxiao Wang Chaochen Gu Hao Tang Jieping Ye 54 2 0 04 Jun 2024
PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funneling Zefan Cai Yichi Zhang Bofei Gao Yuliang Liu Yong Li ... Wayne Xiong Yue Dong Baobao Chang Junjie Hu Wen Xiao 67 84 0 04 Jun 2024
Strengthened Symbol Binding Makes Large Language Models Reliable Multiple-Choice Selectors Mengge Xue Zhenyu Hu Liqun Liu Kuo Liao Shuang Li Honglin Han Meng Zhao Chengguo Yin 43 5 0 03 Jun 2024
UniBias: Unveiling and Mitigating LLM Bias through Internal Attention and FFN Manipulation Hanzhang Zhou Zijian Feng Zixiao Zhu Junlang Qian Kezhi Mao 47 6 0 31 May 2024
On the Noise Robustness of In-Context Learning for Text Generation Hongfu Gao Feipeng Zhang Wenyu Jiang Jun Shu Feng Zheng Hongxin Wei 58 3 0 27 May 2024
Unifying Demonstration Selection and Compression for In-Context Learning Jun Gao Ziqiang Cao Wenjie Li 43 3 0 27 May 2024
Implicit In-context Learning Zhuowei Li Zihao Xu Ligong Han Yunhe Gao Song Wen Di Liu Hao Wang Dimitris N. Metaxas 38 1 0 23 May 2024
DETAIL: Task DEmonsTration Attribution for Interpretable In-context Learning Zijian Zhou Xiaoqiang Lin Xinyi Xu Alok Prakash Daniela Rus K. H. Low 36 2 0 22 May 2024
Initialization is Critical to Whether Transformers Fit Composite Functions by Reasoning or Memorizing Zhongwang Zhang Pengxiao Lin Zhiwei Wang Yaoyu Zhang Z. Xu 39 3 0 08 May 2024
P-ICL: Point In-Context Learning for Named Entity Recognition with Large Language Models Guochao Jiang Zepeng Ding Yuchen Shi Deqing Yang 51 2 0 08 May 2024
Addressing Topic Granularity and Hallucination in Large Language Models for Topic Modelling Yida Mu Peizhen Bai Kalina Bontcheva Xingyi Song 33 6 0 01 May 2024
Improving Diversity of Commonsense Generation by Large Language Models via In-Context Learning Tianhui Zhang Bei Peng Danushka Bollegala LRM 35 7 0 25 Apr 2024
What Makes Multimodal In-Context Learning Work? Folco Bertini Baldassini Mustafa Shukor Matthieu Cord Laure Soulier Benjamin Piwowarski 40 18 0 24 Apr 2024
How Does the Textual Information Affect the Retrieval of Multimodal In-Context Learning? Yang Luo Zangwei Zheng Zirui Zhu Yang You 41 5 0 19 Apr 2024
In-Context Learning State Vector with Inner and Momentum Optimization Dongfang Li Zhenyu Liu Xinshuo Hu Zetian Sun Baotian Hu Min Zhang 40 5 0 17 Apr 2024
Neuron Specialization: Leveraging intrinsic task modularity for multilingual machine translation Shaomu Tan Di Wu Christof Monz MoMe 36 8 0 17 Apr 2024
Decomposing Label Space, Format and Discrimination: Rethinking How LLMs Respond and Solve Tasks via In-Context Learning Quanyu Long Yin Wu Wenya Wang Sinno Jialin Pan 94 1 0 11 Apr 2024
Transferable and Efficient Non-Factual Content Detection via Probe Training with Offline Consistency Checking Xiaokang Zhang Zijun Yao Jing Zhang Kaifeng Yun Jifan Yu Juan-Zi Li Jie Tang HILM 45 3 0 10 Apr 2024
Efficient Prompting Methods for Large Language Models: A Survey Kaiyan Chang Songcheng Xu Chenglong Wang Yingfeng Luo Tong Xiao Jingbo Zhu LRM 45 32 0 01 Apr 2024
Embedded Named Entity Recognition using Probing Classifiers Nicholas Popovic Michael Färber 45 1 0 18 Mar 2024
An Image is Worth 1/2 Tokens After Layer 2: Plug-and-Play Inference Acceleration for Large Vision-Language Models Liang Chen Haozhe Zhao Tianyu Liu Shuai Bai Junyang Lin Chang Zhou Baobao Chang MLLM VLM 48 119 0 11 Mar 2024
Self-Evaluation of Large Language Model based on Glass-box Features Hui Huang Yingqi Qu Jing Liu Muyun Yang Tiejun Zhao 29 2 0 07 Mar 2024
Demonstrating Mutual Reinforcement Effect through Information Flow Chengguang Gan Xuzheng He Qinghao Zhang Tatsunori Mori 19 0 0 05 Mar 2024
Not All Layers of LLMs Are Necessary During Inference Siqi Fan Xin Jiang Xiang Li Xuying Meng Peng Han Shuo Shang Aixin Sun Yequan Wang Zhongyuan Wang 49 32 0 04 Mar 2024
Fantastic Semantics and Where to Find Them: Investigating Which Layers of Generative LLMs Reflect Lexical Semantics Zhu Liu Cunliang Kong Ying Liu Maosong Sun 34 12 0 03 Mar 2024
Whispers that Shake Foundations: Analyzing and Mitigating False Premise Hallucinations in Large Language Models Hongbang Yuan Pengfei Cao Zhuoran Jin Yubo Chen Daojian Zeng Kang Liu Jun Zhao HILM 37 3 0 29 Feb 2024
Decomposed Prompting: Unveiling Multilingual Linguistic Structure Knowledge in English-Centric Large Language Models Ercong Nie Shuzhou Yuan Bolei Ma Helmut Schmid Michael Farber Frauke Kreuter Hinrich Schütze ReLM 99 6 0 28 Feb 2024
Focus on Your Question! Interpreting and Mitigating Toxic CoT Problems in Commonsense Reasoning Jiachun Li Pengfei Cao Chenhao Wang Zhuoran Jin Yubo Chen Daojian Zeng Kang Liu Jun Zhao LRM 51 8 0 28 Feb 2024
Learning or Self-aligning? Rethinking Instruction Fine-tuning Mengjie Ren Boxi Cao Hongyu Lin Liu Cao Xianpei Han Ke Zeng Guanglu Wan Xunliang Cai Le Sun 30 24 0 28 Feb 2024
Cutting Off the Head Ends the Conflict: A Mechanism for Interpreting and Mitigating Knowledge Conflicts in Language Models Zhuoran Jin Pengfei Cao Hongbang Yuan Yubo Chen Jiexin Xu Huaijun Li Xiaojian Jiang Kang Liu Jun Zhao 183 37 0 28 Feb 2024
The Impact of Demonstrations on Multilingual In-Context Learning: A Multidimensional Analysis Miaoran Zhang Vagrant Gautam Mingyang Wang Jesujoba Oluwadara Alabi Xiaoyu Shen Dietrich Klakow Marius Mosbach 47 8 0 20 Feb 2024
Backward Lens: Projecting Language Model Gradients into the Vocabulary Space Shahar Katz Yonatan Belinkov Mor Geva Lior Wolf 63 10 1 20 Feb 2024
Parallel Structures in Pre-training Data Yield In-Context Learning Yanda Chen Chen Zhao Zhou Yu Kathleen McKeown He He 29 14 0 19 Feb 2024
GNNavi: Navigating the Information Flow in Large Language Models by Graph Neural Network Shuzhou Yuan Ercong Nie Michael Farber Helmut Schmid Hinrich Schütze 37 3 0 18 Feb 2024
Visual In-Context Learning for Large Vision-Language Models Yucheng Zhou Xiang Li Qianning Wang Jianbing Shen MLLM 27 58 0 18 Feb 2024