Label Words are Anchors: An Information Flow Perspective for Understanding In-Context Learning

23 May 2023

Lei Li

Jie Zhou

Papers citing "Label Words are Anchors: An Information Flow Perspective for Understanding In-Context Learning"

50 / 135 papers shown

Title
Incomplete In-context Learning Wenqiang Wang Yangshijie Zhang 36 0 0 12 May 2025
A Multi-Dimensional Constraint Framework for Evaluating and Improving Instruction Following in Large Language Models Junjie Ye Caishuang Huang Zhe Chen Wenjie Fu Chenyuan Yang ... Tao Gui Qi Zhang Zhongchao Shi Jianping Fan Xuanjing Huang ALM 41 0 0 12 May 2025
Scaling sparse feature circuit finding for in-context learning Dmitrii Kharlapenko S. Kamath S Fazl Barez Arthur Conmy Neel Nanda 26 0 0 18 Apr 2025
Distilling Transitional Pattern to Large Language Models for Multimodal Session-based Recommendation Jiajie Su Qiyong Zhong Yunshan Ma Weiming Liu Chaochao Chen Xiaolin Zheng Jianwei Yin Tat-Seng Chua 38 0 0 13 Apr 2025
M2IV: Towards Efficient and Fine-grained Multimodal In-Context Learning in Large Vision-Language Models Yanshu Li Hongyang He Yi Cao Qisen Cheng Xiang Fu Ruixiang Tang VLM 40 0 0 06 Apr 2025
Page Classification for Print Imaging Pipeline Shaoyuan Xu Cheng Lu Mark Shaw Peter Bauer J. Allebach VLM 38 1 0 03 Apr 2025
Contextualize-then-Aggregate: Circuits for In-Context Learning in Gemma-2 2B Aleksandra Bakalova Yana Veitsman Xinting Huang Michael Hahn 33 0 0 31 Mar 2025
WindowKV: Task-Adaptive Group-Wise KV Cache Window Selection for Efficient LLM Inference Youhui Zuo Sibo Wei C. Zhang Zhuorui Liu Wenpeng Lu Dawei Song VLM 56 0 0 23 Mar 2025
HICD: Hallucination-Inducing via Attention Dispersion for Contrastive Decoding to Mitigate Hallucinations in Large Language Models Xinyan Jiang Hang Ye Yongxin Zhu Xiaoying Zheng Zikang Chen Jun Gong 44 0 0 17 Mar 2025
Lifting the Veil on Visual Information Flow in MLLMs: Unlocking Pathways to Faster Inference Hao Yin Guangzong Si Zilei Wang 48 0 0 17 Mar 2025
Don't Take Things Out of Context: Attention Intervention for Enhancing Chain-of-Thought Reasoning in Large Language Models Shaotian Yan Chen Shen Wenxiao Wang Liang Xie Junjie Liu Jieping Ye ReLM LRM 52 0 0 14 Mar 2025
LLaVA-MLB: Mitigating and Leveraging Attention Bias for Training-Free Video LLMs Leqi Shen Tao He Guoqiang Gong Fan Yang Y. Zhang Pengzhang Liu Sicheng Zhao Guiguang Ding 47 0 0 14 Mar 2025
Take Off the Training Wheels Progressive In-Context Learning for Effective Alignment Zhenyu Liu Dongfang Li Xinshuo Hu X. Zhao Yibin Chen Baotian Hu Min-Ling Zhang 49 1 0 13 Mar 2025
Your Large Vision-Language Model Only Needs A Few Attention Heads For Visual Grounding Seil Kang Jinyeong Kim Junhyeok Kim Seong Jae Hwang VLM 90 2 0 08 Mar 2025
TestNUC: Enhancing Test-Time Computing Approaches through Neighboring Unlabeled Data Consistency Henry Peng Zou Zhengyao Gu Yue Zhou Yankai Chen Weizhi Zhang Liancheng Fang Yibo Wang Yangning Li Kay Liu Philip S. Yu 74 0 0 26 Feb 2025
LED-Merging: Mitigating Safety-Utility Conflicts in Model Merging with Location-Election-Disjoint Qianli Ma Dongrui Liu Qian Chen Linfeng Zhang Jing Shao MoMe 147 0 0 24 Feb 2025
RTBAS: Defending LLM Agents Against Prompt Injection and Privacy Leakage Peter Yong Zhong Siyuan Chen Ruiqi Wang McKenna McCall Ben L. Titzer Heather Miller Phillip B. Gibbons LLMAG 90 3 0 17 Feb 2025
Lost in the Passage: Passage-level In-context Learning Does Not Necessarily Need a "Passage" Hao Sun Chenming Tang Gengyang Li Yunfang Wu AIMat 47 0 0 15 Feb 2025
Learning Task Representations from In-Context Learning Baturay Saglam Zhuoran Yang Dionysis Kalogerias Amin Karbasi 57 0 0 08 Feb 2025
Advancing General Multimodal Capability of Vision-language Models with Pyramid-descent Visual Position Encoding Z. Chen Mingxiao Li Z. Chen Nan Du Xiaolong Li Yuexian Zou 53 0 0 19 Jan 2025
Controlling Out-of-Domain Gaps in LLMs for Genre Classification and Generated Text Detection D. Roussinov Serge Sharoff Nadezhda Puchnina 27 0 0 31 Dec 2024
A Review of Multimodal Explainable Artificial Intelligence: Past, Present and Future Shilin Sun Wenbin An Feng Tian Fang Nan Qidong Liu Xiaozhong Liu N. Shah Ping Chen 96 2 0 18 Dec 2024
Understanding Knowledge Hijack Mechanism in In-context Learning through Associative Memory Shuo Wang Issei Sato 76 0 0 16 Dec 2024
Video Diffusion Transformers are In-Context Learners Zhengcong Fei Di Qiu Changqian Yu Debang Li Mingyuan Fan VGen DiffM 196 2 0 14 Dec 2024
Beyond Examples: High-level Automated Reasoning Paradigm in In-Context Learning via MCTS Jinyang Wu Mingkuan Feng Shuai Zhang Feihu Che Zengqi Wen J. Tao ReLM LRM 107 9 0 27 Nov 2024
Understanding Multimodal LLMs: the Mechanistic Interpretability of Llava in Visual Question Answering Zeping Yu Sophia Ananiadou 136 0 0 17 Nov 2024
SEEKR: Selective Attention-Guided Knowledge Retention for Continual Learning of Large Language Models Jinghan He Haiyun Guo Kuan Zhu Zihan Zhao Ming Tang J. T. Wang KELM 28 1 0 09 Nov 2024
Shortcut Learning in In-Context Learning: A Survey Rui Song Yingji Li Fausto Giunchiglia Fausto Giunchiglia Hao Xu 45 2 0 04 Nov 2024
Reasoning or a Semblance of it? A Diagnostic Study of Transitive Reasoning in LLMs Houman Mehrafarin Arash Eshghi Ioannis Konstas LRM 28 0 0 26 Oct 2024
Label Set Optimization via Activation Distribution Kurtosis for Zero-shot Classification with Generative Models Yue Li Zhixue Zhao Carolina Scarton 34 0 0 24 Oct 2024
Delving into the Reversal Curse: How Far Can Large Language Models Generalize? Zhengkai Lin Z. Fu Kai Liu Liang Xie Binbin Lin Wenxiao Wang D. Cai Yue Wu Jieping Ye LRM 25 3 0 24 Oct 2024
DEAN: Deactivating the Coupled Neurons to Mitigate Fairness-Privacy Conflicts in Large Language Models Chen Qian Dongrui Liu Jie Zhang Yong Liu Jing Shao 37 1 0 22 Oct 2024
Coarse-to-Fine Highlighting: Reducing Knowledge Hallucination in Large Language Models Qitan Lv Jie Wang Hanzhu Chen Bin Li Yongdong Zhang Feng Wu HILM 28 3 0 19 Oct 2024
Selection-p: Self-Supervised Task-Agnostic Prompt Compression for Faithfulness and Transferability Tsz Ting Chung Leyang Cui Lemao Liu Xinting Huang Shuming Shi Dit-Yan Yeung 38 1 0 15 Oct 2024
MLLM can see? Dynamic Correction Decoding for Hallucination Mitigation Chenxi Wang Xiang Chen N. Zhang Bozhong Tian Haoming Xu Shumin Deng H. Chen MLLM LRM 34 4 0 15 Oct 2024
Jailbreak Instruction-Tuned LLMs via end-of-sentence MLP Re-weighting Yifan Luo Zhennan Zhou Meitan Wang Bin Dong 24 0 0 14 Oct 2024
Can In-context Learning Really Generalize to Out-of-distribution Tasks? Qixun Wang Yifei Wang Yisen Wang Xianghua Ying OOD 31 2 0 13 Oct 2024
MIRAGE: Evaluating and Explaining Inductive Reasoning Process in Language Models Jiachun Li Pengfei Cao Zhuoran Jin Yubo Chen Kang-Jun Liu Jun Zhao LRM ELM 37 3 0 12 Oct 2024
Temporal Reasoning Transfer from Text to Video Lei Li Yuanxin Liu Linli Yao Peiyuan Zhang Chenxin An Lean Wang Xu Sun Lingpeng Kong Qi Liu LRM 45 7 0 08 Oct 2024
Wrong-of-Thought: An Integrated Reasoning Framework with Multi-Perspective Verification and Wrong Information Yongheng Zhang Qiguang Chen Jingxuan Zhou Peng Wang Jiasheng Si Jin Wang Wenpeng Lu Libo Qin LRM 49 3 0 06 Oct 2024
Self-Powered LLM Modality Expansion for Large Speech-Text Models Tengfei Yu Xuebo Liu Zhiyi Hou Liang Ding Dacheng Tao Min Zhang 32 0 0 04 Oct 2024
Calibrate to Discriminate: Improve In-Context Learning with Label-Free Comparative Inference Wei Cheng Tianlu Wang Yanmin Ji Fan Yang Keren Tan Yiyu Zheng 28 0 0 03 Oct 2024
Instance-adaptive Zero-shot Chain-of-Thought Prompting Xiaosong Yuan Chen Shen Shaotian Yan Xiaofeng Zhang Liang Xie Wenxiao Wang Renchu Guan Ying Wang Jieping Ye ReLM LRM 51 4 0 30 Sep 2024
Supervised Fine-Tuning Achieve Rapid Task Adaption Via Alternating Attention Head Activation Patterns Yang Zhao Li Du Xiao Ding Kai Xiong Ting Liu Bing Qin 18 2 0 24 Sep 2024
Teaching Tailored to Talent: Adverse Weather Restoration via Prompt Pool and Depth-Anything Constraint Sixiang Chen Tian-Chun Ye Kaixuan Zhang Zhaohu Xing Yunlong Lin Lei Zhu DiffM 46 9 0 24 Sep 2024
Interpreting Arithmetic Mechanism in Large Language Models through Comparative Neuron Analysis Zeping Yu Sophia Ananiadou LRM MILM 27 6 0 21 Sep 2024
Expediting and Elevating Large Language Model Reasoning via Hidden Chain-of-Thought Decoding Tianqiao Liu Zui Chen Zitao Liu Mi Tian Weiqi Luo LRM 27 2 0 13 Sep 2024
From Yes-Men to Truth-Tellers: Addressing Sycophancy in Large Language Models with Pinpoint Tuning Wei Chen Zhen Huang Liang Xie Binbin Lin Houqiang Li ... Deng Cai Yonggang Zhang Wenxiao Wang Xu Shen Jieping Ye 51 6 0 03 Sep 2024
EE-MLLM: A Data-Efficient and Compute-Efficient Multimodal Large Language Model Feipeng Ma Yizhou Zhou Hebei Li Zilong He Siying Wu Fengyun Rao Siying Wu Fengyun Rao Yueyi Zhang Xiaoyan Sun 31 3 0 21 Aug 2024
Large Language Models Might Not Care What You Are Saying: Prompt Format Beats Descriptions Chenming Tang Zhixiang Wang Yunfang Wu Yunfang Wu LRM 26 0 0 16 Aug 2024