Variational Information Bottleneck for Effective Low-Resource Fine-Tuning

10 June 2021

Rabeeh Karimi Mahabadi

Yonatan Belinkov

James Henderson

DRL

ArXiv (abs)PDF HTML Github (39★)

Papers citing "Variational Information Bottleneck for Effective Low-Resource Fine-Tuning"

47 / 47 papers shown

Title
Learning Distribution-Wise Control in Representation Space for Language Models Chunyuan Deng Ruidi Chang Hanjie Chen 22 0 0 07 Jun 2025
Information-Theoretic Complementary Prompts for Improved Continual Text Classification Duzhen Zhang Yong Ren Chenxing Li Dong Yu Tielin Zhang CLL VLM 115 0 0 27 May 2025
How to Improve the Robustness of Closed-Source Models on NLI Joe Stacey Lisa Alazraki Aran Ubhi Beyza Ermis Aaron Mueller Marek Rei 42 0 0 26 May 2025
Mitigating Hallucinations in Large Vision-Language Models by Adaptively Constraining Information Flow Jiaqi Bai Hongcheng Guo Zhongyuan Peng Jian Yang Zhiyu Li Mingze Li Zhihong Tian VLM 99 2 0 28 Feb 2025
Learning to Discover Regulatory Elements for Gene Expression Prediction Xingyu Su Haiyang Yu D. Zhi Shuiwang Ji 64 0 0 19 Feb 2025
Structural Entropy Guided Probabilistic Coding Xiang Huang Hao Peng Li Sun Hui Lin Chunyang Liu Jiang Cao Philip S. Yu 115 0 0 12 Dec 2024
Capturing Style in Author and Document Representation Enzo Terreau Antoine Gourru Julien Velcin 74 1 0 18 Jul 2024
Representation Learning with Conditional Information Flow Maximization Dou Hu Lingwei Wei Wei Zhou Songlin Hu SSL 98 1 0 08 Jun 2024
TimeX++: Learning Time-Series Explanations with Information Bottleneck Zichuan Liu Tianchun Wang Jimeng Shi Xu Zheng Zhuomin Chen Lei Song Wenqian Dong J. Obeysekera Farhad Shirani Dongsheng Luo AI4TS 86 11 0 15 May 2024
Cauchy-Schwarz Divergence Information Bottleneck for Regression Shujian Yu Xi Yu Sigurd Løkse Robert Jenssen José C. Príncipe UQCV 80 5 0 27 Apr 2024
Explanation based Bias Decoupling Regularization for Natural Language Inference Jianxiang Zang Hui Liu 64 1 0 20 Apr 2024
LayerNorm: A key component in parameter-efficient fine-tuning Taha ValizadehAslani Hualou Liang 85 1 0 29 Mar 2024
Generalizable and Stable Finetuning of Pretrained Language Models on Low-Resource Texts Sai Ashish Somayajula Youwei Liang Abhishek Singh Li Zhang Pengtao Xie 80 2 0 19 Mar 2024
Enhancing Multivariate Time Series Forecasting with Mutual Information-driven Cross-Variable and Temporal Modeling Shiyi Qi Liangjiang Wen Yiduo Li Yuanhang Yang Zhe Li Zhongwen Rao Lujia Pan Zenglin Xu AI4TS 49 1 0 01 Mar 2024
Structured Probabilistic Coding Dou Hu Lingwei Wei Yaxin Liu Wei Zhou Songlin Hu 102 1 0 21 Dec 2023
Weakly Supervised Detection of Hallucinations in LLM Activations Miriam Rateike C. Cintas John Wamburu Tanya Akumu Skyler Speakman 81 14 0 05 Dec 2023
Selective Visual Representations Improve Convergence and Generalization for Embodied AI Ainaz Eftekhar Kuo-Hao Zeng Jiafei Duan Ali Farhadi Aniruddha Kembhavi Ranjay Krishna 102 17 0 07 Nov 2023
Dynamic Multimodal Information Bottleneck for Multimodality Classification Yingying Fang Shuang Wu Sheng Zhang Chao Huang Tieyong Zeng Xiaodan Xing Simon Walsh Guang Yang 65 9 0 02 Nov 2023
Nugget: Neural Agglomerative Embeddings of Text Guanghui Qin Benjamin Van Durme 71 20 0 03 Oct 2023
Individual and Structural Graph Information Bottlenecks for Out-of-Distribution Generalization Ling Yang Jiayi Zheng Heyuan Wang Zhongyi Liu Zhilin Huang Shenda Hong Wentao Zhang Tengjiao Wang 100 16 0 28 Jun 2023
InfoPrompt: Information-Theoretic Soft Prompt Tuning for Natural Language Understanding Junda Wu Tong Yu Rui Wang Zhao Song Ruiyi Zhang Handong Zhao Chaochao Lu Shuai Li Ricardo Henao VLM 94 25 0 08 Jun 2023
Text-Augmented Open Knowledge Graph Completion via Pre-Trained Language Models Pengcheng Jiang Shivam Agarwal Bowen Jin Xuan Wang Jimeng Sun Jiawei Han VLM RALM 52 21 0 24 May 2023
Bi-Drop: Enhancing Fine-tuning Generalization via Synchronous sub-net Estimation and Optimization Shoujie Tong Heming Xia Damai Dai Runxin Xu Tianyu Liu Binghuai Lin Yunbo Cao Zhifang Sui 51 0 0 24 May 2023
Distilling Robustness into Natural Language Inference Models with Domain-Targeted Augmentation Joe Stacey Marek Rei 57 3 0 22 May 2023
Towards understanding neural collapse in supervised contrastive learning with the information bottleneck method Siwei Wang S. Palmer 102 2 0 19 May 2023
Enhancing Multimodal Entity and Relation Extraction with Variational Information Bottleneck Shiyao Cui Jiangxia Cao Xin Cong Shuaiyi Nie Quangang Li Tingwen Liu Jinqiao Shi 67 25 0 05 Apr 2023
Contrastive variational information bottleneck for aspect-based sentiment analysis Ming-Wei Chang Min Yang Qingshan Jiang Ruifeng Xu 85 4 0 06 Mar 2023
MixPHM: Redundancy-Aware Parameter-Efficient Tuning for Low-Resource Visual Question Answering Jingjing Jiang Nanning Zheng MoE 114 6 0 02 Mar 2023
Span-based Named Entity Recognition by Generating and Compressing Information Nhung T. H. Nguyen Makoto Miwa Sophia Ananiadou 67 10 0 10 Feb 2023
Prototypical Fine-tuning: Towards Robust Performance Under Varying Data Sizes Yiqiao Jin Xiting Wang Y. Hao Yizhou Sun Xing Xie 94 11 0 24 Nov 2022
Towards Robust Low-Resource Fine-Tuning with Multi-View Compressed Representations Linlin Liu Xingxuan Li Megh Thakkar Xin Li Shafiq Joty Luo Si Lidong Bing 86 2 0 16 Nov 2022
Fine-Tuning Pre-Trained Language Models Effectively by Optimizing Subnetworks Adaptively Haojie Zhang Ge Li Jia Li Zhongjin Zhang Yuqi Zhu Zhi Jin AI4CE 57 28 0 03 Nov 2022
Disentangled Text Representation Learning with Information-Theoretic Perspective for Adversarial Robustness Jiahao Zhao Wenji Mao DRL OOD 61 3 0 26 Oct 2022
Measures of Information Reflect Memorization Patterns Rachit Bansal Danish Pruthi Yonatan Belinkov 110 10 0 17 Oct 2022
Correlation Information Bottleneck: Towards Adapting Pretrained Multimodal Models for Robust Visual Question Answering Jingjing Jiang Zi-yi Liu Nanning Zheng 96 8 0 14 Sep 2022
A Multi-Format Transfer Learning Model for Event Argument Extraction via Variational Information Bottleneck Jie Zhou Qi Zhang Qin Chen Liang He Xuanjing Huang 100 21 0 27 Aug 2022
Variational Distillation for Multi-View Learning Xudong Tian Zhizhong Zhang Cong Wang Wensheng Zhang Yanyun Qu Lizhuang Ma Zongze Wu Yuan Xie Dacheng Tao 116 5 0 20 Jun 2022
Don't Blame the Annotator: Bias Already Starts in the Annotation Instructions Mihir Parmar Swaroop Mishra Mor Geva Chitta Baral 114 55 0 01 May 2022
Anti-Spoofing Using Transfer Learning with Variational Information Bottleneck Youngsik Eom Yeonghyeon Lee Ji Sub Um Hoi-Rim Kim 118 26 0 04 Apr 2022
Adaptive Information Bottleneck Guided Joint Source and Channel Coding for Image Transmission Lunan Sun Yang Yang Mingzhe Chen Caili Guo Walid Saad H. Vincent Poor 57 26 0 12 Mar 2022
A Differential Entropy Estimator for Training Neural Networks Georg Pichler Pierre Colombo Malik Boudiaf Günther Koliander Pablo Piantanida 170 23 0 14 Feb 2022
Graph Structure Learning with Variational Information Bottleneck Qingyun Sun Jianxin Li Hao Peng Hongzhi Zhang Xingcheng Fu Cheng Ji Philip S. Yu 87 162 0 16 Dec 2021
Gated Information Bottleneck for Generalization in Sequential Environments Francesco Alesiani Shujian Yu Xi Yu OOD AAML 58 13 0 12 Oct 2021
Generalization in NLI: Ways (Not) To Go Beyond Simple Heuristics Prajjwal Bhargava Aleksandr Drozd Anna Rogers 159 108 0 04 Oct 2021
Raise a Child in Large Language Model: Towards Effective and Generalizable Fine-tuning Runxin Xu Fuli Luo Zhiyuan Zhang Chuanqi Tan Baobao Chang Songfang Huang Fei Huang LRM 203 190 0 13 Sep 2021
Variational Information Bottleneck for Effective Low-resource Audio Classification Shijing Si Jianzong Wang Huiming Sun Jianhan Wu Chuan Zhang Xiaoyang Qu Ning Cheng Lei Chen Jing Xiao 28 13 0 10 Jul 2021
Supervising Model Attention with Human Explanations for Robust Natural Language Inference Joe Stacey Yonatan Belinkov Marek Rei 75 48 0 16 Apr 2021