Understanding the Limitations of Variational Mutual Information Estimators

14 October 2019

Papers citing "Understanding the Limitations of Variational Mutual Information Estimators"

50 / 133 papers shown

Title
InfoPO: On Mutual Information Maximization for Large Language Model Alignment Teng Xiao Zhen Ge Sujay Sanghavi Tian Wang Julian Katz-Samuels Marc Versage Qingjun Cui Trishul Chilimbi 31 0 0 13 May 2025
Fundamental Limits of Perfect Concept Erasure Somnath Basu Roy Chowdhury Avinava Dubey Ahmad Beirami Rahul Kidambi Nicholas Monath Amr Ahmed Snigdha Chaturvedi 66 1 0 25 Mar 2025
A Deep Bayesian Nonparametric Framework for Robust Mutual Information Estimation Forough Fazeliasl Michael Minyi Zhang Bei Jiang Linglong Kong 48 0 0 13 Mar 2025
INFO-SEDD: Continuous Time Markov Chains as Scalable Information Metrics Estimators Alberto Foresti Giulio Franzese Pietro Michiardi 90 0 0 26 Feb 2025
A Neural Difference-of-Entropies Estimator for Mutual Information Haoran Ni Martin Lotz SSL DRL 91 0 0 18 Feb 2025
Density Ratio Estimation with Conditional Probability Paths Hanlin Yu Arto Klami Aapo Hyvarinen Anna Korba Omar Chehab 67 0 0 04 Feb 2025
InfoBridge: Mutual Information estimation via Bridge Matching Sergei Kholkin Ivan Butakov E. Burnaev Nikita Gushchin Alexander Korotin DiffM FedML 60 0 0 03 Feb 2025
MAPPING: Debiasing Graph Neural Networks for Fair Node Classification with Limited Sensitive Information Leakage Ying Song Balaji Palanisamy 85 0 0 28 Jan 2025
Information Subtraction: Learning Representations for Conditional Entropy Keng Hou Leong Yuxuan Xiu Wai Kin Chan 43 0 0 02 Jan 2025
A Benchmark Suite for Evaluating Neural Mutual Information Estimators on Unstructured Datasets Kyungeun Lee Wonjong Rhee 32 4 0 14 Oct 2024
Improving Numerical Stability of Normalized Mutual Information Estimator on High Dimensions Marko Tuononen Ville Hautamäki 21 0 0 10 Oct 2024
Channel-aware Contrastive Conditional Diffusion for Multivariate Probabilistic Time Series Forecasting Siyang Li Yize Chen Hui Xiong DiffM AI4TS 38 0 0 03 Oct 2024
Bounds on Lp errors in density ratio estimation via f-divergence loss functions Yoshiaki Kitazawa 26 0 0 02 Oct 2024
Multi-Cohort Framework with Cohort-Aware Attention and Adversarial Mutual-Information Minimization for Whole Slide Image Classification Sharon Peled Y. Maruvka Moti Freiman 31 1 0 17 Sep 2024
Automated Discovery of Pairwise Interactions from Unstructured Data Zuheng Xu Moksh Jain Ali Denton Shawn Whitfield Aniket Didolkar Berton Earnshaw Jason S. Hartford 33 3 0 11 Sep 2024
Learning Deep Kernels for Non-Parametric Independence Testing Nathaniel Xu Feng Liu Danica J. Sutherland BDL 39 0 0 10 Sep 2024
Approximating mutual information of high-dimensional variables using learned representations Gokul Gowri Xiao-Kang Lun Allon M. Klein Peng Yin 27 3 0 03 Sep 2024
Variational Inference Failures Under Model Symmetries: Permutation Invariant Posteriors for Bayesian Neural Networks Yoav Gelberg Tycho F. A. van der Ouderaa Mark van der Wilk Y. Gal AAML 45 4 0 10 Aug 2024
HiLo: A Learning Framework for Generalized Category Discovery Robust to Domain Shifts Hongjun Wang S. Vaze Kai Han 74 4 0 08 Aug 2024
Skill-aware Mutual Information Optimisation for Generalisation in Reinforcement Learning Xuehui Yu Mhairi Dunion Xin Li Stefano V. Albrecht 46 2 0 07 Jun 2024
Information Theoretic Text-to-Image Alignment Chao Wang Giulio Franzese A. Finamore Massimo Gallo Pietro Michiardi 75 0 0 31 May 2024
Constrained Ensemble Exploration for Unsupervised Skill Discovery Chenjia Bai Rushuai Yang Qiaosheng Zhang Kang Xu Yi Chen Ting Xiao Xuelong Li OffRL 42 3 0 25 May 2024
Neural Estimation Of Entropic Optimal Transport Tao Wang Ziv Goldfeld OT 38 1 0 10 May 2024
Deep Generative Sampling in the Dual Divergence Space: A Data-efficient & Interpretative Approach for Generative AI Sahil Garg Anderson Schneider Anant Raj Kashif Rasul Yuriy Nevmyvaka S. Gopal Amit Dhurandhar Guillermo Cecchi Irina Rish 28 0 0 10 Apr 2024
Mutual Information Estimation via Normalizing Flows I. Butakov Alexander Tolmachev S. Malanchuk A. Neopryatnaya Alexey Frolov 40 7 0 04 Mar 2024
TREET: TRansfer Entropy Estimation via Transformers Omer Luxembourg Dor Tsur Haim Permuter 27 1 0 10 Feb 2024
TexShape: Information Theoretic Sentence Embedding for Language Models Kaan Kale H. Esfahanizadeh Noel Elias Oguzhan Baser Muriel Médard S. Vishwanath 29 3 0 05 Feb 2024
BECLR: Batch Enhanced Contrastive Few-Shot Learning Stylianos Poulakakis-Daktylidis Hadi Jamali Rad 30 5 0 04 Feb 2024
Information-Theoretic State Variable Selection for Reinforcement Learning Charles Westphal Stephen Hailes Mirco Musolesi 26 3 0 21 Jan 2024
$f$ -Divergence Based Classification: Beyond the Use of Cross-Entropy Nicola Novello Andrea M. Tonello 32 7 0 02 Jan 2024
A Novel Information-Theoretic Objective to Disentangle Representations for Fair Classification Pierre Colombo Nathan Noiry Guillaume Staerman Pablo Piantanida FaML DRL 38 1 0 21 Oct 2023
Understanding Contrastive Learning via Distributionally Robust Optimization Junkang Wu Jiawei Chen Jiancan Wu Wentao Shi Xiang Wang Xiangnan He 37 24 0 17 Oct 2023
On the Properties and Estimation of Pointwise Mutual Information Profiles Paweł Czyż Frederic Grabowski Julia E. Vogt N. Beerenwinkel Alexander Marx 27 2 0 16 Oct 2023
MINDE: Mutual Information Neural Diffusion Estimation Giulio Franzese Mustapha Bounoua Pietro Michiardi DiffM 24 7 0 13 Oct 2023
Measuring Information in Text Explanations Zining Zhu Frank Rudzicz FAtt 37 0 0 06 Oct 2023
Max-Sliced Mutual Information Dor Tsur Ziv Goldfeld Kristjan Greenewald 32 9 0 28 Sep 2023
Feature Normalization Prevents Collapse of Non-contrastive Learning Dynamics Han Bao SSL MLT 37 1 0 28 Sep 2023
Latent Representation and Simulation of Markov Processes via Time-Lagged Information Bottleneck Marco Federici Patrick Forré Ryota Tomioka Bastiaan S. Veeling 29 3 0 13 Sep 2023
Towards a Rigorous Analysis of Mutual Information in Contrastive Learning Kyungeun Lee Jaeill Kim Suhyun Kang Wonjong Rhee SSL 36 2 0 30 Aug 2023
On Estimating the Gradient of the Expected Information Gain in Bayesian Experimental Design Ziqiao Ao Jinglai Li 30 2 0 19 Aug 2023
Mini-Batch Optimization of Contrastive Loss Jaewoong Cho Kartik K. Sreenivasan Keon Lee Kyunghoo Mun Soheun Yi Jeong-Gwan Lee Anna Lee Jy-yong Sohn Dimitris Papailiopoulos Kangwook Lee SSL 43 7 0 12 Jul 2023
Text Descriptions are Compressive and Invariant Representations for Visual Learning Zhili Feng Anna Bair J. Zico Kolter VLM 24 6 0 10 Jul 2023
Beyond Normal: On the Evaluation of Mutual Information Estimators Paweł Czyż Frederic Grabowski Julia E. Vogt N. Beerenwinkel Alexander Marx 28 32 0 19 Jun 2023
An information-Theoretic Approach to Semi-supervised Transfer Learning Daniel Jakubovitz David Uliel Miguel R. D. Rodrigues Raja Giryes 25 1 0 11 Jun 2023
Factorized Contrastive Learning: Going Beyond Multi-view Redundancy Paul Pu Liang Zihao Deng Martin Q. Ma James Zou Louis-Philippe Morency Ruslan Salakhutdinov SSL 31 49 0 08 Jun 2023
Estimating Conditional Mutual Information for Dynamic Feature Selection S. Gadgil Ian Covert Su-In Lee 34 3 0 05 Jun 2023
On the Effectiveness of Hybrid Mutual Information Estimation Marco Federici David Ruhe Patrick Forré 16 5 0 01 Jun 2023
LIMIT: Learning Interfaces to Maximize Information Transfer Benjamin A. Christie Dylan P. Losey 34 4 0 17 Apr 2023
Scalable Infomin Learning Yanzhi Chen Wei-Der Sun Yingzhen Li Adrian Weller 80 8 0 21 Feb 2023
STERLING: Synergistic Representation Learning on Bipartite Graphs Baoyu Jing Yuchen Yan Kaize Ding Chanyoung Park Yada Zhu Huan Liu Hanghang Tong SSL 32 19 0 25 Jan 2023