Masked Reconstruction Contrastive Learning with Information Bottleneck Principle

15 November 2022

Congying Han

Xuecheng Nie

Papers citing "Masked Reconstruction Contrastive Learning with Information Bottleneck Principle"

42 / 42 papers shown

Title
Language Models are Few-shot Learners for Prognostic Prediction Zekai Chen Mariann Micsinai Balan Kevin Brown LM&MA 61 90 0 24 Feb 2023
Siamese Image Modeling for Self-Supervised Vision Representation Learning Chenxin Tao Xizhou Zhu Weijie Su Gao Huang Bin Li Jie Zhou Yu Qiao Xiaogang Wang Jifeng Dai SSL 81 96 0 02 Jun 2022
Unified Contrastive Learning in Image-Text-Label Space Jianwei Yang Chunyuan Li Pengchuan Zhang Bin Xiao Ce Liu Lu Yuan Jianfeng Gao VLM SSL 118 226 0 07 Apr 2022
Rethinking Minimal Sufficient Representation in Contrastive Learning Haoqing Wang Xun Guo Zhiwei Deng Yan Lu SSL 41 75 0 14 Mar 2022
Context Autoencoder for Self-Supervised Representation Learning Xiaokang Chen Mingyu Ding Xiaodi Wang Ying Xin Shentong Mo Yunhao Wang Shumin Han Ping Luo Gang Zeng Jingdong Wang SSL 83 395 0 07 Feb 2022
Masked Feature Prediction for Self-Supervised Visual Pre-Training Chen Wei Haoqi Fan Saining Xie Chaoxia Wu Alan Yuille Christoph Feichtenhofer ViT 137 666 0 16 Dec 2021
SimMIM: A Simple Framework for Masked Image Modeling Zhenda Xie Zheng Zhang Yue Cao Yutong Lin Jianmin Bao Zhuliang Yao Qi Dai Han Hu 170 1,344 0 18 Nov 2021
iBOT: Image BERT Pre-Training with Online Tokenizer Jinghao Zhou Chen Wei Huiyu Wang Wei Shen Cihang Xie Alan Yuille Tao Kong 72 729 0 15 Nov 2021
Masked Autoencoders Are Scalable Vision Learners Kaiming He Xinlei Chen Saining Xie Yanghao Li Piotr Dollár Ross B. Girshick ViT TPM 427 7,705 0 11 Nov 2021
BEiT: BERT Pre-Training of Image Transformers Hangbo Bao Li Dong Songhao Piao Furu Wei ViT 231 2,812 0 15 Jun 2021
Emerging Properties in Self-Supervised Vision Transformers Mathilde Caron Hugo Touvron Ishan Misra Hervé Jégou Julien Mairal Piotr Bojanowski Armand Joulin 611 6,029 0 29 Apr 2021
An Empirical Study of Training Self-Supervised Vision Transformers Xinlei Chen Saining Xie Kaiming He ViT 146 1,857 0 05 Apr 2021
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows Ze Liu Yutong Lin Yue Cao Han Hu Yixuan Wei Zheng Zhang Stephen Lin B. Guo ViT 415 21,347 0 25 Mar 2021
Barlow Twins: Self-Supervised Learning via Redundancy Reduction Jure Zbontar Li Jing Ishan Misra Yann LeCun Stéphane Deny SSL 291 2,338 0 04 Mar 2021
Image-to-image Translation via Hierarchical Style Disentanglement Xinyang Li Shengchuan Zhang Jie Hu Liujuan Cao Xiaopeng Hong Xudong Mao Feiyue Huang Yongjian Wu Rongrong Ji VLM 49 116 0 02 Mar 2021
Zero-Shot Text-to-Image Generation Aditya A. Ramesh Mikhail Pavlov Gabriel Goh Scott Gray Chelsea Voss Alec Radford Mark Chen Ilya Sutskever VLM 385 4,919 0 24 Feb 2021
Instance Localization for Self-supervised Detection Pretraining Ceyuan Yang Zhirong Wu Bolei Zhou Stephen Lin ViT SSL 125 149 0 16 Feb 2021
DetCo: Unsupervised Contrastive Learning for Object Detection Enze Xie Jian Ding Wenhai Wang Xiaohang Zhan Hang Xu Peize Sun Zhenguo Li Ping Luo 73 321 0 09 Feb 2021
Exploring Cross-Image Pixel Contrast for Semantic Segmentation Wenguan Wang Tianfei Zhou Feng Yu Jifeng Dai E. Konukoglu Luc Van Gool 182 479 0 28 Jan 2021
Contrastive Learning for Label-Efficient Semantic Segmentation Xiangyu Zhao Raviteja Vemulapalli Philip Mansfield Boqing Gong Bradley Green Lior Shapira Ying Nian Wu SSL SSeg 102 174 0 13 Dec 2020
Exploring Simple Siamese Representation Learning Xinlei Chen Kaiming He SSL 247 4,047 0 20 Nov 2020
Graph Information Bottleneck Tailin Wu Hongyu Ren Pan Li J. Leskovec AAML 140 235 0 24 Oct 2020
Bootstrap your own latent: A new approach to self-supervised Learning Jean-Bastien Grill Florian Strub Florent Altché Corentin Tallec Pierre Harvey Richemond ... M. G. Azar Bilal Piot Koray Kavukcuoglu Rémi Munos Michal Valko SSL 343 6,773 0 13 Jun 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 700 41,736 0 28 May 2020
What Makes for Good Views for Contrastive Learning? Yonglong Tian Chen Sun Ben Poole Dilip Krishnan Cordelia Schmid Phillip Isola SSL 84 1,323 0 20 May 2020
Improved Baselines with Momentum Contrastive Learning Xinlei Chen Haoqi Fan Ross B. Girshick Kaiming He SSL 452 3,422 0 09 Mar 2020
A Simple Framework for Contrastive Learning of Visual Representations Ting-Li Chen Simon Kornblith Mohammad Norouzi Geoffrey E. Hinton SSL 335 18,721 0 13 Feb 2020
Momentum Contrast for Unsupervised Visual Representation Learning Kaiming He Haoqi Fan Yuxin Wu Saining Xie Ross B. Girshick SSL 175 12,065 0 13 Nov 2019
Learning Deep Transformer Models for Machine Translation Qiang Wang Bei Li Tong Xiao Jingbo Zhu Changliang Li Derek F. Wong Lidia S. Chao 70 670 0 05 Jun 2019
Learning Representations by Maximizing Mutual Information Across Views Philip Bachman R. Devon Hjelm William Buchwalter SSL 189 1,467 0 03 Jun 2019
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 1.6K 94,511 0 11 Oct 2018
Adaptive Input Representations for Neural Language Modeling Alexei Baevski Michael Auli 101 390 0 28 Sep 2018
Unified Perceptual Parsing for Scene Understanding Tete Xiao Yingcheng Liu Bolei Zhou Yuning Jiang Jian Sun OCL VOS 171 1,871 0 26 Jul 2018
Attention Is All You Need Ashish Vaswani Noam M. Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan Gomez Lukasz Kaiser Illia Polosukhin 3DV 651 130,942 0 12 Jun 2017
Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour Priya Goyal Piotr Dollár Ross B. Girshick P. Noordhuis Lukasz Wesolowski Aapo Kyrola Andrew Tulloch Yangqing Jia Kaiming He 3DH 120 3,675 0 08 Jun 2017
Mask R-CNN Kaiming He Georgia Gkioxari Piotr Dollár Ross B. Girshick ObjD 344 27,129 0 20 Mar 2017
Opening the Black Box of Deep Neural Networks via Information Ravid Shwartz-Ziv Naftali Tishby AI4CE 98 1,407 0 02 Mar 2017
Deep Variational Information Bottleneck Alexander A. Alemi Ian S. Fischer Joshua V. Dillon Kevin Patrick Murphy 98 1,714 0 01 Dec 2016
Semantic Understanding of Scenes through the ADE20K Dataset Bolei Zhou Hang Zhao Xavier Puig Tete Xiao Sanja Fidler Adela Barriuso Antonio Torralba SSeg 380 1,865 0 18 Aug 2016
SGDR: Stochastic Gradient Descent with Warm Restarts I. Loshchilov Frank Hutter ODL 288 8,091 0 13 Aug 2016
Deep Learning and the Information Bottleneck Principle Naftali Tishby Noga Zaslavsky DRL 177 1,580 0 09 Mar 2015
Microsoft COCO: Common Objects in Context Nayeon Lee Michael Maire Serge J. Belongie Lubomir Bourdev Ross B. Girshick James Hays Pietro Perona Deva Ramanan C. L. Zitnick Piotr Dollár ObjD 383 43,524 0 01 May 2014