RoBERTa: A Robustly Optimized BERT Pretraining Approach

26 July 2019

Luke Zettlemoyer

Papers citing "RoBERTa: A Robustly Optimized BERT Pretraining Approach"

50 / 10,874 papers shown

Title
CAB: Comprehensive Attention Benchmarking on Long Sequence Modeling Jinchao Zhang Shuyang Jiang Jiangtao Feng Lin Zheng Dianbo Sui 3DV 212 9 0 14 Oct 2022
Enabling Classifiers to Make Judgements Explicitly Aligned with Human Values Yejin Bang Tiezheng Yu Andrea Madotto Zhaojiang Lin Mona T. Diab Pascale Fung 82 13 0 14 Oct 2022
Automatic Creation of Named Entity Recognition Datasets by Querying Phrase Representations Hyunjae Kim J. Yoo Seunghyun Yoon Jaewoo Kang 77 3 0 14 Oct 2022
MICO: A Multi-alternative Contrastive Learning Framework for Commonsense Knowledge Representation Ying Su Zihao Wang Tianqing Fang Hongming Zhang Yangqiu Song Tong Zhang 67 15 0 14 Oct 2022
A Survey of Parameters Associated with the Quality of Benchmarks in NLP Swaroop Mishra Anjana Arunkumar Chris Bryan Chitta Baral 107 1 0 14 Oct 2022
DyLoRA: Parameter Efficient Tuning of Pre-trained Models using Dynamic Search-Free Low-Rank Adaptation Mojtaba Valipour Mehdi Rezagholizadeh I. Kobyzev A. Ghodsi 173 185 0 14 Oct 2022
Can Language Representation Models Think in Bets? Zhi–Bin Tang Mayank Kejriwal 57 6 0 14 Oct 2022
Psychology-guided Controllable Story Generation Yuqiang Xie Yue Hu Yunpeng Li Guanqun Bi Luxi Xing Wei Peng 114 3 0 14 Oct 2022
MetaFill: Text Infilling for Meta-Path Generation on Heterogeneous Information Networks Zequn Liu Kefei Duan Junwei Yang Hanwen Xu Ming Zhang Sheng Wang MoE 78 0 0 14 Oct 2022
Transparency Helps Reveal When Language Models Learn Meaning Zhaofeng Wu William Merrill Hao Peng Iz Beltagy Noah A. Smith 61 10 0 14 Oct 2022
Noise Audits Improve Moral Foundation Classification Negar Mokhberian F. R. Hopp Bahareh Harandizadeh Fred Morstatter Kristina Lerman NoLa 84 7 0 13 Oct 2022
Early Discovery of Disappearing Entities in Microblogs Satoshi Akasaki Naoki Yoshinaga Masashi Toyoda 69 0 0 13 Oct 2022
Frustratingly Easy Sentiment Analysis of Text Streams: Generating High-Quality Emotion Arcs Using Emotion Lexicons Daniela Teodorescu Saif M. Mohammad 55 8 0 13 Oct 2022
Mind the Labels: Describing Relations in Knowledge Graphs With Pretrained Models Zdeněk Kasner Ioannis Konstas Ondrej Dusek 84 6 0 13 Oct 2022
Can Demographic Factors Improve Text Classification? Revisiting Demographic Adaptation in the Age of Transformers Chia-Chien Hung Anne Lauscher Dirk Hovy Simone Paolo Ponzetto Goran Glavaš VLM AI4CE 71 14 0 13 Oct 2022
Machine Generated Text: A Comprehensive Survey of Threat Models and Detection Methods Evan Crothers Nathalie Japkowicz H. Viktor DeLMO 161 113 0 13 Oct 2022
SODAPOP: Open-Ended Discovery of Social Biases in Social Commonsense Reasoning Models Haozhe An Zongxia Li Jieyu Zhao Rachel Rudinger 87 26 0 13 Oct 2022
Exploring Long-Sequence Masked Autoencoders Ronghang Hu Shoubhik Debnath Saining Xie Xinlei Chen 65 18 0 13 Oct 2022
On the Utility of Self-supervised Models for Prosody-related Tasks Guan-Ting Lin Chiyu Feng Wei-Ping Huang Yuan Tseng Tzu-Han Lin Chen-An Li Hung-yi Lee Nigel G. Ward 63 51 0 13 Oct 2022
Prompt-based Connective Prediction Method for Fine-grained Implicit Discourse Relation Recognition Hao Zhou Man Lan Yuanbin Wu YueFeng Chen Meirong Ma 67 26 0 13 Oct 2022
Self-explaining deep models with logic rule reasoning Seungeon Lee Xiting Wang Sungwon Han Xiaoyuan Yi Xing Xie M. Cha NAI ReLM LRM 96 17 0 13 Oct 2022
LSG Attention: Extrapolation of pretrained Transformers to long sequences Charles Condevaux S. Harispe 86 24 0 13 Oct 2022
An Empirical Study on Finding Spans Weiwei Gu Boyuan Zheng Yunmo Chen Tongfei Chen Benjamin Van Durme 62 4 0 13 Oct 2022
Benchmarking Long-tail Generalization with Likelihood Splits Ameya Godbole Robin Jia ALM 79 9 0 13 Oct 2022
SubeventWriter: Iterative Sub-event Sequence Generation with Coherence Controller Zhaowei Wang Hongming Zhang Tianqing Fang Yangqiu Song Ginny Wong Simon See 118 14 0 13 Oct 2022
PoliGraph: Automated Privacy Policy Analysis using Knowledge Graphs (Journal Version) Hao Cui R. Trimananda A. Markopoulou Scott Jordan 101 18 0 13 Oct 2022
DATScore: Evaluating Translation with Data Augmented Translations Moussa Kamal Eddine Guokan Shang Michalis Vazirgiannis 73 5 0 12 Oct 2022
Developing a general-purpose clinical language inference model from a large corpus of clinical notes Madhumita Sushil Dana Ludwig A. Butte V. Rudrapatna LM&MA 85 12 0 12 Oct 2022
Foundation Transformers Hongyu Wang Shuming Ma Shaohan Huang Li Dong Wenhui Wang ... Barun Patra Zhun Liu Vishrav Chaudhary Xia Song Furu Wei AI4CE 98 27 0 12 Oct 2022
Relational Graph Convolutional Neural Networks for Multihop Reasoning: A Comparative Study Ieva Staliunaite P. Gorinski Ignacio Iacobacci GNN 70 0 0 12 Oct 2022
RedHOT: A Corpus of Annotated Medical Questions, Experiences, and Claims on Social Media Somin Wadhwa Vivek Khetan Silvio Amir Byron C. Wallace 68 19 0 12 Oct 2022
Task Compass: Scaling Multi-task Pre-training with Task Prefix Zhuosheng Zhang Shuohang Wang Yichong Xu Yuwei Fang Wenhao Yu Yang Liu Han Zhao Chenguang Zhu Michael Zeng SSL LRM 80 16 0 12 Oct 2022
Back to the Future: On Potential Histories in NLP Zeerak Talat Anne Lauscher AI4TS 78 4 0 12 Oct 2022
A context-aware knowledge transferring strategy for CTC-based ASR Keda Lu Kuan-Yu Chen 64 16 0 12 Oct 2022
ERNIE-Layout: Layout Knowledge Enhanced Pre-training for Visually-rich Document Understanding Qiming Peng Yinxu Pan Wenjin Wang Bin Luo Zhenyu Zhang ... Shi Feng Yu Sun Hao Tian Hua Wu Haifeng Wang 83 83 0 12 Oct 2022
MedJEx: A Medical Jargon Extraction Model with Wiki's Hyperlink Span and Contextualized Masked Language Model Score Sunjae Kwon Zonghai Yao H. Jordan David Levy Brian Corner Hong-ye Yu 85 20 0 12 Oct 2022
Designing Robust Transformers using Robust Kernel Density Estimation Xing Han Zhaolin Ren T. Nguyen Khai Nguyen Joydeep Ghosh Nhat Ho 112 6 0 11 Oct 2022
Cross-Lingual Speaker Identification Using Distant Supervision Ben Zhou Dian Yu Dong Yu Dan Roth 40 1 0 11 Oct 2022
Measuring and Improving Semantic Diversity of Dialogue Generation Seungju Han Beomsu Kim Buru Chang 87 15 0 11 Oct 2022
Understanding Embodied Reference with Touch-Line Transformer Yongqian Li Xiaoxue Chen Hao Zhao Jiangtao Gong Guyue Zhou Federico Rossano Yixin Zhu 177 17 0 11 Oct 2022
Enriching Biomedical Knowledge for Low-resource Language Through Large-Scale Translation Long Phan Tai Dang H. Tran Trieu H. Trinh Vy Phan Lam D. Chau Minh-Thang Luong 64 8 0 11 Oct 2022
Aggregating Crowdsourced and Automatic Judgments to Scale Up a Corpus of Anaphoric Reference for Fiction and Wikipedia Texts Juntao Yu Silviu Paun Maris Camilleri Paloma Carretero García Jon Chamberlain Udo Kruschwitz Massimo Poesio 74 8 0 11 Oct 2022
Continual Training of Language Models for Few-Shot Learning Zixuan Ke Haowei Lin Yijia Shao Hu Xu Lei Shu Bin Liu KELM BDL CLL 148 36 0 11 Oct 2022
An Exploration of Hierarchical Attention Transformers for Efficient Long Document Classification Ilias Chalkidis Xiang Dai Manos Fergadiotis Prodromos Malakasiotis Desmond Elliott 92 35 0 11 Oct 2022
Model Cascading: Towards Jointly Improving Efficiency and Accuracy of NLP Systems Neeraj Varshney Chitta Baral 92 28 0 11 Oct 2022
Improving Sharpness-Aware Minimization with Fisher Mask for Better Generalization on Language Models Qihuang Zhong Liang Ding Li Shen Peng Mi Juhua Liu Bo Du Dacheng Tao AAML 98 51 0 11 Oct 2022
T5 for Hate Speech, Augmented Data and Ensemble Tosin Adewumi Sana Sabah Sabry Nosheen Abid F. Liwicki Marcus Liwicki 77 11 0 11 Oct 2022
Instance Regularization for Discriminative Language Model Pre-training Zhuosheng Zhang Hai Zhao M. Zhou 99 1 0 11 Oct 2022
MAP: Multimodal Uncertainty-Aware Vision-Language Pre-training Model Yatai Ji Junjie Wang Yuan Gong Lin Zhang Yan Zhu Hongfa Wang Jiaxing Zhang Tetsuya Sakai Yujiu Yang MLLM 82 33 0 11 Oct 2022
Rethinking the Event Coding Pipeline with Prompt Entailment C. Lefebvre Niklas Stoehr 82 6 0 11 Oct 2022