RoBERTa: A Robustly Optimized BERT Pretraining Approach

26 July 2019

Luke Zettlemoyer

Papers citing "RoBERTa: A Robustly Optimized BERT Pretraining Approach"

50 / 10,831 papers shown

Title
CAPSTONE: Curriculum Sampling for Dense Retrieval with Document Expansion Xingwei He Yeyun Gong Alex Jin Hang Zhang Anlei Dong Jian Jiao Siu-Ming Yiu Nan Duan RALM 100 3 0 18 Dec 2022
LaSQuE: Improved Zero-Shot Classification from Explanations Through Quantifier Modeling and Curriculum Learning Sayan Ghosh Rakesh R Menon Shashank Srivastava 80 3 0 18 Dec 2022
BEATs: Audio Pre-Training with Acoustic Tokenizers Sanyuan Chen Yu-Huan Wu Chengyi Wang Shujie Liu Daniel C. Tompkins Zhuo Chen Furu Wei 124 299 0 18 Dec 2022
Neural Rankers for Effective Screening Prioritisation in Medical Systematic Review Literature Search Shuai Wang Harrisen Scells Bevan Koopman Guido Zuccon 76 24 0 18 Dec 2022
PoE: a Panel of Experts for Generalized Automatic Dialogue Assessment Chen Zhang L. F. D’Haro Qiquan Zhang Thomas Friedrichs Haizhou Li 84 7 0 18 Dec 2022
Low-Resource Authorship Style Transfer: Can Non-Famous Authors Be Imitated? Ajay Patel Nicholas Andrews Chris Callison-Burch 75 7 0 18 Dec 2022
Language model acceptability judgements are not always robust to context Koustuv Sinha Jon Gauthier Aaron Mueller Kanishka Misra Keren Fuentes R. Levy Adina Williams 103 18 0 18 Dec 2022
HyPe: Better Pre-trained Language Model Fine-tuning with Hidden Representation Perturbation Hongyi Yuan Zheng Yuan Chuanqi Tan Fei Huang Songfang Huang 99 15 0 17 Dec 2022
Better Datastore, Better Translation: Generating Datastores from Pre-Trained Models for Nearest Neural Machine Translation Jiahuan Li Shanbo Cheng Zewei Sun Mingxuan Wang Shujian Huang 95 2 0 17 Dec 2022
Relational Sentence Embedding for Flexible Semantic Matching Bin Wang Haizhou Li 60 4 0 17 Dec 2022
DuNST: Dual Noisy Self Training for Semi-Supervised Controllable Text Generation Yuxi Feng Xiaoyuan Yi Xiting Wang L. Lakshmanan Xing Xie DiffM 103 5 0 16 Dec 2022
Plansformer: Generating Symbolic Plans using Transformers Vishal Pallagani Bharath Muppasani K. Murugesan F. Rossi L. Horesh Biplav Srivastava F. Fabiano Andrea Loreggia LM&Ro LLMAG OffRL 74 38 0 16 Dec 2022
Enhancing Multi-modal and Multi-hop Question Answering via Structured Knowledge and Unified Retrieval-Generation Qian Yang Qian Chen Wen Wang Baotian Hu Min Zhang 109 27 0 16 Dec 2022
Fine-grained Czech News Article Dataset: An Interdisciplinary Approach to Trustworthiness Analysis Matyáš Boháček Michal Bravansky Filip Trhlík Václav Moravec 63 2 0 16 Dec 2022
Decoder Tuning: Efficient Language Understanding as Decoding Ganqu Cui Wentao Li Ning Ding Longtao Huang Zhiyuan Liu Maosong Sun 80 6 0 16 Dec 2022
Assessing the Impact of Sequence Length Learning on Classification Tasks for Transformer Encoder Models Jean-Thomas Baillargeon Luc Lamontagne 75 1 0 16 Dec 2022
Metaphorical Polysemy Detection: Conventional Metaphor meets Word Sense Disambiguation Rowan Hall Maudslay Simone Teufel 44 9 0 16 Dec 2022
Lessons learned from the evaluation of Spanish Language Models Rodrigo Agerri Eneko Agirre ELM 113 15 0 16 Dec 2022
Homonymy Information for English WordNet Rowan Hall Maudslay Simone Teufel 25 2 0 16 Dec 2022
Feature Dropout: Revisiting the Role of Augmentations in Contrastive Learning Alex Tamkin Margalit Glasgow Xiluo He Noah D. Goodman SSL 123 7 0 16 Dec 2022
Convolution-enhanced Evolving Attention Networks Yujing Wang Yaming Yang Zhuowan Li Jiangang Bai Mingliang Zhang Xiangtai Li Jiahao Yu Ce Zhang Gao Huang Yu Tong ViT 104 6 0 16 Dec 2022
ALERT: Adapting Language Models to Reasoning Tasks Ping Yu Tianlu Wang O. Yu. Golovneva Badr AlKhamissi Siddharth Verma Zhijing Jin Gargi Ghosh Mona T. Diab Asli Celikyilmaz ReLM LRM 87 19 0 16 Dec 2022
Werewolf Among Us: A Multimodal Dataset for Modeling Persuasion Behaviors in Social Deduction Games Bolin Lai Hongxin Zhang Miao Liu Aryan Pariani Fiona Ryan Wenqi Jia Shirley Anugrah Hayati James M. Rehg Diyi Yang 59 10 0 16 Dec 2022
A unified information-theoretic model of EEG signatures of human language processing Jiaxuan Li Richard Futrell 35 1 0 16 Dec 2022
LegalRelectra: Mixed-domain Language Modeling for Long-range Legal Text Comprehension Wenyue Hua Yuchen Zhang Zhe Chen Josie Li Melanie Weber AILaw 64 7 0 16 Dec 2022
Saved You A Click: Automatically Answering Clickbait Titles Andrey Kurenkov TA Mentor Yian Zhang O. Johnson 54 5 0 15 Dec 2022
Efficient Long Sequence Modeling via State Space Augmented Transformer Simiao Zuo Xiaodong Liu Jian Jiao Denis Xavier Charles Eren Manavoglu Tuo Zhao Jianfeng Gao 180 37 0 15 Dec 2022
MAViL: Masked Audio-Video Learners Po-Yao (Bernie) Huang Vasu Sharma Hu Xu Chaitanya K. Ryali Haoqi Fan Yanghao Li Shang-Wen Li Gargi Ghosh Jitendra Malik Christoph Feichtenhofer 85 54 0 15 Dec 2022
Multi-VALUE: A Framework for Cross-Dialectal English NLP Caleb Ziems William B. Held Jingfeng Yang Jwala Dhamala Rahul Gupta Diyi Yang 141 44 0 15 Dec 2022
Revisiting the Gold Standard: Grounding Summarization Evaluation with Robust Human Evaluation Yixin Liu Alexander R. Fabbri Pengfei Liu Yilun Zhao Linyong Nan ... Simeng Han Shafiq Joty Chien-Sheng Wu Caiming Xiong Dragomir R. Radev ALM 86 134 0 15 Dec 2022
Ring That Bell: A Corpus and Method for Multimodal Metaphor Detection in Videos Khalid Alnajjar Mika Hämäläinen Shuo Zhang 70 8 0 15 Dec 2022
Visually-augmented pretrained language models for NLP tasks without images Hangyu Guo Kun Zhou Wayne Xin Zhao Qinyu Zhang Ji-Rong Wen VLM 67 10 0 15 Dec 2022
ROSCOE: A Suite of Metrics for Scoring Step-by-Step Reasoning O. Yu. Golovneva Moya Chen Spencer Poff Martin Corredor Luke Zettlemoyer Maryam Fazel-Zarandi Asli Celikyilmaz ReLM LRM 119 152 0 15 Dec 2022
The Effects of In-domain Corpus Size on pre-training BERT Chris Sanchez Zheyu Zhang AI4CE 36 4 0 15 Dec 2022
MASTER: Multi-task Pre-trained Bottlenecked Masked Autoencoders are Better Dense Retrievers Kun Zhou Xiao Liu Yeyun Gong Wayne Xin Zhao Daxin Jiang Nan Duan Ji-Rong Wen 108 16 0 15 Dec 2022
FreCDo: A Large Corpus for French Cross-Domain Dialect Identification Mihaela Găman Adrian-Gabriel Chifu William Domingues Radu Tudor Ionescu 42 3 0 15 Dec 2022
Using Two Losses and Two Datasets Simultaneously to Improve TempoWiC Accuracy Mohammad Javad Pirhadi Motahhare Mirzaei Sauleh Eetemadi 60 0 0 15 Dec 2022
Improve Text Classification Accuracy with Intent Information Yifeng Xie VLM 65 0 0 15 Dec 2022
Gradient-based Intra-attention Pruning on Pre-trained Language Models Ziqing Yang Yiming Cui Xin Yao Shijin Wang VLM 73 12 0 15 Dec 2022
Leveraging Natural Language Processing to Augment Structured Social Determinants of Health Data in the Electronic Health Record K. Lybarger Nicholas J. Dobbins Ritche Long Angad Singh Patrick Wedgeworth Özlem Ozuner Meliha Yetisgen-Yildiz 49 25 0 14 Dec 2022
Efficient Self-supervised Learning with Contextualized Target Representations for Vision, Speech and Language Alexei Baevski Arun Babu Wei-Ning Hsu Michael Auli VLM SSL 131 97 0 14 Dec 2022
VTCC-NLP at NL4Opt competition subtask 1: An Ensemble Pre-trained language models for Named Entity Recognition Xuan-Dung Doan 75 6 0 14 Dec 2022
Speech and Natural Language Processing Technologies for Pseudo-Pilot Simulator Amrutha Prasad Juan Pablo Zuluaga P. Motlícek Seyyed Saeed Sarfjoo Iuliia Nigmatulina Karel Veselý 61 3 0 14 Dec 2022
MIST: a Large-Scale Annotated Resource and Neural Models for Functions of Modal Verbs in English Scientific Text Sophie Henning Nicole Macher Stefan Grünewald Annemarie Friedrich 44 2 0 14 Dec 2022
Towards Linguistically Informed Multi-Objective Pre-Training for Natural Language Inference Maren Pielka Svetlana Schmidt Lisa Pucknat R. Sifa CLIP AI4CE 73 2 0 14 Dec 2022
Towards mapping the contemporary art world with ArtLM: an art-specific NLP model Qinkai Chen Mohamed El-Mennaoui Antoine Fosset Amine Rebei Haoyang Cao Philine Bouscasse Christy Eóin O'Beirne Sasha Shevchenko Mathieu Rosenbaum KELM 97 1 0 14 Dec 2022
Efficient Speech Representation Learning with Low-Bit Quantization Ching-Feng Yeh Wei-Ning Hsu Paden Tomasello Abdel-rahman Mohamed MQ 62 10 0 14 Dec 2022
Pre-trained Language Models Can be Fully Zero-Shot Learners Xuandong Zhao Siqi Ouyang Zhiguo Yu Ming-li Wu Lei Li VLM LRM 103 34 0 14 Dec 2022
Paraphrase Identification with Deep Learning: A Review of Datasets and Methods Chao Zhou Cheng Qiu Daniel Ernesto Acuna 127 26 0 13 Dec 2022
CREPE: Can Vision-Language Foundation Models Reason Compositionally? Zixian Ma Jerry Hong Mustafa Omer Gul Mona Gandhi Irena Gao Ranjay Krishna CoGe 100 143 0 13 Dec 2022