RoBERTa: A Robustly Optimized BERT Pretraining Approach

26 July 2019

Luke Zettlemoyer

Papers citing "RoBERTa: A Robustly Optimized BERT Pretraining Approach"

50 / 10,814 papers shown

Title
Towards Unifying Medical Vision-and-Language Pre-training via Soft Prompts Zhihong Chen Shizhe Diao Benyou Wang Guanbin Li Xiang Wan MedIm 127 33 0 17 Feb 2023
Like a Good Nearest Neighbor: Practical Content Moderation and Text Classification Luke Bates Iryna Gurevych BDL AI4MH 84 0 0 17 Feb 2023
Multimodal Propaganda Processing Vincent Ng Shengjie Li 107 2 0 17 Feb 2023
DREEAM: Guiding Attention with Evidence for Improving Document-Level Relation Extraction Youmi Ma An Wang Naoaki Okazaki 85 70 0 17 Feb 2023
Role of Bias Terms in Dot-Product Attention Mahdi Namazifar Devamanyu Hazarika Dilek Z. Hakkani-Tür 95 3 0 16 Feb 2023
Pretraining Language Models with Human Preferences Tomasz Korbak Kejian Shi Angelica Chen Rasika Bhalerao C. L. Buckley Jason Phang Sam Bowman Ethan Perez ALM SyDa 106 232 0 16 Feb 2023
THC: Accelerating Distributed Deep Learning Using Tensor Homomorphic Compression Minghao Li Ran Ben-Basat S. Vargaftik Chon-In Lao Ke Xu Michael Mitzenmacher Minlan Yu Harvard University 94 19 0 16 Feb 2023
LEVER: Learning to Verify Language-to-Code Generation with Execution Ansong Ni Srini Iyer Dragomir R. Radev Ves Stoyanov Wen-tau Yih Sida I. Wang Xi Lin 145 227 0 16 Feb 2023
GLUECons: A Generic Benchmark for Learning Under Constraints Hossein Rajaby Faghihi Aliakbar Nafar Chen Zheng Roshanak Mirzaee Yue Zhang Andrzej Uszok Alexander Wan Tanawan Premsri Dan Roth Parisa Kordjamshidi ELM VLM 96 16 0 16 Feb 2023
Conversation Style Transfer using Few-Shot Learning Shamik Roy Raphael Shu Nikolaos Pappas Elman Mansimov Yi Zhang Saab Mansour Dan Roth 69 8 0 16 Feb 2023
Reanalyzing L2 Preposition Learning with Bayesian Mixed Effects and a Pretrained Language Model Jakob Prange Man Ho Ivy Wong 49 2 0 16 Feb 2023
Auto-Parallelizing Large Models with Rhino: A Systematic Approach on Production AI Platform Shiwei Zhang Lansong Diao Siyu Wang Zongyan Cao Yiliang Gu Chang Si Ziji Shi Zhen Zheng Chuan Wu W. Lin AI4CE 61 4 0 16 Feb 2023
Do We Still Need Clinical Language Models? Eric P. Lehman Evan Hernandez Diwakar Mahajan Jonas Wulff Micah J. Smith Zachary M. Ziegler Daniel Nadler Peter Szolovits Alistair E. W. Johnson Emily Alsentzer LM&MA AI4MH 96 142 0 16 Feb 2023
LabelPrompt: Effective Prompt-based Learning for Relation Classification Weinan Zhang Xiaoning Song Zhenhua Feng Tianyang Xu Xiaojun Wu VLM 72 4 0 16 Feb 2023
MINOTAUR: Multi-task Video Grounding From Multimodal Queries Raghav Goyal E. Mavroudi Xitong Yang Sainbayar Sukhbaatar Leonid Sigal Matt Feiszli Lorenzo Torresani Du Tran 95 7 0 16 Feb 2023
Slapo: A Schedule Language for Progressive Optimization of Large Deep Learning Model Training Hongzheng Chen Cody Hao Yu Shuai Zheng Zhen Zhang Zhiru Zhang Yida Wang 84 8 0 16 Feb 2023
COVID-VTS: Fact Extraction and Verification on Short Video Platforms Fuxiao Liu Yaser Yacoob Abhinav Shrivastava 85 28 0 15 Feb 2023
Meeting the Needs of Low-Resource Languages: The Value of Automatic Alignments via Pretrained Models Abteen Ebrahimi Arya D. McCarthy Arturo Oncevay Luis Chiruzzo J. Ortega Gustavo A. Giménez-Lugo Rolando A. Coto Solano Katharina Kann 75 6 0 15 Feb 2023
Platform-Independent and Curriculum-Oriented Intelligent Assistant for Higher Education Ramteja Sajja Y. Sermet David M. Cwiertny Ibrahim Demir 60 68 0 15 Feb 2023
Measuring the Instability of Fine-Tuning Yupei Du D. Nguyen 76 4 0 15 Feb 2023
Whats New? Identifying the Unfolding of New Events in Narratives Seyed Mahed Mousavi Shohei Tanaka Gabriel Roccabruna Koichiro Yoshino Satoshi Nakamura Giuseppe Riccardi 59 1 0 15 Feb 2023
A Pilot Evaluation of ChatGPT and DALL-E 2 on Decision Making and Spatial Reasoning Zhi–Bin Tang Mayank Kejriwal LRM 44 5 0 15 Feb 2023
The Capacity for Moral Self-Correction in Large Language Models Deep Ganguli Amanda Askell Nicholas Schiefer Thomas I. Liao Kamil.e Lukovsiut.e ... Tom B. Brown C. Olah Jack Clark Sam Bowman Jared Kaplan LRM ReLM 92 171 0 15 Feb 2023
READIN: A Chinese Multi-Task Benchmark with Realistic and Diverse Input Noises Chenglei Si Zhengyan Zhang Yingfa Chen Xiaozhi Wang Zhiyuan Liu Maosong Sun AAML 88 1 0 14 Feb 2023
AdapterSoup: Weight Averaging to Improve Generalization of Pretrained Language Models Alexandra Chronopoulou Matthew E. Peters Alexander Fraser Jesse Dodge MoMe 93 72 0 14 Feb 2023
Few-shot learning approaches for classifying low resource domain specific software requirements Anmol Nayak Hariprasad Timmapathini Vidhya Murali A. Gohad 15 1 0 14 Feb 2023
Exploring Category Structure with Contextual Language Models and Lexical Semantic Networks Joseph Renner Pascal Denis Rémi Gilleron Angèle Brunellière 40 4 0 14 Feb 2023
SwitchPrompt: Learning Domain-Specific Gated Soft Prompts for Classification in Low-Resource Domains Koustava Goswami Lukas Lange Jun Araki Heike Adel VLM 25 10 0 14 Feb 2023
Large-Scale Knowledge Synthesis and Complex Information Retrieval from Biomedical Documents Shreya Saxena Raj Sangani Siva Prasad Shubham Kumar Mihir Athale Rohan Awhad Vishal Vaddina 41 5 0 14 Feb 2023
Language Model Analysis for Ontology Subsumption Inference Yuan He Jiaoyan Chen Ernesto Jiménez-Ruiz Hang Dong Ian Horrocks 66 25 0 14 Feb 2023
Bag of Tricks for In-Distribution Calibration of Pretrained Transformers Jaeyoung Kim Dongbin Na Sungchul Choi Sungbin Lim VLM 85 5 0 13 Feb 2023
Symbolic Discovery of Optimization Algorithms Xiangning Chen Chen Liang Da Huang Esteban Real Kaiyuan Wang ... Xuanyi Dong Thang Luong Cho-Jui Hsieh Yifeng Lu Quoc V. Le 178 383 0 13 Feb 2023
Task-Specific Skill Localization in Fine-tuned Language Models A. Panigrahi Nikunj Saunshi Haoyu Zhao Sanjeev Arora MoMe 102 75 0 13 Feb 2023
AbLit: A Resource for Analyzing and Generating Abridged Versions of English Literature Melissa Roemmele Kyle Shaffer Katrina Olsen Yiyi Wang Steve DeNeefe 49 1 0 13 Feb 2023
Distinguishability Calibration to In-Context Learning Hongjing Li Hanqi Yan Yanran Li Li Qian Yulan He Lin Gui 80 2 0 13 Feb 2023
The Framework Tax: Disparities Between Inference Efficiency in NLP Research and Deployment Jared Fernandez Jacob Kahn Clara Na Yonatan Bisk Emma Strubell FedML 91 11 0 13 Feb 2023
RESDSQL: Decoupling Schema Linking and Skeleton Parsing for Text-to-SQL Haoyang Li Jing Zhang Cuiping Li Hong Chen 97 195 0 12 Feb 2023
An Extended Sequence Tagging Vocabulary for Grammatical Error Correction Stuart Mesham Christopher Bryant Marek Rei Zheng Yuan 78 8 0 12 Feb 2023
Discourse Structure Extraction from Pre-Trained and Fine-Tuned Language Models in Dialogues Chuyuan Li Patrick Huber Wen Xiao M. Amblard Chloé Braud Giuseppe Carenini 56 8 0 12 Feb 2023
TextDefense: Adversarial Text Detection based on Word Importance Entropy Lujia Shen Xuhong Zhang S. Ji Yuwen Pu Chunpeng Ge Xing Yang Yanghe Feng AAML 59 8 0 12 Feb 2023
Transformer models: an introduction and catalog X. Amatriain Ananth Sankar Jie Bing Praveen Kumar Bodigutla Timothy J. Hazen Michaeel Kazi 146 53 0 12 Feb 2023
Mutation-Based Adversarial Attacks on Neural Text Detectors G. Liang Jesus Guerrero I. Alsmadi DeLMO 74 9 0 11 Feb 2023
Cross-Modal Fine-Tuning: Align then Refine Junhong Shen Liam Li Lucio Dery Corey Staten M. Khodak Graham Neubig Ameet Talwalkar 85 47 0 11 Feb 2023
MTTM: Metamorphic Testing for Textual Content Moderation Software Wenxuan Wang Jen-tse Huang Weibin Wu Jianping Zhang Yizhan Huang Shuqing Li Pinjia He Michael Lyu 83 32 0 11 Feb 2023
DocILE Benchmark for Document Information Localization and Extraction vStvepán vSimsa Milan vSulc Michal Uvrivcávr Yash J. Patel Ahmed Hamdi ... Matyávs Skalický Jivrí Matas Antoine Doucet Mickael Coustaty Dimosthenis Karatzas 67 36 0 11 Feb 2023
Dialectograms: Machine Learning Differences between Discursive Communities Thyge R Enggaard August Lohse M. Pedersen Sune Lehmann 56 2 0 11 Feb 2023
See Your Heart: Psychological states Interpretation through Visual Creations Likun Yang Xiaokun Feng Xiaotang Chen Shiyu Zhang Kaiqi Huang 20 0 0 11 Feb 2023
Evaluating the Robustness of Discrete Prompts Yoichi Ishibashi Danushka Bollegala Katsuhito Sudoh Satoshi Nakamura 65 19 0 11 Feb 2023
Metaphor Detection with Effective Context Denoising Shunyu Wang Yucheng Li Chenghua Lin Loïc Barrault Frank Guerin 60 18 0 11 Feb 2023
Differentiable Outlier Detection Enable Robust Deep Multimodal Analysis Zhu Wang Sourav Medya Sathya Ravi VLM 100 0 0 11 Feb 2023