RoBERTa: A Robustly Optimized BERT Pretraining Approach

26 July 2019

Luke Zettlemoyer

Papers citing "RoBERTa: A Robustly Optimized BERT Pretraining Approach"

50 / 10,742 papers shown

Title
eCat: An End-to-End Model for Multi-Speaker TTS & Many-to-Many Fine-Grained Prosody Transfer Ammar Abbas S. Karlapati Bastian Schnell Penny Karanasou M. G. Moya Amith Nagaraj Ayman Boustati Nicole Peinelt Alexis Moinet Thomas Drugman 120 3 0 20 Jun 2023
LoSparse: Structured Compression of Large Language Models based on Low-Rank and Sparse Approximation Yixiao Li Yifan Yu Qingru Zhang Chen Liang Pengcheng He Weizhu Chen Tuo Zhao 120 76 0 20 Jun 2023
UVSCAN: Detecting Third-Party Component Usage Violations in IoT Firmware Binbin Zhao S. Ji Xuhong Zhang Yuan Tian Qinying Wang Yuwen Pu Chenyang Lyu R. Beyah 54 5 0 20 Jun 2023
Large Language Models are Fixated by Red Herrings: Exploring Creative Problem Solving and Einstellung Effect using the Only Connect Wall Dataset S. Naeini Raeid Saqur M. Saeidi John Giorgi Babak Taati 119 11 0 19 Jun 2023
Adversarial Robustness of Prompt-based Few-Shot Learning for Natural Language Understanding Venkata Prabhakara Sarath Nookala Gaurav Verma Subhabrata Mukherjee Srijan Kumar ELM 129 6 0 19 Jun 2023
Cross-Modal Attribute Insertions for Assessing the Robustness of Vision-and-Language Learning Shivaen Ramshetty Gaurav Verma Srijan Kumar 80 2 0 19 Jun 2023
Preserving Commonsense Knowledge from Pre-trained Language Models via Causal Inference Junhao Zheng Qianli Ma Shengjie Qiu Yue Wu Peitian Ma Junlong Liu Hu Feng Xichen Shang Haibin Chen AAML KELM CML CLL 130 15 0 19 Jun 2023
Adaptive Ordered Information Extraction with Deep Reinforcement Learning Wenhao Huang Jiaqing Liang Zhixu Li Yanghua Xiao Chuanjun Ji OffRL 78 2 0 19 Jun 2023
Distributed Marker Representation for Ambiguous Discourse Markers and Entangled Relations Dongyu Ru Lin Qiu Xipeng Qiu Yue Zhang Zheng Zhang 72 3 0 19 Jun 2023
Comparison of Machine Learning Methods for Assigning Software Issues to Team Members Bucsra Tabak Fatma Bacsak Aydemir 50 0 0 18 Jun 2023
DropCompute: simple and more robust distributed synchronous training via compute variance reduction Niv Giladi Shahar Gottlieb Moran Shkolnik A. Karnieli Ron Banner Elad Hoffer Kfir Y. Levy Daniel Soudry 82 3 0 18 Jun 2023
Summarization from Leaderboards to Practice: Choosing A Representation Backbone and Ensuring Robustness David Demeter Oshin Agarwal Simon Ben Igeri Marko Sterbentz Neil P. Molino John M. Conroy A. Nenkova 47 1 0 18 Jun 2023
Gender Bias in Transformer Models: A comprehensive survey Praneeth Nemani Yericherla Deepak Joel Pallavi Vijay Farhana Ferdousi Liza 56 3 0 18 Jun 2023
Evolutionary Verbalizer Search for Prompt-based Few Shot Text Classification Tongtao Ling Lei Chen Yutao Lai Haiyan Liu VLM 55 4 0 18 Jun 2023
Instant Soup: Cheap Pruning Ensembles in A Single Pass Can Draw Lottery Tickets from Large Models A. Jaiswal Shiwei Liu Tianlong Chen Ying Ding Zhangyang Wang VLM 115 21 0 18 Jun 2023
Universal Information Extraction with Meta-Pretrained Self-Retrieval Yu Bowen Mengcheng Fang Tingwen Liu Haiyang Yu Zhongkai Hu Fei Huang Yongbin Li Bin Wang RALM SSL 81 8 0 18 Jun 2023
Multilingual Multiword Expression Identification Using Lateral Inhibition and Domain Adaptation Andrei-Marius Avram V. Mititelu V. Pais Dumitru-Clementin Cercel Stefan Trausan-Matu 81 3 0 17 Jun 2023
KEST: Kernel Distance Based Efficient Self-Training for Improving Controllable Text Generation Yuxi Feng Xiaoyuan Yi L. Lakshmanan Xing Xie 67 1 0 17 Jun 2023
Empowering NLG: Offline Reinforcement Learning for Informal Summarization in Online Domains Zhiwei Tai Po-Chuan Chen OffRL 50 0 0 17 Jun 2023
Seen to Unseen: Exploring Compositional Generalization of Multi-Attribute Controllable Dialogue Generation Weihao Zeng Lulu Zhao Keqing He Ruotong Geng Jingang Wang Wei Wu Weiran Xu 73 3 0 17 Jun 2023
FutureTOD: Teaching Future Knowledge to Pre-trained Language Model for Task-Oriented Dialogue Weihao Zeng Keqing He Yejie Wang Chen Zeng Jingang Wang Yunsen Xian Weiran Xu 53 1 0 17 Jun 2023
Snowman: A Million-scale Chinese Commonsense Knowledge Graph Distilled from Foundation Model Jiaan Wang Jianfeng Qu Yunlong Liang Zhixu Li An Liu Guanfeng Liu Xin Zheng 82 2 0 17 Jun 2023
Data Selection for Fine-tuning Large Language Models Using Transferred Shapley Values S. Schoch Ritwick Mishra Yangfeng Ji TDI 138 18 0 16 Jun 2023
Democratizing Chatbot Debugging: A Computational Framework for Evaluating and Explaining Inappropriate Chatbot Responses Xu Han Michelle X. Zhou Yichen Wang Wenxi Chen Tom Yeh 38 4 0 16 Jun 2023
Investigating Masking-based Data Generation in Language Models Edward Ma 61 0 0 16 Jun 2023
No Strong Feelings One Way or Another: Re-operationalizing Neutrality in Natural Language Inference Animesh Nighojkar Antonio Laverghetta John Licato 59 4 0 16 Jun 2023
Revealing the impact of social circumstances on the selection of cancer therapy through natural language processing of social work notes Shenghuan Sun T. Zack C. Y. Williams A. Butte Madhumita Sushil 38 1 0 16 Jun 2023
Process Knowledge-infused Learning for Clinician-friendly Explanations Kaushik Roy Yuxin Zi Manas Gaur Jinendra Malekar Qi Zhang Vignesh Narayanan Amit P. Sheth AI4MH 62 14 0 16 Jun 2023
ActiveGLAE: A Benchmark for Deep Active Learning with Transformers Lukas Rauch Yi Men Denis Huseljic Moritz Wirth Bernd Bischl Bernhard Sick 91 13 0 16 Jun 2023
Pushing the Limits of ChatGPT on NLP Tasks Xiaofei Sun Linfeng Dong Xiaoya Li Zhen Wan Shuhe Wang ... Jiwei Li Fei Cheng Lingjuan Lyu Leilei Gan Guoyin Wang AI4MH LRM 117 32 0 16 Jun 2023
Semi-Offline Reinforcement Learning for Optimized Text Generation Changyu Chen Xiting Wang Yiqiao Jin Victor Ye Dong Li Dong Jie Cao Yi Liu Rui Yan OffRL 81 15 0 16 Jun 2023
Class-Adaptive Self-Training for Relation Extraction with Incompletely Annotated Training Data Qingyu Tan Lu Xu Lidong Bing Hwee Tou Ng 76 4 0 16 Jun 2023
CMLM-CSE: Based on Conditional MLM Contrastive Learning for Sentence Embeddings Zhang Wei Xu Chen 44 1 0 16 Jun 2023
Clickbait Detection via Large Language Models H. Wang Yi Zhu Ye Wang Yun Li Yunhao Yuan Jipeng Qiang 112 3 0 16 Jun 2023
The 2023 Video Similarity Dataset and Challenge Ed Pizzi Giorgos Kordopatis-Zilos Hiral Patel Gheorghe Postelnicu Sugosh Nagavara Ravindra A. Gupta Symeon Papadopoulos Giorgos Tolias Matthijs Douze 81 7 0 15 Jun 2023
Explore, Establish, Exploit: Red Teaming Language Models from Scratch Stephen Casper Jason Lin Joe Kwon Gatlen Culp Dylan Hadfield-Menell AAML 60 99 0 15 Jun 2023
PaReprop: Fast Parallelized Reversible Backpropagation Tyler Lixuan Zhu K. Mangalam 65 1 0 15 Jun 2023
Language-Guided Music Recommendation for Video via Prompt Analogies Daniel McKee Justin Salamon Josef Sivic Bryan C. Russell VGen 85 27 0 15 Jun 2023
Lexical Speaker Error Correction: Leveraging Language Models for Speaker Diarization Error Correction Rohit Paturi S. Srinivasan Xiang Li 62 15 0 15 Jun 2023
Matching Pairs: Attributing Fine-Tuned Models to their Pre-Trained Large Language Models Myles Foley Ambrish Rawat Taesung Lee Yufang Hou Gabriele Picco Giulio Zizzo DeLMO 138 6 0 15 Jun 2023
Can ChatGPT pass the Vietnamese National High School Graduation Examination? Xuan-Quy Dao Ngoc-Bich Le X. Phan Bac-Bien Ngo ELM 29 10 0 15 Jun 2023
Opportunities for Large Language Models and Discourse in Engineering Design Jan Göpfert J. Weinand Patrick Kuckertz D. Stolten AI4CE 65 5 0 15 Jun 2023
Relational Temporal Graph Reasoning for Dual-task Dialogue Language Understanding Bowen Xing Ivor W. Tsang 70 15 0 15 Jun 2023
DiPlomat: A Dialogue Dataset for Situated Pragmatic Reasoning Hengli Li Songchun Zhu Zilong Zheng 54 9 0 15 Jun 2023
Towards Benchmarking and Improving the Temporal Reasoning Capability of Large Language Models Qingyu Tan Hwee Tou Ng Lidong Bing LRM 117 29 0 15 Jun 2023
DocumentNet: Bridging the Data Gap in Document Pre-Training Lijun Yu Jin Miao Xiaoyu Sun Jiayi Chen Alexander G. Hauptmann H. Dai Wei Wei 31 3 0 15 Jun 2023
Opinion Tree Parsing for Aspect-based Sentiment Analysis Xiaoyi Bao Xiaotong Jiang Zhongqing Wang Yue Zhang Guodong Zhou LRM 44 7 0 15 Jun 2023
Pushing the Limits of Unsupervised Unit Discovery for SSL Speech Representation Ziyang Ma Zhisheng Zheng Guanrou Yang Yu Wang Chuxu Zhang Xie Chen SSL 72 9 0 15 Jun 2023
Bridging the Gap between Decision and Logits in Decision-based Knowledge Distillation for Pre-trained Language Models Qinhong Zhou Zonghan Yang Peng Li Yang Liu 102 3 0 15 Jun 2023
Interleaving Pre-Trained Language Models and Large Language Models for Zero-Shot NL2SQL Generation Zihui Gu Ju Fan Nan Tang Songyue Zhang Yuxin Zhang Zui Chen Lei Cao Guoliang Li Sam Madden Xiaoyong Du 89 23 0 15 Jun 2023