RoBERTa: A Robustly Optimized BERT Pretraining Approach

26 July 2019

Luke Zettlemoyer

Papers citing "RoBERTa: A Robustly Optimized BERT Pretraining Approach"

50 / 10,839 papers shown

Title
Revisiting Grammatical Error Correction Evaluation and Beyond Peiyuan Gong Xuebo Liu Heyan Huang Min Zhang 101 19 0 03 Nov 2022
Using Large Pre-Trained Language Model to Assist FDA in Premarket Medical Device Zongzhe Xu LM&MA MedIm 71 0 0 03 Nov 2022
Open-Vocabulary Argument Role Prediction for Event Extraction Yizhu Jiao Sha Li Yiqing Xie Ming Zhong Heng Ji Jiawei Han 115 17 0 03 Nov 2022
PINTO: Faithful Language Reasoning Using Prompt-Generated Rationales Peifeng Wang Aaron Chan Filip Ilievski Muhao Chen Xiang Ren LRM ReLM 123 65 0 03 Nov 2022
Generative Entity-to-Entity Stance Detection with Knowledge Graph Augmentation Xinliang Frederick Zhang Nick Beauchamp Lu Wang 62 10 0 02 Nov 2022
MPCFormer: fast, performant and private Transformer inference with MPC Dacheng Li Rulin Shao Hongyi Wang Han Guo Eric P. Xing Haotong Zhang 101 87 0 02 Nov 2022
Generative Adversarial Training Can Improve Neural Language Models Sajad Movahedi A. Shakery GAN AI4CE 77 2 0 02 Nov 2022
Improving Named Entity Recognition in Telephone Conversations via Effective Active Learning with Human in the Loop Md Tahmid Rahman Laskar Cheng Chen Xue-Yong Fu TN ShashiBhushan 87 7 0 02 Nov 2022
Multi-level Distillation of Semantic Knowledge for Pre-training Multilingual Language Model Mingqi Li Fei Ding Dan Zhang Long Cheng Hongxin Hu Feng Luo 92 7 0 02 Nov 2022
Processing Long Legal Documents with Pre-trained Transformers: Modding LegalBERT and Longformer Dimitris Mamakas Petros Tsotsi Ion Androutsopoulos Ilias Chalkidis VLM AILaw 69 29 0 02 Nov 2022
VarMAE: Pre-training of Variational Masked Autoencoder for Domain-adaptive Language Understanding Dou Hu Xiaolong Hou Xiyang Du Mengyuan Zhou Lian-Xin Jiang Yang Mo Xiaofeng Shi 101 13 0 01 Nov 2022
Looking Beyond IoCs: Automatically Extracting Attack Patterns from External CTI Md Tanvirul Alam Dipkamal Bhusal Youngja Park Nidhi Rastogi 83 51 0 01 Nov 2022
The future is different: Large pre-trained language models fail in prediction tasks K. Cvejoski Ramses J. Sanchez C. Ojeda 87 4 0 01 Nov 2022
Recognizing Nested Entities from Flat Supervision: A New NER Subtask, Feasibility and Challenges Enwei Zhu Yiyang Liu Ming Jin Jinpeng Li 105 1 0 01 Nov 2022
CONDAQA: A Contrastive Reading Comprehension Dataset for Reasoning about Negation Abhilasha Ravichander Matt Gardner Ana Marasović 120 35 0 01 Nov 2022
Training Vision-Language Models with Less Bimodal Supervision Elad Segal Ben Bogin Jonathan Berant VLM 53 2 0 01 Nov 2022
A Close Look into the Calibration of Pre-trained Language Models Yangyi Chen Lifan Yuan Ganqu Cui Zhiyuan Liu Heng Ji 153 53 0 31 Oct 2022
Unsafe's Betrayal: Abusing Unsafe Rust in Binary Reverse Engineering via Machine Learning Sangdon Park Xiang Cheng Taesoo Kim 105 1 0 31 Oct 2022
Where to start? Analyzing the potential value of intermediate models Leshem Choshen Elad Venezian Shachar Don-Yehiya Noam Slonim Yoav Katz MoMe 119 27 0 31 Oct 2022
WHEN FLUE MEETS FLANG: Benchmarks and Large Pre-trained Language Model for Financial Domain Raj Sanjay Shah Kunal Chawla Dheeraj Eidnani Agam Shah Wendi Du Sudheer Chava Natraj Raman Charese Smiley Jiaao Chen Diyi Yang AIFin 100 112 0 31 Oct 2022
AdaMix: Mixture-of-Adaptations for Parameter-efficient Model Tuning Yaqing Wang Sahaj Agarwal Subhabrata Mukherjee Xiaodong Liu Jing Gao Ahmed Hassan Awadallah Jianfeng Gao MoE 122 136 0 31 Oct 2022
SSD-LM: Semi-autoregressive Simplex-based Diffusion Language Model for Text Generation and Modular Control Xiaochuang Han Sachin Kumar Yulia Tsvetkov 169 91 0 31 Oct 2022
Controllable Factuality in Document-Grounded Dialog Systems Using a Noisy Channel Model Nico Daheim David Thulke Christian Dugast Hermann Ney HILM 99 4 0 31 Oct 2022
Pneg: Prompt-based Negative Response Generation for Dialogue Response Selection Task Nyoungwoo Lee Yujin Baek Ho-Jin Choi Jaegul Choo 76 6 0 31 Oct 2022
QuaLA-MiniLM: a Quantized Length Adaptive MiniLM Shira Guskin Moshe Wasserblat Chang Wang Haihao Shen MQ 80 2 0 31 Oct 2022
CodeEditor: Learning to Edit Source Code with Pre-trained Models Jia Li Ge Li Zhuo Li Zhi Jin Xing Hu Kechi Zhang Zhiyi Fu KELM 86 28 0 31 Oct 2022
Character-level White-Box Adversarial Attacks against Transformers via Attachable Subwords Substitution Aiwei Liu Honghai Yu Xuming Hu Shuang Li Li Lin Fukun Ma Yawen Yang Lijie Wen 86 35 0 31 Oct 2022
Validity Assessment of Legal Will Statements as Natural Language Inference A. Kwak Jacob O. Israelsen Clayton T. Morrison Derek E. Bambauer Mihai Surdeanu AILaw 48 3 0 30 Oct 2022
Using Context-to-Vector with Graph Retrofitting to Improve Word Embeddings Jiangbin Zheng Yile Wang Ge Wang Jun Xia Yufei Huang Guojiang Zhao Yue Zhang Stan Y. Li 68 26 0 30 Oct 2022
Parameter-Efficient Tuning Makes a Good Classification Head Zhuoyi Yang Ming Ding Yanhui Guo Qingsong Lv Jie Tang VLM 110 14 0 30 Oct 2022
Beyond Prompting: Making Pre-trained Language Models Better Zero-shot Learners by Clustering Representations Yu Fei Ping Nie Zhao Meng Roger Wattenhofer Mrinmaya Sachan VLM 100 20 0 29 Oct 2022
Empirical Evaluation of Post-Training Quantization Methods for Language Tasks Ting Hu Christoph Meinel Haojin Yang MQ 96 3 0 29 Oct 2022
CascadeXML: Rethinking Transformers for End-to-end Multi-resolution Training in Extreme Multi-label Classification Siddhant Kharbanda Atmadeep Banerjee Erik Schultheis Rohit Babbar 103 14 0 29 Oct 2022
End-to-end Spoken Language Understanding with Tree-constrained Pointer Generator Guangzhi Sun Chuxu Zhang P. Woodland 71 8 0 29 Oct 2022
Exploiting prompt learning with pre-trained language models for Alzheimer's Disease detection Yi Wang Jiajun Deng Tianzi Wang Bo Zheng Shoukang Hu Xunying Liu Helen M. Meng 97 17 0 29 Oct 2022
Differentiable Data Augmentation for Contrastive Sentence Representation Learning Tianduo Wang Wei Lu SSL 67 10 0 29 Oct 2022
Knowledge-in-Context: Towards Knowledgeable Semi-Parametric Language Models Xiaoman Pan Wenlin Yao Hongming Zhang Dian Yu Dong Yu Jianshu Chen KELM 303 25 0 28 Oct 2022
Just-DREAM-about-it: Figurative Language Understanding with DREAM-FLUTE Yuling Gu Yao Fu Valentina Pyatkin Ian H. Magnusson Bhavana Dalvi Peter Clark 255 8 0 28 Oct 2022
Probing for targeted syntactic knowledge through grammatical error detection Christopher Davis Christopher Bryant Andrew Caines Marek Rei P. Buttery 58 4 0 28 Oct 2022
Stop Measuring Calibration When Humans Disagree Joris Baan Wilker Aziz Barbara Plank Raquel Fernández 100 56 0 28 Oct 2022
Stanceosaurus: Classifying Stance Towards Multilingual Misinformation Jonathan Zheng Ashutosh Baheti Tarek Naous Wei Xu Alan Ritter 106 13 0 28 Oct 2022
RoChBert: Towards Robust BERT Fine-tuning for Chinese Zihan Zhang Jinfeng Li Ning Shi Bo Yuan Xiangyu Liu Rong Zhang Hui Xue Donghong Sun Chao Zhang AAML 61 4 0 28 Oct 2022
"It's Not Just Hate'': A Multi-Dimensional Perspective on Detecting Harmful Speech Online Federico Bianchi S. A. Hills Patrícia G. C. Rossini Dirk Hovy Rebekah Tromble N. Tintarev 95 15 0 28 Oct 2022
Bi-Directional Iterative Prompt-Tuning for Event Argument Extraction Lu Dai Bang Wang Wei Xiang Yijun Mo 55 17 0 28 Oct 2022
COST-EFF: Collaborative Optimization of Spatial and Temporal Efficiency with Slenderized Multi-exit Language Models Bowen Shen Zheng Lin Yuanxin Liu Zhengxiao Liu Lei Wang Weiping Wang VLM 77 5 0 27 Oct 2022
MorphTE: Injecting Morphology in Tensorized Embeddings Guobing Gan Peng Zhang Sunzhu Li Xiuqing Lu Benyou Wang 80 6 0 27 Oct 2022
Dial2vec: Self-Guided Contrastive Learning of Unsupervised Dialogue Embeddings Che Liu Rui Wang Junfeng Jiang Yongbin Li Fei Huang SSL 115 9 0 27 Oct 2022
Unsupervised Knowledge Graph Construction and Event-centric Knowledge Infusion for Scientific NLI Chenglin Wang Yucheng Zhou Guodong Long Xiaodong Wang Xiaowei Xu 41 2 0 27 Oct 2022
Unsupervised Boundary-Aware Language Model Pretraining for Chinese Sequence Labeling Peijie Jiang Dingkun Long Yanzhao Zhang Pengjun Xie Meishan Zhang Hao Fei SSL 60 13 0 27 Oct 2022
BERT-Flow-VAE: A Weakly-supervised Model for Multi-Label Text Classification Ziwen Liu J. Grau-Bové Scott Orr 82 1 0 27 Oct 2022