RoBERTa: A Robustly Optimized BERT Pretraining Approach

26 July 2019

Luke Zettlemoyer

Papers citing "RoBERTa: A Robustly Optimized BERT Pretraining Approach"

50 / 10,734 papers shown

Title
WinoQueer: A Community-in-the-Loop Benchmark for Anti-LGBTQ+ Bias in Large Language Models Virginia K. Felkner Ho-Chun Herbert Chang Eugene Jang Jonathan May OSLM 82 37 0 26 Jun 2023
Composing Parameter-Efficient Modules with Arithmetic Operations Jinghan Zhang Shiqi Chen Junteng Liu Junxian He KELM MoMe 113 126 0 26 Jun 2023
Vietnamese multi-document summary using subgraph selection approach -- VLSP 2022 AbMuSu Shared Task Huu-Thin Nguyen Tam Doan Thanh Cam-Van Thi Nguyen 44 0 0 26 Jun 2023
Label-Aware Hyperbolic Embeddings for Fine-grained Emotion Classification Chih-Yao Chen Tun-Min Hung Yi-Li Hsu Lun-Wei Ku 77 11 0 26 Jun 2023
A Positive-Unlabeled Metric Learning Framework for Document-Level Relation Extraction with Incomplete Labeling Ye Wang Huazheng Pan Tao Zhang Wen Wu Wen-zhong Hu 87 5 0 26 Jun 2023
Exploring the Robustness of Large Language Models for Solving Programming Problems Atsushi Shirafuji Yutaka Watanobe Takumi Ito Makoto Morishita Yuki Nakamura Yusuke Oda Jun Suzuki ELM 97 21 0 26 Jun 2023
Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuning Fuxiao Liu Kevin Qinghong Lin Linjie Li Jianfeng Wang Yaser Yacoob Lijuan Wang VLM MLLM 175 287 0 26 Jun 2023
Mutual Query Network for Multi-Modal Product Image Segmentation Yu Guo Wei Feng Zheng Zhang Xiancong Ren Yaoyu Li Jing Lv Xinshuai Zhu Zhangang Lin Jingping Shao 58 0 0 26 Jun 2023
Constraint-aware and Ranking-distilled Token Pruning for Efficient Transformer Inference Junyan Li Li Zhang Jiahang Xu Yujing Wang Shaoguang Yan ... Ting Cao Hao Sun Weiwei Deng Qi Zhang Mao Yang 64 10 0 26 Jun 2023
Unveiling the Potential of Sentiment: Can Large Language Models Predict Chinese Stock Price Movements? Haohan Zhang Fengrui Hua Chengjin Xu Hao Kong Ruiting Zuo Jian Guo AIFin 59 17 0 25 Jun 2023
Switch-BERT: Learning to Model Multimodal Interactions by Switching Attention and Input Qingpei Guo Kaisheng Yao Wei Chu MLLM 45 5 0 25 Jun 2023
Low-Rank Prune-And-Factorize for Language Model Compression Siyu Ren Kenny Q. Zhu 89 9 0 25 Jun 2023
Language models are weak learners Hariharan Manikandan Yiding Jiang J Zico Kolter 87 19 0 25 Jun 2023
On the Uses of Large Language Models to Interpret Ambiguous Cyberattack Descriptions Reza Fayyazi S. Yang 75 15 0 24 Jun 2023
DesCo: Learning Object Recognition with Rich Language Descriptions Liunian Harold Li Zi-Yi Dou Nanyun Peng Kai-Wei Chang ObjD VLM 82 22 0 24 Jun 2023
H $_2$ O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models Zhenyu Zhang Ying Sheng Dinesh Manocha Tianlong Chen Lianmin Zheng ... Yuandong Tian Christopher Ré Clark W. Barrett Zhangyang Wang Beidi Chen VLM 180 314 0 24 Jun 2023
Partitioning-Guided K-Means: Extreme Empty Cluster Resolution for Extreme Model Compression Tianhong Huang Victor Agostinelli Lizhong Chen MQ 51 0 0 24 Jun 2023
My Boli: Code-mixed Marathi-English Corpora, Pretrained Language Models and Evaluation Benchmarks Tanmay Chavan Omkar Gokhale Aditya Kane Shantanu Patankar Raviraj Joshi 73 3 0 24 Jun 2023
Weakly Supervised Multi-Label Classification of Full-Text Scientific Papers Yu Zhang Bowen Jin Xiusi Chen Yan-Jun Shen Yunyi Zhang Yu Meng Jiawei Han 76 13 0 24 Jun 2023
Towards Robust Aspect-based Sentiment Analysis through Non-counterfactual Augmentations Xinyu Liu Yanl Ding Kaikai An Chunyang Xiao Pranava Madhyastha Tong Xiao Jingbo Zhu 87 2 0 24 Jun 2023
Comparison of Pre-trained Language Models for Turkish Address Parsing Muhammed Cihat Unal Betul Aygun Aydın Gerek 23 4 0 24 Jun 2023
Math Word Problem Solving by Generating Linguistic Variants of Problem Statements Syed Rifat Raiyan Md. Nafis Faiyaz S. Kabir Mohsinul Kabir H. Mahmud Md. Kamrul Hasan 78 14 0 24 Jun 2023
Estimating the Causal Effect of Early ArXiving on Paper Acceptance Yanai Elazar Jiayao Zhang David Wadden Boshen Zhang Noah A. Smith CML AI4CE 80 5 0 24 Jun 2023
Cross-Language Speech Emotion Recognition Using Multimodal Dual Attention Transformers Syed Muhammad talha Zaidi S. Latif Junaid Qadir 69 8 0 23 Jun 2023
Deconstructing Classifiers: Towards A Data Reconstruction Attack Against Text Classification Models Adel M. Elmahdy A. Salem SILM 121 6 0 23 Jun 2023
Resume Information Extraction via Post-OCR Text Processing Selahattin Serdar Helli Senem Tanberk Sena Nur Cavsak 31 1 0 23 Jun 2023
Knowledge-Infused Self Attention Transformers Kaushik Roy Yuxin Zi Vignesh Narayanan Manas Gaur Amit P. Sheth KELM 48 7 0 23 Jun 2023
Toward Sustainable HPC: Carbon Footprint Estimation and Environmental Implications of HPC Systems Baolin Li Rohan Basu Roy Daniel Wang S. Samsi V. Gadepally Devesh Tiwari 113 39 0 22 Jun 2023
Named entity recognition in resumes Ege Kesim Aysu Deliahmetoglu 29 1 0 22 Jun 2023
Deep Metric Learning with Soft Orthogonal Proxies F. Saberi-Movahed M. K. Ebrahimpour Farid Saberi-Movahed Monireh Moshavash Dorsa Rahmatian Mahvash Mohazzebi Mahdi Shariatzadeh M. Eftekhari 53 3 0 22 Jun 2023
Impacts and Risk of Generative AI Technology on Cyber Defense Subash Neupane Ivan A. Fernandez Sudip Mittal Shahram Rahimi 98 18 0 22 Jun 2023
Quantizable Transformers: Removing Outliers by Helping Attention Heads Do Nothing Yelysei Bondarenko Markus Nagel Tijmen Blankevoort MQ 123 93 0 22 Jun 2023
Resources and Evaluations for Multi-Distribution Dense Information Retrieval Soumya Chatterjee Omar Khattab Simran Arora 124 0 0 21 Jun 2023
Evaluating Large Language Models with NeuBAROCO: Syllogistic Reasoning Ability and Human-like Biases Risako Ando Takanobu Morishita Hirohiko Abe K. Mineshima Mitsuhiro Okada LRM ELM 104 13 0 21 Jun 2023
Solving Dialogue Grounding Embodied Task in a Simulated Environment using Further Masked Language Modeling Weijie Zhang 61 0 0 21 Jun 2023
Iterated Piecewise Affine (IPA) Approximation for Language Modeling Davood Shamsi Wenhui Hua Brian Williams 47 0 0 21 Jun 2023
SIFTER: A Task-specific Alignment Strategy for Enhancing Sentence Embeddings Chaohui Yu Wenhao Zhu Chaoming Liu Xiaoyu Zhang Qiuhong Zhai 52 0 0 21 Jun 2023
Fantastic Weights and How to Find Them: Where to Prune in Dynamic Sparse Training A. Nowak Bram Grooten Decebal Constantin Mocanu Jacek Tabor 84 12 0 21 Jun 2023
Limits for Learning with Language Models Nicholas M. Asher Swarnadeep Bhar Akshay Chaturvedi Julie Hunter Soumya Paul 83 25 0 21 Jun 2023
Investigating Pre-trained Language Models on Cross-Domain Datasets, a Step Closer to General AI Mohamad Ballout U. Krumnack Gunther Heidemann Kai-Uwe Kühnberger 64 4 0 21 Jun 2023
Feature Interactions Reveal Linguistic Structure in Language Models Jaap Jumelet Willem H. Zuidema FAtt 53 7 0 21 Jun 2023
Which Spurious Correlations Impact Reasoning in NLI Models? A Visual Interactive Diagnosis through Data-Constrained Counterfactuals Robin Shing Moon Chan Afra Amini Mennatallah El-Assady LRM AAML 78 2 0 21 Jun 2023
Task-Robust Pre-Training for Worst-Case Downstream Adaptation Jianghui Wang Cheng Yang Xingyu Xie Cong Fang Zhouchen Lin OOD 68 0 0 21 Jun 2023
Sample Attackability in Natural Language Adversarial Attacks Vyas Raina Mark Gales SILM 110 1 0 21 Jun 2023
Towards Understanding What Code Language Models Learned Toufique Ahmed Dian Yu Chen Huang Cathy Wang Prem Devanbu Kenji Sagae ELM 77 5 0 20 Jun 2023
RoTaR: Efficient Row-Based Table Representation Learning via Teacher-Student Training Zui Chen Lei Cao S. Madden 117 0 0 20 Jun 2023
The Ecological Fallacy in Annotation: Modelling Human Label Variation goes beyond Sociodemographics Matthias Orlikowski Paul Röttger Philipp Cimiano Italy 74 29 0 20 Jun 2023
Give Us the Facts: Enhancing Large Language Models with Knowledge Graphs for Fact-aware Language Modeling Lin F. Yang Hongyang Chen Zhao Li Xiao Ding Xindong Wu KELM 112 93 0 20 Jun 2023
Towards Theory-based Moral AI: Moral AI with Aggregating Models Based on Normative Ethical Theory Masashi Takeshita Rafal Rzepka K. Araki 56 9 0 20 Jun 2023
Exploring the Performance and Efficiency of Transformer Models for NLP on Mobile Devices Ioannis Panopoulos Sokratis Nikolaidis Stylianos I. Venieris I. Venieris MedIm 71 4 0 20 Jun 2023