RoBERTa: A Robustly Optimized BERT Pretraining Approach

26 July 2019

Luke Zettlemoyer

Papers citing "RoBERTa: A Robustly Optimized BERT Pretraining Approach"

50 / 10,864 papers shown

Title
Tele-Knowledge Pre-training for Fault Analysis Zhuo Chen Wen Zhang Yufen Huang Yin Hua Yuxia Geng ... Song Jiang Zhaoyang Lian Yuchen Li Lei Cheng Hua-zeng Chen 97 17 0 20 Oct 2022
Late Prompt Tuning: A Late Prompt Could Be Better Than Many Prompts Xiangyang Liu Tianxiang Sun Xuanjing Huang Xipeng Qiu VLM 105 29 0 20 Oct 2022
Evidence > Intuition: Transferability Estimation for Encoder Selection Elisa Bassignana Max Müller-Eberstein Mike Zhang Barbara Plank 72 8 0 20 Oct 2022
Pre-training Language Models with Deterministic Factual Knowledge Shaobo Li Xiaoguang Li Lifeng Shang Chengjie Sun Bingquan Liu Zhenzhou Ji Xin Jiang Qun Liu KELM 101 11 0 20 Oct 2022
lo-fi: distributed fine-tuning without communication Mitchell Wortsman Suchin Gururangan Shen Li Ali Farhadi Ludwig Schmidt Michael G. Rabbat Ari S. Morcos 115 24 0 19 Oct 2022
TOIST: Task Oriented Instance Segmentation Transformer with Noun-Pronoun Distillation Pengfei Li Beiwen Tian Yongliang Shi Xiaoxue Chen Hao Zhao Guyue Zhou Ya Zhang 127 22 0 19 Oct 2022
Robustness of Demonstration-based Learning Under Limited Data Scenario Hongxin Zhang Yanzhe Zhang Ruiyi Zhang Diyi Yang 95 15 0 19 Oct 2022
Towards Realistic Low-resource Relation Extraction: A Benchmark with Empirical Baseline Study Xin Xu Xiang Chen Ningyu Zhang Xin Xie Xi Chen Huajun Chen 107 10 0 19 Oct 2022
Attribution and Obfuscation of Neural Text Authorship: A Data Mining Perspective Adaku Uchendu Thai Le Dongwon Lee DeLMO 123 45 0 19 Oct 2022
An Empirical Analysis of SMS Scam Detection Systems Muhammad Salman Muhammad Ikram M. Kâafar 100 8 0 19 Oct 2022
A Linguistic Investigation of Machine Learning based Contradiction Detection Models: An Empirical Analysis and Future Perspectives Maren Pielka F. Rode Lisa Pucknat Tobias Deuβer R. Sifa 67 2 0 19 Oct 2022
Group is better than individual: Exploiting Label Topologies and Label Relations for Joint Multiple Intent Detection and Slot Filling Bowen Xing Ivor W. Tsang BDL 92 22 0 19 Oct 2022
Leveraging a New Spanish Corpus for Multilingual and Crosslingual Metaphor Detection Elisa Sanchez-Bayona Rodrigo Agerri 85 10 0 19 Oct 2022
BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Mining Renqian Luo Liai Sun Yingce Xia Tao Qin Sheng Zhang Hoifung Poon Tie-Yan Liu MedIm AI4CE LM&MA 167 859 0 19 Oct 2022
The Devil in Linear Transformer Zhen Qin Xiaodong Han Weixuan Sun Dongxu Li Lingpeng Kong Nick Barnes Yiran Zhong 87 74 0 19 Oct 2022
Tempo: Accelerating Transformer-Based Model Training through Memory Footprint Reduction Muralidhar Andoorveedu Zhanda Zhu Bojian Zheng Gennady Pekhimenko 51 7 0 19 Oct 2022
ELASTIC: Numerical Reasoning with Adaptive Symbolic Compiler Jiaxin Zhang Yashar Moshfeghi AIMat 68 18 0 18 Oct 2022
How to Boost Face Recognition with StyleGAN? Artem Sevastopolsky Yury Malkov Nikita Durasov L. Verdoliva Matthias Nießner PICV 95 14 0 18 Oct 2022
The Tail Wagging the Dog: Dataset Construction Biases of Social Bias Benchmarks Nikil Selvam Sunipa Dev Daniel Khashabi Tushar Khot Kai-Wei Chang ALM 85 26 0 18 Oct 2022
Tiny-Attention Adapter: Contexts Are More Important Than the Number of Parameters Hongyu Zhao Hao Tan Hongyuan Mei MoE 87 18 0 18 Oct 2022
Sentiment-Aware Word and Sentence Level Pre-training for Sentiment Analysis Shuai Fan Chen Lin Haonan Li Zheng-Wen Lin Jinsong Su Hang Zhang Yeyun Gong Jian Guo Nan Duan VLM 88 19 0 18 Oct 2022
ROSE: Robust Selective Fine-tuning for Pre-trained Language Models Lan Jiang Hao Zhou Yankai Lin Peng Li Jie Zhou R. Jiang AAML 89 8 0 18 Oct 2022
Summary Workbench: Unifying Application and Evaluation of Text Summarization Models S. Syed Dominik Schwabe Martin Potthast 49 0 0 18 Oct 2022
Less is More: A Lightweight and Robust Neural Architecture for Discourse Parsing Ming Li Ruihong Huang 61 2 0 18 Oct 2022
Deepfake Text Detection: Limitations and Opportunities Jiameng Pu Zain Sarwar Sifat Muhammad Abdullah A. Rehman Yoonjin Kim P. Bhattacharya M. Javed Bimal Viswanath AAML 80 57 0 17 Oct 2022
Measures of Information Reflect Memorization Patterns Rachit Bansal Danish Pruthi Yonatan Belinkov 122 10 0 17 Oct 2022
Deep Bidirectional Language-Knowledge Graph Pretraining Michihiro Yasunaga Antoine Bosselut Hongyu Ren Xikun Zhang Christopher D. Manning Percy Liang J. Leskovec 105 205 0 17 Oct 2022
Fine-tuned Sentiment Analysis of COVID-19 Vaccine-Related Social Media Data: Comparative Study Chad A. Melton B. White Robert L. Davis R. Bednarczyk A. Shaban-Nejad 67 25 0 17 Oct 2022
Zero-Shot Ranking Socio-Political Texts with Transformer Language Models to Reduce Close Reading Time Kiymet Akdemir Ali Hürriyetoǧlu 58 2 0 17 Oct 2022
KPI-EDGAR: A Novel Dataset and Accompanying Metric for Relation Extraction from Financial Documents Tobias Deuβer Syed Musharraf Ali L. Hillebrand Desiana Nurchalifah Basil Jacob Christian Bauckhage R. Sifa 58 15 0 17 Oct 2022
Prompting GPT-3 To Be Reliable Chenglei Si Zhe Gan Zhengyuan Yang Shuohang Wang Jianfeng Wang Jordan L. Boyd-Graber Lijuan Wang KELM LRM 128 303 0 17 Oct 2022
PeerDA: Data Augmentation via Modeling Peer Relation for Span Identification Tasks Weiwen Xu Xin Li Yang Deng W. Lam Lidong Bing 86 10 0 17 Oct 2022
PACIFIC: Towards Proactive Conversational Question Answering over Tabular and Textual Data in Finance Yang Deng Wenqiang Lei Wenxuan Zhang W. Lam Tat-Seng Chua 107 56 0 17 Oct 2022
Plug-and-Play VQA: Zero-shot VQA by Conjoining Large Pretrained Models with Zero Training A. M. H. Tiong Junnan Li Boyang Albert Li Silvio Savarese Guosheng Lin MLLM 133 109 0 17 Oct 2022
A Unified Positive-Unlabeled Learning Framework for Document-Level Relation Extraction with Different Levels of Labeling Ye Wang Xin-Xin Liu Wen-zhong Hu Tao Zhang 80 19 0 17 Oct 2022
ConReader: Exploring Implicit Relations in Contracts for Contract Clause Extraction Weiwen Xu Yang Deng Wenqiang Lei Wenlong Zhao Tat-Seng Chua W. Lam AILaw 73 6 0 17 Oct 2022
Selective Query-guided Debiasing for Video Corpus Moment Retrieval Sunjae Yoon Jiajing Hong Eunseop Yoon Dahyun Kim Junyeong Kim Hee Suk Yoon Changdong Yoo 142 23 0 17 Oct 2022
Zero-Shot Learners for Natural Language Understanding via a Unified Multiple Choice Perspective Ping Yang Junjie Wang Ruyi Gan Xinyu Zhu Lin Zhang Ziwei Wu Xinyu Gao Jiaxing Zhang Tetsuya Sakai BDL 73 26 0 16 Oct 2022
Coordinated Topic Modeling Pritom Saha Akash Jie Huang Kevin Chen-Chuan Chang 76 1 0 16 Oct 2022
Knowledge Prompting in Pre-trained Language Model for Natural Language Understanding Jiadong Wang Wenkang Huang Qiuhui Shi Hongbin Wang Minghui Qiu Xiang Li Ming Gao KELM VLM 92 19 0 16 Oct 2022
StoryER: Automatic Story Evaluation via Ranking, Rating and Reasoning Hong Chen D. Vo Hiroya Takamura Yusuke Miyao Hideki Nakayama 110 20 0 16 Oct 2022
Model Criticism for Long-Form Text Generation Yuntian Deng Volodymyr Kuleshov Alexander M. Rush 119 19 0 16 Oct 2022
PAR: Political Actor Representation Learning with Social Context and Expert Knowledge Shangbin Feng Zhaoxuan Tan Zilong Chen Ningnan Wang Peisheng Yu Qinghua Zheng Xiao Chang Minnan Luo 81 9 0 15 Oct 2022
Code Recommendation for Open Source Software Developers Yiqiao Jin Yunsheng Bai Yanqiao Zhu Yizhou Sun Wei Wang 97 24 0 15 Oct 2022
Injecting Domain Knowledge from Empirical Interatomic Potentials to Neural Networks for Predicting Material Properties Amit Gupta Daniel S. Karls Mingjian Wen Ilia Nikiforov E. Tadmor George Karypis 82 8 0 14 Oct 2022
PseudoReasoner: Leveraging Pseudo Labels for Commonsense Knowledge Base Population Tianqing Fang Quyet V. Do Hongming Zhang Yangqiu Song Ginny Wong Simon See LRM 104 11 0 14 Oct 2022
Pretrained Transformers Do not Always Improve Robustness Swaroop Mishra Bhavdeep Singh Sachdeva Chitta Baral VLM 58 2 0 14 Oct 2022
CAB: Comprehensive Attention Benchmarking on Long Sequence Modeling Jinchao Zhang Shuyang Jiang Jiangtao Feng Lin Zheng Dianbo Sui 3DV 212 9 0 14 Oct 2022
Enabling Classifiers to Make Judgements Explicitly Aligned with Human Values Yejin Bang Tiezheng Yu Andrea Madotto Zhaojiang Lin Mona T. Diab Pascale Fung 82 13 0 14 Oct 2022
Automatic Creation of Named Entity Recognition Datasets by Querying Phrase Representations Hyunjae Kim J. Yoo Seunghyun Yoon Jaewoo Kang 77 3 0 14 Oct 2022