RoBERTa: A Robustly Optimized BERT Pretraining Approach

26 July 2019

Luke Zettlemoyer

Papers citing "RoBERTa: A Robustly Optimized BERT Pretraining Approach"

50 / 10,839 papers shown

Title
Transformers for End-to-End InfoSec Tasks: A Feasibility Study Ethan M. Rudd Mohammad Saidur Rahman Philip Tully 85 5 0 05 Dec 2022
Images Speak in Images: A Generalist Painter for In-Context Visual Learning Xinlong Wang Wen Wang Yue Cao Chunhua Shen Tiejun Huang VLM MLLM 166 262 0 05 Dec 2022
In-context Examples Selection for Machine Translation Sweta Agrawal Chunting Zhou M. Lewis Luke Zettlemoyer Marjan Ghazvininejad LRM 139 198 0 05 Dec 2022
Improving Few-Shot Performance of Language Models via Nearest Neighbor Calibration Feng Nie Meixi Chen Zhirui Zhang Xuan Cheng 72 33 0 05 Dec 2022
Video Games as a Corpus: Sentiment Analysis using Fallout New Vegas Dialog Mika Hämäläinen Khalid Alnajjar Thierry Poibeau 52 5 0 05 Dec 2022
GNN-SL: Sequence Labeling Based on Nearest Examples via GNN Shuhe Wang Yuxian Meng Rongbin Ouyang Jiwei Li Tianwei Zhang Lingjuan Lyu Guoyin Wang 87 10 0 05 Dec 2022
Grounded Keys-to-Text Generation: Towards Factual Open-Ended Generation Faeze Brahman Baolin Peng Michel Galley Sudha Rao Bill Dolan Snigdha Chaturvedi Jianfeng Gao HILM 71 5 0 04 Dec 2022
Toward Efficient Language Model Pretraining and Downstream Adaptation via Self-Evolution: A Case Study on SuperGLUE Qihuang Zhong Liang Ding Yibing Zhan Yu Qiao Yonggang Wen ... Yixin Chen Xinbo Gao Steven C. H. Hoi Xiaoou Tang Dacheng Tao VLM ELM 132 35 0 04 Dec 2022
MiLMo:Minority Multilingual Pre-trained Language Model Sisi Liu Hanru Shi Xinhe Yu Wugedele Bao Yuan Sun Xiaobing Zhao 83 0 0 04 Dec 2022
Utilizing Background Knowledge for Robust Reasoning over Traffic Situations Jiarui Zhang Filip Ilievski Aravinda Kollaa Jonathan M Francis Kaixin Ma A. Oltramari 59 2 0 04 Dec 2022
Harnessing label semantics to extract higher performance under noisy label for Company to Industry matching Apoorva Jaiswal A. Mitra 100 0 0 03 Dec 2022
T-STAR: Truthful Style Transfer using AMR Graph as Intermediate Representation Anubhav Jangra Preksha Nema A. Raghuveer 51 7 0 03 Dec 2022
CoP: Factual Inconsistency Detection by Controlling the Preference Shuaijie She Xiang Geng Shujian Huang Jiajun Chen 103 5 0 03 Dec 2022
Modeling Label Correlations for Ultra-Fine Entity Typing with Neural Pairwise Conditional Random Field Chengyue Jiang Yong Jiang Weiqi Wu Pengjun Xie Kewei Tu 59 5 0 03 Dec 2022
Exploring the Limits of Differentially Private Deep Learning with Group-wise Clipping Jiyan He Xuechen Li Da Yu Huishuai Zhang Janardhan Kulkarni Y. Lee A. Backurs Nenghai Yu Jiang Bian 126 49 0 03 Dec 2022
Event knowledge in large language models: the gap between the impossible and the unlikely Carina Kauf Anna A. Ivanova Giulia Rambelli Emmanuele Chersoni Jingyuan Selena She Zawad Chowdhury Evelina Fedorenko Alessandro Lenci 124 70 0 02 Dec 2022
NarraSum: A Large-Scale Dataset for Abstractive Narrative Summarization Chao Zhao Faeze Brahman Kaiqiang Song Wenlin Yao Dian Yu Snigdha Chaturvedi HILM 95 8 0 02 Dec 2022
ColD Fusion: Collaborative Descent for Distributed Multitask Finetuning Shachar Don-Yehiya Elad Venezian Colin Raffel Noam Slonim Yoav Katz Leshem Choshen MoMe 109 55 0 02 Dec 2022
Improving Iterative Text Revision by Learning Where to Edit from Other Revision Tasks Zae Myung Kim Wanyu Du Vipul Raheja Dhruv Kumar Dongyeop Kang 99 18 0 02 Dec 2022
Nonparametric Masked Language Modeling Sewon Min Weijia Shi M. Lewis Xilun Chen Wen-tau Yih Hannaneh Hajishirzi Luke Zettlemoyer RALM 168 51 0 02 Dec 2022
Legal Prompting: Teaching a Language Model to Think Like a Lawyer Fang Yu Lee Quartey Frank Schilder ELM LRM 61 70 0 02 Dec 2022
Exploring Faithful Rationale for Multi-hop Fact Verification via Salience-Aware Graph Learning Jiasheng Si Yingjie Zhu Deyu Zhou 109 16 0 02 Dec 2022
Systematic Analysis for Pretrained Language Model Priming for Parameter-Efficient Fine-tuning Shih-Cheng Huang Shi Wang Min-Han Shih Saurav Sahay Hung-yi Lee 101 0 0 02 Dec 2022
Relation-Aware Language-Graph Transformer for Question Answering Jinyoung Park Hyeong Kyu Choi Juyeon Ko Hyeon-ju Park Ji-Hoon Kim Jisu Jeong Kyungmin Kim Hyunwoo J. Kim KELM LMTD ViT 58 10 0 02 Dec 2022
UniKGQA: Unified Retrieval and Reasoning for Solving Multi-hop Question Answering Over Knowledge Graph Jinhao Jiang Kun Zhou Wayne Xin Zhao Ji-Rong Wen RALM 134 84 0 02 Dec 2022
AGRO: Adversarial Discovery of Error-prone groups for Robust Optimization Bhargavi Paranjape Pradeep Dasigi Vivek Srikumar Luke Zettlemoyer Hannaneh Hajishirzi 103 8 0 02 Dec 2022
Focus! Relevant and Sufficient Context Selection for News Image Captioning Mingyang Zhou Grace Luo Anna Rohrbach Zhou Yu CLIP 81 13 0 01 Dec 2022
UniT3D: A Unified Transformer for 3D Dense Captioning and Visual Grounding Dave Zhenyu Chen Ronghang Hu Xinlei Chen Matthias Nießner Angel X. Chang 120 54 0 01 Dec 2022
Scaling Language-Image Pre-training via Masking Yanghao Li Haoqi Fan Ronghang Hu Christoph Feichtenhofer Kaiming He CLIP VLM 115 330 0 01 Dec 2022
CliMedBERT: A Pre-trained Language Model for Climate and Health-related Text Babak Jalalzadeh Hasan A. BellSadid 76 7 0 01 Dec 2022
Embedding generation for text classification of Brazilian Portuguese user reviews: from bag-of-words to transformers F. Souza J. B. O. S. Filho 55 7 0 01 Dec 2022
CultureBERT: Measuring Corporate Culture With Transformer-Based Language Models Sebastian P. Koch Stefan Pasch VLM 77 5 0 01 Dec 2022
IRRGN: An Implicit Relational Reasoning Graph Network for Multi-turn Response Selection Jingcheng Deng Hengwei Dai Xuewei Guo Yuanchen Ju Wei Peng LRM 72 2 0 01 Dec 2022
Language Model Pre-training on True Negatives Zhuosheng Zhang Hai Zhao Masao Utiyama Eiichiro Sumita 75 2 0 01 Dec 2022
Learning to Select from Multiple Options Jiangshu Du Wenpeng Yin Congying Xia Philip S. Yu 107 7 0 01 Dec 2022
Towards Practical Few-shot Federated NLP Dongqi Cai Yaozong Wu Haitao Yuan Shangguang Wang F. Lin Mengwei Xu FedML 84 6 0 01 Dec 2022
CREPE: Open-Domain Question Answering with False Presuppositions Xinyan Velocity Yu Sewon Min Luke Zettlemoyer Hannaneh Hajishirzi 105 54 0 30 Nov 2022
BudgetLongformer: Can we Cheaply Pretrain a SotA Legal Language Model From Scratch? Joel Niklaus Daniele Giofré 75 12 0 30 Nov 2022
Transformers are Short Text Classifiers: A Study of Inductive Short Text Classifiers on Benchmarks and Real-world Datasets Fabian Karl A. Scherp VLM 76 20 0 30 Nov 2022
Revisiting text decomposition methods for NLI-based factuality scoring of summaries John Glover Federico Fancellu V. Jagannathan Matthew R. Gormley Thomas Schaaf HILM 100 17 0 30 Nov 2022
Protein Language Models and Structure Prediction: Connection and Progression Bozhen Hu Jun Xia Jiangbin Zheng Cheng Tan Yufei Huang Yongjie Xu Stan Z. Li 74 41 0 30 Nov 2022
SPARTAN: Sparse Hierarchical Memory for Parameter-Efficient Transformers Ameet Deshpande Md Arafat Sultan Anthony Ferritto Ashwin Kalyan Karthik Narasimhan Avirup Sil MoE 84 1 0 29 Nov 2022
Chaining Simultaneous Thoughts for Numerical Reasoning Zhihong Shao Fei Huang Minlie Huang AIMat AI4CE 71 18 0 29 Nov 2022
BARTSmiles: Generative Masked Language Models for Molecular Representations Gayane Chilingaryan Hovhannes Tamoyan Ani Tevosyan N. Babayan L. Khondkaryan Karen Hambardzumyan Zaven Navoyan Hrant Khachatrian Armen Aghajanyan SSL 111 28 0 29 Nov 2022
AutoCAD: Automatically Generating Counterfactuals for Mitigating Shortcut Learning Jiaxin Wen Yeshuang Zhu Jinchao Zhang Jie Zhou Minlie Huang CML AAML 123 9 0 29 Nov 2022
Model Extraction Attack against Self-supervised Speech Models Tsung-Yuan Hsu Chen-An Li Tung-Yu Wu Hung-yi Lee 55 1 0 29 Nov 2022
Survey on Self-Supervised Multimodal Representation Learning and Foundation Models Sushil Thapa AI4TS SSL 50 1 0 29 Nov 2022
On the Effectiveness of Parameter-Efficient Fine-Tuning Z. Fu Haoran Yang Anthony Man-Cho So Wai Lam Lidong Bing Nigel Collier 82 162 0 28 Nov 2022
Predicting Digital Asset Prices using Natural Language Processing: a survey Trang Tran 62 1 0 28 Nov 2022
DQ-DETR: Dual Query Detection Transformer for Phrase Extraction and Grounding Siyi Liu Yaoyuan Liang Feng Li Shijia Huang Hao Zhang Hang Su Jun Zhu Lei Zhang ObjD 105 28 0 28 Nov 2022