RoBERTa: A Robustly Optimized BERT Pretraining Approach

26 July 2019

Luke Zettlemoyer

Papers citing "RoBERTa: A Robustly Optimized BERT Pretraining Approach"

50 / 10,805 papers shown

Title
oBERTa: Improving Sparse Transfer Learning via improved initialization, distillation, and pruning regimes Daniel Fernando Campos Alexandre Marques Mark Kurtz Chengxiang Zhai VLM AAML 52 2 0 30 Mar 2023
DERA: Enhancing Large Language Model Completions with Dialog-Enabled Resolving Agents Varun Nair Elliot Schumacher Geoffrey Tso Anitha Kannan VLM 71 64 0 30 Mar 2023
P-Transformer: A Prompt-based Multimodal Transformer Architecture For Medical Tabular Data Y. Ruan Xiang Lan Daniel J. Tan H. Abdullah Mengling Feng LMTD MedIm 149 1 0 30 Mar 2023
BERT4ETH: A Pre-trained Transformer for Ethereum Fraud Detection Sihao Hu Zhen Zhang B. Luo Shengliang Lu Bingsheng He Ling Liu 74 44 0 29 Mar 2023
How do decoding algorithms distribute information in dialogue responses? Saranya Venkatraman He He David Reitter 52 5 0 29 Mar 2023
BEVERS: A General, Simple, and Performant Framework for Automatic Fact Verification Mitchell DeHaven Stephen Scott 65 23 0 29 Mar 2023
PMAA: A Progressive Multi-scale Attention Autoencoder Model for High-performance Cloud Removal from Multi-temporal Satellite Imagery Xuechao Zou Keqin Li Junliang Xing Pin Tao Yachao Cui 62 15 0 29 Mar 2023
Hierarchical Video-Moment Retrieval and Step-Captioning Abhaysinh Zala Jaemin Cho Satwik Kottur Xilun Chen Barlas Ouguz Yasher Mehdad Joey Tianyi Zhou 3DV 98 54 0 29 Mar 2023
ChatGPT or academic scientist? Distinguishing authorship with over 99% accuracy using off-the-shelf machine learning tools H. Desaire Aleesa E Chua Madeline Isom Romana Jarosova David C. Hua DeLMO 60 6 0 28 Mar 2023
LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attention Renrui Zhang Jiaming Han Chris Liu Peng Gao Aojun Zhou Xiangfei Hu Shilin Yan Pan Lu Hongsheng Li Yu Qiao MLLM 186 787 0 28 Mar 2023
Exploring Natural Language Processing Methods for Interactive Behaviour Modelling Guanhua Zhang Matteo Bortoletto Zhiming Hu Lei Shi Mihai Bâce Andreas Bulling 46 3 0 28 Mar 2023
SELF-VS: Self-supervised Encoding Learning For Video Summarization Hojjat Mokhtarabadi Kaveh Bahraman M. Hosseinzadeh M. Eftekhari AI4TS SSL ViT 45 0 0 28 Mar 2023
A Multi-Granularity Matching Attention Network for Query Intent Classification in E-commerce Retrieval Chunyuan Yuan Yiming Qiu Mingming Li Haiqing Hu Songlin Wang Sulong Xu 23 9 0 28 Mar 2023
Soft-prompt tuning to predict lung cancer using primary care free-text Dutch medical notes Auke Elfrink Iacopo Vagliano A. Abu-Hanna Iacer Calixto 57 5 0 28 Mar 2023
One Adapter for All Programming Languages? Adapter Tuning for Code Search and Summarization Deze Wang Boxing Chen Shanshan Li Wei Luo Shaoliang Peng Wei Dong Xiang-ke Liao 53 41 0 28 Mar 2023
Explicit Planning Helps Language Models in Logical Reasoning Hongyu Zhao Kangrui Wang Mo Yu Hongyuan Mei LRM ReLM 130 17 0 28 Mar 2023
ChatGPT as a Factual Inconsistency Evaluator for Text Summarization Zheheng Luo Qianqian Xie Sophia Ananiadou ELM HILM ALM 92 80 0 27 Mar 2023
Unlocking the Potential of ChatGPT: A Comprehensive Exploration of its Applications, Advantages, Limitations, and Future Directions in Natural Language Processing Walid Hariri AI4MH LM&MA 180 94 0 27 Mar 2023
Improving Dual-Encoder Training through Dynamic Indexes for Negative Mining Nicholas Monath Manzil Zaheer Kelsey R. Allen Andrew McCallum 72 6 0 27 Mar 2023
Gazeformer: Scalable, Effective and Fast Prediction of Goal-Directed Human Attention Sounak Mondal Zhibo Yang Seoyoung Ahn Dimitris Samaras G. Zelinsky Minh Hoai 89 31 0 27 Mar 2023
An Information Extraction Study: Take In Mind the Tokenization! Christos Theodoropoulos Marie-Francine Moens 54 6 0 27 Mar 2023
InterviewBot: Real-Time End-to-End Dialogue System to Interview Students for College Admission Zihao Wang Nathan Keyes Terry Crawford Jinho Choi 65 0 0 27 Mar 2023
Borrowing Human Senses: Comment-Aware Self-Training for Social Media Multimodal Classification Chunpu Xu Jing Li VLM 62 5 0 27 Mar 2023
Continuous Intermediate Token Learning with Implicit Motion Manifold for Keyframe Based Motion Interpolation Clinton Mo Kun Hu Chengjiang Long Zhiyong Wang 72 14 0 27 Mar 2023
Adapting Pretrained Language Models for Solving Tabular Prediction Problems in the Electronic Health Record C. McMaster D. Liew Douglas E. V. Pires 109 5 0 27 Mar 2023
Meeting Action Item Detection with Regularized Context Modeling Jiaqing Liu Chong Deng Qinglin Zhang Qian Chen Wen Wang 21 0 0 27 Mar 2023
SEM-POS: Grammatically and Semantically Correct Video Captioning Asmar Nadeem A. Hilton R. Dawes Graham A. Thomas A. Mustafa 73 8 0 26 Mar 2023
MGTBench: Benchmarking Machine-Generated Text Detection Xinlei He Xinyue Shen Zhenpeng Chen Michael Backes Yang Zhang DeLMO 134 114 0 26 Mar 2023
Koala: An Index for Quantifying Overlaps with Pre-training Corpora Thuy-Trang Vu Xuanli He Gholamreza Haffari Ehsan Shareghi CLL 81 15 0 26 Mar 2023
CelebV-Text: A Large-Scale Facial Text-Video Dataset Jianhui Yu Hao Zhu Liming Jiang Chen Change Loy Weidong (Tom) Cai Wayne Wu 77 62 0 26 Mar 2023
Task-oriented Memory-efficient Pruning-Adapter Guorun Wang Jun Yang Yaoru Sun 48 4 0 26 Mar 2023
SASS: Data and Methods for Subject Aware Sentence Simplification Bradford T. Windsor Luke Martin Anand Tyagi 65 0 0 26 Mar 2023
Automatic Generation of Multiple-Choice Questions Cheng Zhang 56 7 0 25 Mar 2023
Energy-efficient Task Adaptation for NLP Edge Inference Leveraging Heterogeneous Memory Architectures Zirui Fu Aleksandre Avaliani M. Donato 82 1 0 25 Mar 2023
Informed Machine Learning, Centrality, CNN, Relevant Document Detection, Repatriation of Indigenous Human Remains M. A. Bashar R. Nayak G. Knapman Paul Turnbull C. Fforde 93 1 0 25 Mar 2023
COFFEE: A Contrastive Oracle-Free Framework for Event Extraction Meiru Zhang Yixuan Su Zaiqiao Meng Z. Fu Nigel Collier 75 4 0 25 Mar 2023
Sem4SAP: Synonymous Expression Mining From Open Knowledge Graph For Language Model Synonym-Aware Pretraining Zhouhong Gu Sihang Jiang Wenhao Huang Jiaqing Liang Hongwei Feng Yanghua Xiao VLM 76 1 0 25 Mar 2023
SmartBook: AI-Assisted Situation Report Generation for Intelligence Analysts R. Reddy Daniel Lee Yi R. Fung Khanh Duy Nguyen Qi Zeng Manling Li Ziqi Wang Clare R. Voss Heng Ji 67 6 0 25 Mar 2023
SIGMORPHON 2023 Shared Task of Interlinear Glossing: Baseline Model Michael Ginn 51 7 0 24 Mar 2023
Accelerating Vision-Language Pretraining with Free Language Modeling Teng Wang Yixiao Ge Feng Zheng Ran Cheng Ying Shan Xiaohu Qie Ping Luo VLM MLLM 118 10 0 24 Mar 2023
MUG: A General Meeting Understanding and Generation Benchmark Qinglin Zhang Chong Deng Jiaqing Liu Hai Yu Qian Chen Wen Wang Zhijie Yan Jinglin Liu Yi Ren Zhou Zhao 83 8 0 24 Mar 2023
Towards Fair Patient-Trial Matching via Patient-Criterion Level Fairness Constraint Chia-Yuan Chang Jiayi Yuan Sirui Ding Qiaoyu Tan Kai Zhang Xiaoqian Jiang Helen Zhou Na Zou FaML 74 9 0 24 Mar 2023
Towards Making the Most of ChatGPT for Machine Translation Keqin Peng Liang Ding Qihuang Zhong Li Shen Xuebo Liu Min Zhang Y. Ouyang Dacheng Tao LRM 152 233 0 24 Mar 2023
Large Language Models for Healthcare Data Augmentation: An Example on Patient-Trial Matching Jiayi Yuan Ruixiang Tang Xiaoqian Jiang Helen Zhou LM&MA 77 42 0 24 Mar 2023
How Does Attention Work in Vision Transformers? A Visual Analytics Attempt Yiran Li Junpeng Wang Xin Dai Liang Wang Chin-Chia Michael Yeh Yan Zheng Wei Zhang Kwan-Liu Ma ViT 59 26 0 24 Mar 2023
Promptable Game Models: Text-Guided Game Simulation via Masked Diffusion Models Willi Menapace Aliaksandr Siarohin Stéphane Lathuilière Panos Achlioptas Vladislav Golyanik Sergey Tulyakov Elisa Ricci LM&Ro VGen DiffM 107 16 0 23 Mar 2023
Multi-View Zero-Shot Open Intent Induction from Dialogues: Multi Domain Batch and Proxy Gradient Transfer Hyukhun Koh Haesung Pyun Nakyeong Yang Kyomin Jung 104 1 0 23 Mar 2023
Retrieval-Augmented Classification with Decoupled Representation Xinnian Liang Shuangzhi Wu Hui Huang Jiaqi Bai Chao Bian Zhoujun Li 48 0 0 23 Mar 2023
Towards Better Dynamic Graph Learning: New Architecture and Unified Library Le Yu Leilei Sun Bowen Du Weifeng Lv AI4CE 104 121 0 23 Mar 2023
JaCoText: A Pretrained Model for Java Code-Text Generation Jessica Nayeli López Espejel Mahaman Sanoussi Yahaya Alassan Walid Dahhane E. Ettifouri 56 4 0 22 Mar 2023