RoBERTa: A Robustly Optimized BERT Pretraining Approach

26 July 2019

Luke Zettlemoyer

Papers citing "RoBERTa: A Robustly Optimized BERT Pretraining Approach"

50 / 10,807 papers shown

Title
Correlational Image Modeling for Self-Supervised Visual Pre-Training Wei Li Jiahao Xie Chen Change Loy SSL 96 12 0 22 Mar 2023
Can We Identify Stance Without Target Arguments? A Study for Rumour Stance Classification Yue Li Carolina Scarton 64 0 0 22 Mar 2023
Man vs the machine: The Struggle for Effective Text Anonymisation in the Age of Large Language Models Constantinos Patsakis Nikolaos Lykousas 59 9 0 22 Mar 2023
GrapeQA: GRaph Augmentation and Pruning to Enhance Question-Answering Dhaval Taunk Lakshya Khanna Pavan Kandru Vasudeva Varma Charu Sharma Makarand Tapaswi 78 24 0 22 Mar 2023
Generate labeled training data using Prompt Programming and GPT-3. An example of Big Five Personality Classification Eason Chen 40 3 0 22 Mar 2023
Fundamentals of Generative Large Language Models and Perspectives in Cyber-Defense Andrei Kucharavy Z. Schillaci Loic Maréchal Maxime Wursch Ljiljana Dolamic Remi Sabonnadiere Dimitri Percia David Alain Mermoud Vincent Lenders ELM AI4CE 83 33 0 21 Mar 2023
Is BERT Blind? Exploring the Effect of Vision-and-Language Pretraining on Visual Language Understanding Morris Alper Michael Fiman Hadar Averbuch-Elor VLM LRM 80 15 0 21 Mar 2023
cTBLS: Augmenting Large Language Models with Conversational Tables Anirudh S. Sundar Larry Heck LMTD 65 9 0 21 Mar 2023
Logical Reasoning over Natural Language as Knowledge Representation: A Survey Zonglin Yang Xinya Du Rui Mao Jinjie Ni Min Zhang LRM ReLM 78 27 0 21 Mar 2023
ChatGPT and a New Academic Reality: Artificial Intelligence-Written Research Papers and the Ethics of the Large Language Models in Scholarly Publishing Brady Lund Ting Wang Nishith Reddy Mannuru Bing Nie S. Shimray Ziang Wang AI4CE 94 530 0 21 Mar 2023
A Complete Survey on Generative AI (AIGC): Is ChatGPT from GPT-4 to GPT-5 All You Need? Chaoning Zhang Chenshuang Zhang Sheng Zheng Yu Qiao Chenghao Li ... Lik-Hang Lee Yang Yang Heng Tao Shen In So Kweon Choong Seon Hong 190 170 0 21 Mar 2023
Large AI Models in Health Informatics: Applications, Challenges, and the Future Jianing Qiu Lin Li Jiankai Sun Jiachuan Peng Peilun Shi ... Bo Xiao Wu Yuan Ningli Wang Dong Xu Benny Lo AI4MH LM&MA 114 142 0 21 Mar 2023
Language Model Behavior: A Comprehensive Survey Tyler A. Chang Benjamin Bergen VLM LRM LM&MA 117 109 0 20 Mar 2023
Did You Train on My Dataset? Towards Public Dataset Protection with Clean-Label Backdoor Watermarking Ruixiang Tang Qizhang Feng Ninghao Liu Fan Yang Helen Zhou 97 42 0 20 Mar 2023
EVA-02: A Visual Representation for Neon Genesis Yuxin Fang Quan-Sen Sun Xinggang Wang Tiejun Huang Xinlong Wang Yue Cao VLM ViT CLIP 135 289 0 20 Mar 2023
Context-faithful Prompting for Large Language Models Wenxuan Zhou Sheng Zhang Hoifung Poon Muhao Chen KELM 61 65 0 20 Mar 2023
Learning Semantic Text Similarity to rank Hypernyms of Financial Terms Sohom Ghosh Ankush Chopra S. Naskar 36 2 0 20 Mar 2023
Conversation Modeling to Predict Derailment Jiaqing Yuan Munindar P. Singh 64 4 0 20 Mar 2023
DeID-GPT: Zero-shot Medical Text De-Identification by GPT-4 Zheng-Long Liu Yue Huang Xiao-Xing Yu Lu Zhang Zihao Wu ... Dinggang Shen Quanzheng Li Tianming Liu Dajiang Zhu Xiang Li LM&MA MedIm 129 179 0 20 Mar 2023
Character, Word, or Both? Revisiting the Segmentation Granularity for Chinese Pre-trained Language Models Xinnian Liang Zefan Zhou Hui Huang Shuangzhi Wu Tong Xiao Muyun Yang Zhoujun Li Chao Bian VLM 65 2 0 20 Mar 2023
Audio-Text Models Do Not Yet Leverage Natural Language Ho-Hsiang Wu Oriol Nieto J. P. Bello Justin Salamon VLM 74 33 0 19 Mar 2023
Multi-modal Facial Action Unit Detection with Large Pre-trained Models for the 5th Competition on Affective Behavior Analysis in-the-wild Yufeng Yin Minh Tran Di Chang Xinrui Wang M. Soleymani CVBM 60 15 0 19 Mar 2023
Extracting Incidents, Effects, and Requested Advice from MeToo Posts Vaibhav Garg Jiaqing Yuan Rujie Xi Munindar P. Singh 22 1 0 19 Mar 2023
PACO: Provocation Involving Action, Culture, and Oppression Vaibhav Garg Ganning Xu Munindar P. Singh 35 0 0 19 Mar 2023
Two Kinds of Recall Yoav Goldberg 50 1 0 19 Mar 2023
AdaLoRA: Adaptive Budget Allocation for Parameter-Efficient Fine-Tuning Qingru Zhang Minshuo Chen Alexander Bukharin Nikos Karampatziakis Pengcheng He Yu Cheng Weizhu Chen Tuo Zhao 71 129 0 18 Mar 2023
GazeReader: Detecting Unknown Word Using Webcam for English as a Second Language (ESL) Learners Jiexin Ding Bowen Zhao Yuqi Huang Yuntao wang Yuanchun Shi 33 9 0 18 Mar 2023
An Empirical Study of Pre-trained Language Models in Simple Knowledge Graph Question Answering Nan Hu Yike Wu Guilin Qi Dehai Min Jiaoyan Chen Jeff Z. Pan Z. Ali ELM AI4MH 86 40 0 18 Mar 2023
Revisiting Automatic Question Summarization Evaluation in the Biomedical Domain Hongyi Yuan Yaoyun Zhang Fei Huang Songfang Huang 77 1 0 18 Mar 2023
Conversational Tree Search: A New Hybrid Dialog Task Dirk Vath Lindsey Vanderlyn Ngoc Thang Vu 71 9 0 17 Mar 2023
GlueGen: Plug and Play Multi-modal Encoders for X-to-image Generation Can Qin Ning Yu Chen Xing Shu Zhen Zhang Zeyuan Chen Stefano Ermon Yun Fu Caiming Xiong Ran Xu DiffM 129 21 0 17 Mar 2023
Trained on 100 million words and still in shape: BERT meets British National Corpus David Samuel Andrey Kutuzov Lilja Øvrelid Erik Velldal 101 32 0 17 Mar 2023
Neural Architecture Search for Effective Teacher-Student Knowledge Transfer in Language Models Aashka Trivedi Takuma Udagawa Michele Merler Yikang Shen Yousef El-Kurdi Bishwaranjan Bhattacharjee 84 7 0 16 Mar 2023
Revealing Weaknesses of Vietnamese Language Models Through Unanswerable Questions in Machine Reading Comprehension Son Quoc Tran Phong Nguyen-Thuan Do Kiet Van Nguyen Ngan Luu-Thuy Nguyen 70 0 0 16 Mar 2023
cito: An R package for training neural networks using torch Christian Amesoeder F. Hartig Maximilian Pichler 54 3 0 16 Mar 2023
Logical Implications for Visual Question Answering Consistency Sergio Tascon-Morales Pablo Márquez-Neila Raphael Sznitman 81 9 0 16 Mar 2023
SheffieldVeraAI at SemEval-2023 Task 3: Mono and multilingual approaches for news genre, topic and persuasion technique classification Ben Wu Olesya Razuvayevskaya Freddy Heppell João A. Leite Carolina Scarton Kalina Bontcheva Xingyi Song 42 9 0 16 Mar 2023
Tollywood Emotions: Annotation of Valence-Arousal in Telugu Song Lyrics R. G. R. Shanker B. Gupta BV Koushik Vinoo Alluri 38 1 0 16 Mar 2023
SmartBERT: A Promotion of Dynamic Early Exiting Mechanism for Accelerating BERT Inference Boren Hu Yun Zhu Jiacheng Li Siliang Tang 58 9 0 16 Mar 2023
Not Seen, Not Heard in the Digital World! Measuring Privacy Practices in Children's Apps Ruoxi Sun Minhui Xue Gareth Tyson Shuo Wang S. Çamtepe Surya Nepal 68 8 0 16 Mar 2023
SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models Potsawee Manakul Adian Liusie Mark Gales HILM LRM 245 448 0 15 Mar 2023
Large Language Model Is Not a Good Few-shot Information Extractor, but a Good Reranker for Hard Samples! Yubo Ma Yixin Cao YongChing Hong Aixin Sun RALM 179 157 0 15 Mar 2023
Attention-likelihood relationship in transformers Valeria Ruscio Valentino Maiorca Fabrizio Silvestri 23 1 0 15 Mar 2023
Neuro-symbolic Commonsense Social Reasoning David Chanin Anthony Hunter NAI LRM 69 3 0 14 Mar 2023
Clinical Concept and Relation Extraction Using Prompt-based Machine Reading Comprehension C.A.I. Peng Xi Yang Zehao Yu Jiang Bian W. Hogan Yonghui Wu 176 24 0 14 Mar 2023
Contextualized Medication Information Extraction Using Transformer-based Deep Learning Architectures Aokun Chen Zehao Yu Xi Yang Yi Guo Jiang Bian Yonghui Wu 60 24 0 14 Mar 2023
MEDBERT.de: A Comprehensive German BERT Model for the Medical Domain Keno K. Bressem Jens-Michalis Papaioannou Paul Grundmann Florian Borchert Lisa Christine Adams ... Moritz Augustin Lennart Grosser Marcus R. Makowski Hugo J. W. L. Aerts Alexander Loser AI4MH 63 34 0 14 Mar 2023
Do Transformers Parse while Predicting the Masked Word? Haoyu Zhao A. Panigrahi Rong Ge Sanjeev Arora 153 35 0 14 Mar 2023
Finding the Needle in a Haystack: Unsupervised Rationale Extraction from Long Text Classifiers Kamil Bujel Andrew Caines H. Yannakoudakis Marek Rei AI4TS 38 1 0 14 Mar 2023
Cross-lingual Alzheimer's Disease detection based on paralinguistic and pre-trained features Xuchu Chen Yujiang Pu Jinpeng Li Weiqiang Zhang 49 14 0 14 Mar 2023