RoBERTa: A Robustly Optimized BERT Pretraining Approach

26 July 2019

Luke Zettlemoyer

Papers citing "RoBERTa: A Robustly Optimized BERT Pretraining Approach"

50 / 10,793 papers shown

Title
VNHSGE: VietNamese High School Graduation Examination Dataset for Large Language Models Dao Xuan-Quy Le Ngoc-Bich Vo The-Duy Phan Xuan-Dung Ngo Bac-Bien Nguyen Van-Tien Nguyen Thi-My-Thanh Nguyen Hong-Phuoc 63 16 0 20 May 2023
Learning to Compose Representations of Different Encoder Layers towards Improving Compositional Generalization Lei Lin Shuangtao Li Yafang Zheng Biao Fu Shantao Liu Yidong Chen Xiaodon Shi CoGe 90 3 0 20 May 2023
Learning Horn Envelopes via Queries from Large Language Models Sophie Blum Raoul Koudijs Ana Ozaki Samia Touileb 62 1 0 20 May 2023
Lifting the Curse of Capacity Gap in Distilling Language Models Chen Zhang Yang Yang Jiahao Liu Jingang Wang Yunsen Xian Benyou Wang Dawei Song MoE 69 20 0 20 May 2023
ESCOXLM-R: Multilingual Taxonomy-driven Pre-training for the Job Market Domain Mike Zhang Rob van der Goot Barbara Plank 63 16 0 20 May 2023
"What do others think?": Task-Oriented Conversational Modeling with Subjective Knowledge Chao Zhao Spandana Gella Seokhwan Kim Di Jin Devamanyu Hazarika Alexandros Papangelis Behnam Hedayatnia Mahdi Namazifar Yang Liu Dilek Z. Hakkani-Tür 94 7 0 20 May 2023
Prefix Propagation: Parameter-Efficient Tuning for Long Sequences Jonathan Li Will Aitken R. Bhambhoria Xiao-Dan Zhu 47 15 0 20 May 2023
DisCo: Distilled Student Models Co-training for Semi-supervised Text Mining Weifeng Jiang Qianren Mao Chenghua Lin Jianxin Li Ting Deng Weiyi Yang Ziyi Wang 36 3 0 20 May 2023
Polar Ducks and Where to Find Them: Enhancing Entity Linking with Duck Typing and Polar Box Embeddings Mattia Atzeni Mikhail Plekhanov F. Dreyer Nora Kassner Simone Merello Louis Martin Nicola Cancedda 90 2 0 19 May 2023
BOLT: Fast Energy-based Controlled Text Generation with Tunable Biases Xin Liu Muhammad Khalifa Lu Wang 115 20 0 19 May 2023
Deep Learning Approaches to Lexical Simplification: A Survey Kai North Tharindu Ranasinghe Matthew Shardlow Marcos Zampieri 50 15 0 19 May 2023
Interpretable Word Sense Representations via Definition Generation: The Case of Semantic Change Analysis Mario Giulianelli Iris Luden Raquel Fernández Andrey Kutuzov 72 26 0 19 May 2023
Reducing Sequence Length by Predicting Edit Operations with Large Language Models Masahiro Kaneko Naoaki Okazaki 69 4 0 19 May 2023
SeeGULL: A Stereotype Benchmark with Broad Geo-Cultural Coverage Leveraging Generative Models Akshita Jha Aida Mostafazadeh Davani Chandan K. Reddy Shachi Dave Vinodkumar Prabhakaran Sunipa Dev 87 50 0 19 May 2023
ReTAG: Reasoning Aware Table to Analytic Text Generation Deepanway Ghosal Preksha Nema A. Raghuveer LMTD LRM 78 5 0 19 May 2023
Solving NLP Problems through Human-System Collaboration: A Discussion-based Approach Masahiro Kaneko Graham Neubig Naoaki Okazaki 112 6 0 19 May 2023
LLM-Pruner: On the Structural Pruning of Large Language Models Xinyin Ma Gongfan Fang Xinchao Wang 175 446 0 19 May 2023
CCT-Code: Cross-Consistency Training for Multilingual Clone Detection and Code Search Nikita Sorokin Dmitry Abulkhanov Sergey I. Nikolenko Valentin Malykh 64 3 0 19 May 2023
Searching by Code: a New SearchBySnippet Dataset and SnippeR Retrieval Model for Searching by Code Snippets I. Sedykh Dmitry Abulkhanov Nikita Sorokin Sergey I. Nikolenko Valentin Malykh 67 2 0 19 May 2023
Mitigating Backdoor Poisoning Attacks through the Lens of Spurious Correlation Xuanli He Xingliang Yuan Jun Wang Benjamin I. P. Rubinstein Trevor Cohn AAML 88 20 0 19 May 2023
Speech-Text Dialog Pre-training for Spoken Dialog Understanding with Explicit Cross-Modal Alignment Tianshu Yu Haoyu Gao Ting-En Lin Min Yang Yuchuan Wu Wen-Cheng Ma Chao Wang Fei Huang Yongbin Li 68 23 0 19 May 2023
Decouple knowledge from parameters for plug-and-play language modeling Xin Cheng Yankai Lin Preslav Nakov Dongyan Zhao Rui Yan KELM 86 2 0 19 May 2023
Constructing Word-Context-Coupled Space Aligned with Associative Knowledge Relations for Interpretable Language Modeling Fanyu Wang Zhenping Xie 79 0 0 19 May 2023
PlugMed: Improving Specificity in Patient-Centered Medical Dialogue Generation using In-Context Learning Chengfeng Dou Zhi Jin Wenpin Jiao Haiyan Zhao Zhengwei Tao Yongqiang Zhao LM&MA MedIm 106 8 0 19 May 2023
Enhancing Personalized Dialogue Generation with Contrastive Latent Variables: Combining Sparse and Dense Persona Yihong Tang Bo Wang Miao Fang Dongming Zhao Kun Huang Ruifang He Yuexian Hou 91 23 0 19 May 2023
CM-MaskSD: Cross-Modality Masked Self-Distillation for Referring Image Segmentation Wenxuan Wang Jing Liu Xingjian He Yisi Zhang Cheng Chen Jiachen Shen Yan Zhang Jiangyun Li 70 14 0 19 May 2023
Self-Agreement: A Framework for Fine-tuning Language Models to Find Agreement among Diverse Opinions Shiyao Ding Takayuki Ito SyDa 35 7 0 19 May 2023
Zero-Shot Text Classification via Self-Supervised Tuning Chaoqun Liu Wenxuan Zhang Guizhen Chen Xiaobao Wu Anh Tuan Luu Chip Hong Chang Lidong Bing VLM 85 11 0 19 May 2023
Phonetic and Prosody-aware Self-supervised Learning Approach for Non-native Fluency Scoring Kaiqi Fu Shaojun Gao Shuju Shi Xiaohai Tian Wei Li Zejun Ma 47 2 0 19 May 2023
Federated Foundation Models: Privacy-Preserving and Collaborative Learning for Large Models Sixing Yu J. P. Muñoz Ali Jannesari AI4CE 78 51 0 19 May 2023
Fast-StrucTexT: An Efficient Hourglass Transformer with Modality-guided Dynamic Token Merge for Document Understanding Mingliang Zhai Yulin Li Xiameng Qin Chen Yi Qunyi Xie Chengquan Zhang Kun Yao Yuwei Wu Yunde Jia 42 8 0 19 May 2023
A Survey of Safety and Trustworthiness of Large Language Models through the Lens of Verification and Validation Xiaowei Huang Wenjie Ruan Wei Huang Gao Jin Yizhen Dong ... Sihao Wu Peipei Xu Dengyu Wu André Freitas Mustafa A. Mustafa ALM 132 96 0 19 May 2023
Unsupervised Domain-agnostic Fake News Detection using Multi-modal Weak Signals Amila Silva Ling Luo S. Karunasekera C. Leckie 104 5 0 18 May 2023
In the Name of Fairness: Assessing the Bias in Clinical Record De-identification Yuxin Xiao S. Lim Tom Pollard Marzyeh Ghassemi 79 16 0 18 May 2023
Recent Trends in Unsupervised Summarization Mohammad Khosravani Amine Trabelsi 85 0 0 18 May 2023
ONE-PEACE: Exploring One General Representation Model Toward Unlimited Modalities Peng Wang Shijie Wang Junyang Lin Shuai Bai Xiaohuan Zhou Jingren Zhou Xinggang Wang Chang Zhou VLM MLLM ObjD 151 122 0 18 May 2023
Aligning Instruction Tasks Unlocks Large Language Models as Zero-Shot Relation Extractors Kai Zhang Bernal Jiménez Gutiérrez Yu-Chuan Su 94 73 0 18 May 2023
SimOAP: Improve Coherence and Consistency in Persona-based Dialogue Generation via Over-sampling and Post-evaluation Junkai Zhou Liang Pang Huawei Shen Xueqi Cheng 68 13 0 18 May 2023
The Web Can Be Your Oyster for Improving Large Language Models Junyi Li Tianyi Tang Wayne Xin Zhao Jingyuan Wang Jian-Yun Nie Ji-Rong Wen RALM KELM 97 5 0 18 May 2023
How does the task complexity of masked pretraining objectives affect downstream performance? Atsuki Yamaguchi Hiroaki Ozaki Terufumi Morishita Gaku Morio Yasuhiro Sogawa 89 2 0 18 May 2023
Multilingual Event Extraction from Historical Newspaper Adverts Nadav Borenstein N. Perez Isabelle Augenstein 79 4 0 18 May 2023
TEPrompt: Task Enlightenment Prompt Learning for Implicit Discourse Relation Recognition Wei Xiang Chao Liang Bang Wang 45 7 0 18 May 2023
Large Language Models can be Guided to Evade AI-Generated Text Detection Ning Lu Shengcai Liu Ruidan He Qi Wang Yew-Soon Ong Jiaheng Zhang SILM 139 54 0 18 May 2023
Ahead-of-Time P-Tuning Daniil Gavrilov Nikita Balagansky 56 1 0 18 May 2023
DiffUTE: Universal Text Editing Diffusion Model Haoxing Chen Zhuoer Xu Zhangxuan Gu Jun Lan Xing Zheng Yaohui Li Changhua Meng Huijia Zhu Weiqiang Wang DiffM 102 35 0 18 May 2023
Diffusion Language Models Generation Can Be Halted Early Sofia Maria Lo Cicero Vaina Nikita Balagansky Daniil Gavrilov DiffM 84 0 0 18 May 2023
Ditto: A Simple and Efficient Approach to Improve Sentence Embeddings Qian Chen Wen Wang Qinglin Zhang Siqi Zheng Chong Deng Hai Yu Jiaqing Liu Yukun Ma Chong Zhang 60 3 0 18 May 2023
Transforming Human-Centered AI Collaboration: Redefining Embodied Agents Capabilities through Interactive Grounded Language Instructions Shrestha Mohanty Negar Arabzadeh Julia Kiseleva Artem Zholus Milagro Teruel Ahmed Hassan Awadallah Yuxuan Sun Kavya Srinet Arthur Szlam LM&Ro 127 13 0 18 May 2023
Human Behavioral Benchmarking: Numeric Magnitude Comparison Effects in Large Language Models Raj Sanjay Shah Vijay Marupudi Reba Koenen Khushi Bhardwaj Sashank Varma 78 6 0 18 May 2023
Vision-Language Pre-training with Object Contrastive Learning for 3D Scene Understanding Zhang Tao Su He D. Tao Bin Chen Zhi Wang Shutao Xia VLM 82 27 0 18 May 2023