RoBERTa: A Robustly Optimized BERT Pretraining Approach

26 July 2019

Luke Zettlemoyer

Papers citing "RoBERTa: A Robustly Optimized BERT Pretraining Approach"

50 / 4,740 papers shown

Title
BabyLM Challenge: Exploring the Effect of Variation Sets on Language Model Training Efficiency Akari Haga Akiyo Fukatsu Miyu Oba Arianna Bisazza Yohei Oseki 35 1 0 14 Nov 2024
Prompt-enhanced Network for Hateful Meme Classification Junxi Liu Yanyan Feng Jiehai Chen Yun Xue Fenghuan Li VLM 63 0 0 12 Nov 2024
HeartBERT: A Self-Supervised ECG Embedding Model for Efficient and Effective Medical Signal Analysis Saedeh Tahery Fatemeh Hamid Akhlaghi Termeh Amirsoleimani OOD 88 1 0 08 Nov 2024
Beemo: Benchmark of Expert-edited Machine-generated Outputs Ekaterina Artemova Jason Samuel Lucas Saranya Venkatraman Jooyoung Lee Sergei Tilga Adaku Uchendu Vladislav Mikhailov DeLMO MoE 71 6 0 06 Nov 2024
Confidence Calibration of Classifiers with Many Classes Adrien LeCoz Stéphane Herbin Faouzi Adjed UQCV 44 1 0 05 Nov 2024
ATM: Improving Model Merging by Alternating Tuning and Merging Luca Zhou Daniele Solombrino Donato Crisostomi Maria Sofia Bucarelli Fabrizio Silvestri Emanuele Rodolà MoMe 52 5 0 05 Nov 2024
Navigating Extremes: Dynamic Sparsity in Large Output Spaces Nasib Ullah Erik Schultheis Mike Lasby Yani Andrew Ioannou Rohit Babbar 37 0 0 05 Nov 2024
Human-inspired Perspectives: A Survey on AI Long-term Memory Zihong He Weizhe Lin Hao Zheng Fan Zhang Matt Jones Laurence Aitchison X. Xu Miao Liu Per Ola Kristensson Junxiao Shen 79 2 0 01 Nov 2024
On the Impact of White-box Deployment Strategies for Edge AI on Latency and Model Performance Jaskirat Singh Bram Adams Ahmed E. Hassan VLM 45 0 0 01 Nov 2024
Dynamic Strategy Planning for Efficient Question Answering with Large Language Models Tanmay Parekh Pradyot Prakash Alexander Radovic Akshay Shekher Denis Savenkov LRM 142 1 0 30 Oct 2024
Smaller Large Language Models Can Do Moral Self-Correction Guangliang Liu Zhiyu Xue Rongrong Wang K. Johnson Kristen Marie Johnson LRM 34 0 0 30 Oct 2024
Online Detecting LLM-Generated Texts via Sequential Hypothesis Testing by Betting Can Chen Jun-Kun Wang DeLMO 42 0 0 29 Oct 2024
SubjECTive-QA: Measuring Subjectivity in Earnings Call Transcripts' QA Through Six-Dimensional Feature Analysis Huzaifa Pardawala Siddhant Sukhani Agam Shah Veer Kejriwal Abhishek Pillai Rohan Bhasin Andrew DiBiasio Tarun Mandapati Dhruv Adha Sudheer Chava 125 1 0 28 Oct 2024
Decoding Reading Goals from Eye Movements Omer Shubi Cfir Avraham Hadar Yevgeni Berzak AIMat 49 1 0 28 Oct 2024
Natural Language Processing for the Legal Domain: A Survey of Tasks, Datasets, Models, and Challenges Farid Ariai Gianluca Demartini ELM AILaw VLM 48 4 0 25 Oct 2024
Building Dialogue Understanding Models for Low-resource Language Indonesian from Scratch Donglin Di Weinan Zhang Yue Zhang Fanglin Wang 31 1 0 24 Oct 2024
MiniPLM: Knowledge Distillation for Pre-Training Language Models Yuxian Gu Hao Zhou Fandong Meng Jie Zhou Minlie Huang 73 5 0 22 Oct 2024
Conjuring Semantic Similarity Tian Yu Liu Stefano Soatto DiffM 32 0 0 21 Oct 2024
DefVerify: Do Hate Speech Models Reflect Their Dataset's Definition? Urja Khurana Eric T. Nalisnick Antske Fokkens 64 1 0 21 Oct 2024
LDAdam: Adaptive Optimization from Low-Dimensional Gradient Statistics Thomas Robert M. Safaryan Ionut-Vlad Modoranu Dan Alistarh ODL 36 2 0 21 Oct 2024
BIG5-CHAT: Shaping LLM Personalities Through Training on Human-Grounded Data Wenkai Li Jiarui Liu Andy Liu Xuhui Zhou Mona Diab Maarten Sap 62 7 0 21 Oct 2024
CompAct: Compressed Activations for Memory-Efficient LLM Training Yara Shamshoum Nitzan Hodos Yuval Sieradzki Assaf Schuster MQ VLM 57 1 0 20 Oct 2024
Contextual Augmented Multi-Model Programming (CAMP): A Hybrid Local-Cloud Copilot Framework Yuchen Wang Shangxin Guo C. Tan 42 0 0 20 Oct 2024
Coarse-to-Fine Highlighting: Reducing Knowledge Hallucination in Large Language Models Qitan Lv Jie Wang Hanzhu Chen Bin Li Yongdong Zhang Feng Wu HILM 33 3 0 19 Oct 2024
Medical-GAT: Cancer Document Classification Leveraging Graph-Based Residual Network for Scenarios with Limited Data Elias Hossain Tasfia Nuzhat Shamsul Masum Shahram Rahimi Sudip Mittal 47 0 0 19 Oct 2024
Towards Robust Knowledge Representations in Multilingual LLMs for Equivalence and Inheritance based Consistent Reasoning Gaurav Arora Srujana Merugu Shreya Jain Vaibhav Saxena LRM 37 0 0 18 Oct 2024
Are AI Detectors Good Enough? A Survey on Quality of Datasets With Machine-Generated Texts German Gritsai Anastasia Voznyuk Andrey Grabovoy Yury Chekhovich DeLMO 80 1 0 18 Oct 2024
CLaMP 2: Multimodal Music Information Retrieval Across 101 Languages Using Large Language Models Shangda Wu Yashan Wang Ruibin Yuan Zhancheng Guo Xu Tan ... Yuanliang Dong Jiafeng Liu Xiaobing Li Feng Yu Maosong Sun 48 4 0 17 Oct 2024
Preference Diffusion for Recommendation Shuo Liu An Zhang Guoqing Hu Hong Qian Tat-Seng Chua 60 1 0 17 Oct 2024
Linguistically Grounded Analysis of Language Models using Shapley Head Values Marcell Richard Fekete Johannes Bjerva 31 0 0 17 Oct 2024
LLM-Human Pipeline for Cultural Context Grounding of Conversations Rajkumar Pujari Dan Goldwasser 38 1 0 17 Oct 2024
Mastering the Craft of Data Synthesis for CodeLLMs Meng Chen Philip Arthur Qianyu Feng Cong Duy Vu Hoang Yu-Heng Hong ... Mark Johnson Kemal Kurniawan Don Dharmasiri Long Duong Yuan-Fang Li SyDa 62 1 0 16 Oct 2024
StyleDistance: Stronger Content-Independent Style Embeddings with Synthetic Parallel Examples Ajay Patel Jiacheng Zhu Justin Qiu Zachary Horvitz Marianna Apidianaki Kathleen McKeown Chris Callison-Burch 63 3 0 16 Oct 2024
Pixology: Probing the Linguistic and Visual Capabilities of Pixel-based Language Models Kushal Tatariya Vladimir Araujo Thomas Bauwens Miryam de Lhoneux VLM 38 0 0 15 Oct 2024
Open World Object Detection: A Survey Yiming Li Yi Wang Wenqian Wang Dan Lin Bingbing Li Kim-Hui Yap ObjD 47 0 0 15 Oct 2024
HR-Agent: A Task-Oriented Dialogue (TOD) LLM Agent Tailored for HR Applications Weijie Xu Jay Desai Fanyou Wu Josef Valvoda Srinivasan H. Sengamedu LLMAG 46 1 0 15 Oct 2024
Token Pruning using a Lightweight Background Aware Vision Transformer Sudhakar Sah Ravish Kumar Honnesh Rohmetra Ehsan Saboori ViT 31 1 0 12 Oct 2024
DARE the Extreme: Revisiting Delta-Parameter Pruning For Fine-Tuned Models Wenlong Deng Yize Zhao V. Vakilian Minghui Chen Xiaoxiao Li Christos Thrampoulidis 50 4 0 12 Oct 2024
Beyond Exact Match: Semantically Reassessing Event Extraction by Large Language Models Yi-Fan Lu Xian-Ling Mao Tian Lan Heyan Huang Heyan Huang Xiaoyan Gao 55 0 0 12 Oct 2024
StablePrompt: Automatic Prompt Tuning using Reinforcement Learning for Large Language Models Minchan Kwon Gaeun Kim Jongsuk Kim Haeil Lee Junmo Kim OffRL LRM LLMAG 26 2 0 10 Oct 2024
DICE: Discrete Inversion Enabling Controllable Editing for Multinomial Diffusion and Masked Generative Models Xiaoxiao He Ligong Han Quan Dao Song Wen Minhao Bai ... Hongdong Li Junzhou Huang Faez Ahmed Akash Srivastava Dimitris Metaxas DiffM SyDa 42 5 0 10 Oct 2024
Do Current Language Models Support Code Intelligence for R Programming Language? ZiXiao Zhao Fatemeh H. Fard ELM 50 0 0 10 Oct 2024
Data Selection via Optimal Control for Language Models Yuxian Gu Li Dong Hongning Wang Y. Hao Qingxiu Dong Furu Wei Minlie Huang AI4CE 58 5 0 09 Oct 2024
Parameter Efficient Fine-tuning via Explained Variance Adaptation Fabian Paischer Lukas Hauzenberger Thomas Schmied Benedikt Alkin Marc Peter Deisenroth Sepp Hochreiter 42 4 0 09 Oct 2024
Functional-level Uncertainty Quantification for Calibrated Fine-tuning on LLMs Ruijia Niu D. Wu Rose Yu Yi Ma 38 1 0 09 Oct 2024
GLOV: Guided Large Language Models as Implicit Optimizers for Vision Language Models Muhammad Jehanzeb Mirza Mengjie Zhao Zhuoyuan Mao Sivan Doveh Wei Lin ... Yuki Mitsufuji Horst Possegger Rogerio Feris Leonid Karlinsky James Glass VLM 84 1 0 08 Oct 2024
CasiMedicos-Arg: A Medical Question Answering Dataset Annotated with Explanatory Argumentative Structures Ekaterina Sviridova Anar Yeginbergen A. Estarrona Elena Cabrio S. Villata Rodrigo Agerri 63 2 0 07 Oct 2024
SFTMix: Elevating Language Model Instruction Tuning with Mixup Recipe Yuxin Xiao Shujian Zhang Wenxuan Zhou Marzyeh Ghassemi Sanqiang Zhao 192 0 0 07 Oct 2024
Collapsed Language Models Promote Fairness Jingxuan Xu Wuyang Chen Linyi Li Yao Zhao Yunchao Wei 48 0 0 06 Oct 2024
Large Language Model Inference Acceleration: A Comprehensive Hardware Perspective Jinhao Li Jiaming Xu Shan Huang Yonghua Chen Wen Li ... Jiayi Pan Li Ding Hao Zhou Yu Wang Guohao Dai 62 17 0 06 Oct 2024