Title
LLM-as-a-qualitative-judge: automating error analysis in natural language generation Nadezhda Chirkova Tunde Oluwaseyi Ajayi Seth Aycock Zain Muhammad Mujahid Vladana Perlić Ekaterina Borisova Markarit Vartampetian ELM 25 0 0 10 Jun 2025
Ignoring Directionality Leads to Compromised Graph Neural Network Explanations Changsheng Sun Xinke Li Jin Song Dong AAML 118 0 0 05 Jun 2025
RewardAnything: Generalizable Principle-Following Reward Models Zhuohao Yu Jiali Zeng Weizheng Gu Yidong Wang Jindong Wang Fandong Meng Jie Zhou Yue Zhang Shikun Zhang Wei Ye LRM 97 1 0 04 Jun 2025
Multi-Domain Explainability of Preferences Nitay Calderon Liat Ein-Dor Roi Reichart LRM 56 0 0 26 May 2025
Assistant-Guided Mitigation of Teacher Preference Bias in LLM-as-a-Judge Zhuo Liu Moxin Li Xun Deng Qifan Wang Fuli Feng ELM 64 0 0 25 May 2025
Flex-Judge: Think Once, Judge Anywhere Jongwoo Ko S. Kim Sungwoo Cho Se-Young Yun ELM LRM 210 0 0 24 May 2025
Keep Security! Benchmarking Security Policy Preservation in Large Language Model Contexts Against Indirect Attacks in Question Answering Hwan Chang Yumin Kim Yonghyun Jun Hwanhee Lee AAML ELM 67 0 0 21 May 2025
YESciEval: Robust LLM-as-a-Judge for Scientific Question Answering Jennifer D'Souza Hamed Babaei Giglou Quentin Münch ELM 109 0 0 20 May 2025
DECASTE: Unveiling Caste Stereotypes in Large Language Models through Multi-Dimensional Bias Analysis Prashanth Vijayaraghavan Soroush Vosoughi Lamogha Chizor Raya Horesh Rogerio Abreu de Paula Ehsan Degan Vandana Mukherjee 46 1 0 20 May 2025
R3: Robust Rubric-Agnostic Reward Models David Anugraha Zilu Tang Lester James V. Miranda Hanyang Zhao Mohammad Rifqi Farhansyah Garry Kuwanto Derry Wijaya Genta Indra Winata 209 1 0 19 May 2025
J4R: Learning to Judge with Equivalent Initial State Group Relative Policy Optimization Austin Xu Yilun Zhou Xuan-Phi Nguyen Caiming Xiong Shafiq Joty ELM LRM 142 0 0 19 May 2025
TRAIL: Trace Reasoning and Agentic Issue Localization Darshan Deshpande Varun Gangal Hersh Mehta Jitin Krishnan Anand Kannappan Rebecca Qian 124 0 0 13 May 2025
LLAMAPIE: Proactive In-Ear Conversation Assistants Tuochao Chen Nicholas Batchelder Alisa Liu Noah A. Smith Shyamnath Gollakota 403 0 0 07 May 2025
Process Reward Models That Think Muhammad Khalifa Rishabh Agarwal Lajanugen Logeswaran Jaekyeom Kim Hao Peng Moontae Lee Honglak Lee Lu Wang OffRL ALM LRM 143 9 0 23 Apr 2025
Evaluating Judges as Evaluators: The JETTS Benchmark of LLM-as-Judges as Test-Time Scaling Evaluators Yilun Zhou Austin Xu Peifeng Wang Caiming Xiong Shafiq Joty ELM ALM LRM 167 5 0 21 Apr 2025
Pairwise or Pointwise? Evaluating Feedback Protocols for Bias in LLM-Based Evaluation Tuhina Tripathi Manya Wadhwa Greg Durrett S. Niekum 78 0 0 20 Apr 2025
Large Language Models as Span Annotators Zdeněk Kasner Vilém Zouhar Patrícia Schmidtová Ivan Kartáč Kristýna Onderková Ondřej Plátek Dimitra Gkatzia Saad Mahamood Ondrej Dusek Simone Balloccu ALM 124 0 0 11 Apr 2025
DocAgent: A Multi-Agent System for Automated Code Documentation Generation Dayu Yang Antoine Simoulin Xin Qian Xiaoyi Liu Yuwei Cao Zhaopu Teng Grey Yang LLMAG 143 0 0 11 Apr 2025
Toward Holistic Evaluation of Recommender Systems Powered by Generative Models Yashar Deldjoo Nikhil Mehta M. Sathiamoorthy Shuai Zhang Pablo Castells Julian McAuley EGVM ELM 131 2 0 09 Apr 2025
Zero-shot Benchmarking: A Framework for Flexible and Scalable Automatic Evaluation of Language Models José P. Pombal Nuno M. Guerreiro Ricardo Rei André F. T. Martins ALM 136 2 0 01 Apr 2025
Do We Truly Need So Many Samples? Multi-LLM Repeated Sampling Efficiently Scales Test-Time Compute Jianhao Chen Zishuo Xun Bocheng Zhou Han Qi Qiaosheng Zhang ... Wei Hu Yuzhong Qu W. Ouyang Wanli Ouyang Shuyue Hu 187 2 0 01 Apr 2025
A Survey on Personalized Alignment -- The Missing Piece for Large Language Models in Real-World Applications Jian Guan Jian Wu Jia-Nan Li Chuanqi Cheng Wei Wu LM&MA 168 3 0 21 Mar 2025
REPA: Russian Error Types Annotation for Evaluating Text Generation and Judgment Capabilities Alexander Pugachev Alena Fenogenova Vladislav Mikhailov Ekaterina Artemova 104 0 0 17 Mar 2025
MentalChat16K: A Benchmark Dataset for Conversational Mental Health Assistance Jia Xu Tianyi Wei Bojian Hou Patryk Orzechowski Shu Yang Ruochen Jin Rachael Paulbeck Joost B. Wagenaar George Demiris Li Shen AI4MH 81 1 0 13 Mar 2025
DeFine: A Decomposed and Fine-Grained Annotated Dataset for Long-form Article Generation Ming Wang Fang Wang Minghao Hu Li He Haiyang Wang ... Li Li Zhunchen Luo Wei Luo Xiaoying Bai Guotong Geng 128 0 0 10 Mar 2025
Is Your Video Language Model a Reliable Judge? M. Liu Wensheng Zhang 104 5 0 07 Mar 2025
Teaching Metric Distance to Autoregressive Multimodal Foundational Models Jiwan Chung Saejin Kim Yongrae Jo Jinho Park Dongjun Min Youngjae Yu 250 0 0 04 Mar 2025
SePer: Measure Retrieval Utility Through The Lens Of Semantic Perplexity Reduction Lu Dai Yijie Xu Jinhui Ye Hao Liu Hui Xiong 3DV RALM 203 3 0 03 Mar 2025
Learning to Align Multi-Faceted Evaluation: A Unified and Robust Framework Kaishuai Xu Tiezheng YU Wenjun Hou Yi Cheng Liangyou Li Xin Jiang Lifeng Shang Qiang Liu Wenjie Li ELM 147 0 0 26 Feb 2025
SYNTHIA: Novel Concept Design with Affordance Composition Xiaomeng Jin Xiaomeng Jin Jeonghwan Kim Qingbin Liu Zhenhailong Wang Khanh Duy Nguyen Ansel Blume Nanyun Peng Kai-Wei Chang Heng Ji DiffM 494 2 0 25 Feb 2025
Independent Mobility GPT (IDM-GPT): A Self-Supervised Multi-Agent Large Language Model Framework for Customized Traffic Mobility Analysis Using Machine Learning Models Fengze Yang Xiaoyue Cathy Liu Lingjiu Lu Bingzhang Wang Chenxi 90 1 0 25 Feb 2025
Evaluating Step-by-step Reasoning Traces: A Survey Jinu Lee Julia Hockenmaier LRM ELM 153 2 0 17 Feb 2025
Preference Leakage: A Contamination Problem in LLM-as-a-judge Dawei Li Renliang Sun Yue Huang Ming Zhong Bohan Jiang Jiawei Han Wei Wei Wei Wang Huan Liu 172 30 0 03 Feb 2025
CoddLLM: Empowering Large Language Models for Data Analytics Jiani Zhang Hengrui Zhang Rishav Chakravarti Yiqun Hu Patrick Ng Asterios Katsifodimos Huzefa Rangwala George Karypis Alon Halevy SyDa ELM 452 0 0 01 Feb 2025
Learning to Explore and Select for Coverage-Conditioned Retrieval-Augmented Generation Takyoung Kim Kyungjae Lee Y. Jang Ji Yong Cho Gangwoo Kim Minseok Cho Moontae Lee 285 1 0 28 Jan 2025
Beyond correlation: The Impact of Human Uncertainty in Measuring the Effectiveness of Automatic Evaluation and LLM-as-a-Judge Aparna Elangovan Jongwoo Ko Lei Xu Mahsa Elyasi Ling Liu S. Bodapati Dan Roth 125 6 0 28 Jan 2025
A 2-step Framework for Automated Literary Translation Evaluation: Its Promises and Pitfalls Sheikh Shafayat Dongkeun Yoon Woori Jang Jiwoo Choi Alice Oh Seohyon Jung 205 1 0 03 Jan 2025
Disentangling Preference Representation and Text Generation for Efficient Individual Preference Alignment Jianfei Zhang Jun Bai Yangqiu Song Yanmeng Wang Rumei Li Chenghua Lin Wenge Rong 152 0 0 31 Dec 2024
LLM-as-an-Interviewer: Beyond Static Testing Through Dynamic LLM Evaluation Eunsu Kim Juyoung Suk Seungone Kim Niklas Muennighoff Dongkwan Kim Alice Oh ELM 186 1 0 10 Dec 2024
VISCO: Benchmarking Fine-Grained Critique and Correction Towards Self-Improvement in Visual Reasoning Xueqing Wu Yuheng Ding Bingxuan Li Pan Lu Da Yin Kai-Wei Chang Nanyun Peng LRM 152 4 0 03 Dec 2024
From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judge Dawei Li Bohan Jiang Liangjie Huang Alimohammad Beigi Chengshuai Zhao ... Canyu Chen Tianhao Wu Kai Shu Lu Cheng Huan Liu ELM AILaw 350 112 0 25 Nov 2024
Self-Generated Critiques Boost Reward Modeling for Language Models Yue Yu Zhengxing Chen Aston Zhang L Tan Chenguang Zhu ... Suchin Gururangan Chao-Yue Zhang Melanie Kambadur Dhruv Mahajan Rui Hou LRM ALM 177 27 0 25 Nov 2024
From Jack of All Trades to Master of One: Specializing LLM-based Autoraters to a Test Set M. Finkelstein Dan Deutsch Parker Riley Juraj Juraska Geza Kovacs Markus Freitag 108 0 0 23 Nov 2024
DELIFT: Data Efficient Language model Instruction Fine Tuning Ishika Agarwal Krishnateja Killamsetty Lucian Popa Marina Danilevksy ALM VLM 126 4 0 07 Nov 2024
Improving Model Factuality with Fine-grained Critique-based Evaluator Yiqing Xie Wenxuan Zhou Pradyot Prakash Di Jin Yuning Mao ... Sinong Wang Han Fang Carolyn Rose Daniel Fried Hejia Zhang HILM 156 8 0 24 Oct 2024
MCQG-SRefine: Multiple Choice Question Generation and Evaluation with Iterative Self-Critique, Correction, and Comparison Feedback Zonghai Yao Aditya Parashar Huixue Zhou Won Seok Jang Feiyun Ouyang Zhichao Yang Hong-ye Yu ELM 137 2 0 17 Oct 2024
JudgeBench: A Benchmark for Evaluating LLM-based Judges Sijun Tan Siyuan Zhuang Kyle Montgomery William Y. Tang Alejandro Cuadron Chenguang Wang Raluca A. Popa Ion Stoica ELM ALM 146 52 0 16 Oct 2024
4-LEGS: 4D Language Embedded Gaussian Splatting Gal Fiebelman Tamir Cohen Ayellet Morgenstern Peter Hedman Hadar Averbuch-Elor 3DGS 143 3 0 14 Oct 2024
EasyJudge: an Easy-to-use Tool for Comprehensive Response Evaluation of LLMs Yijie Li Yuan Sun ELM 60 1 0 13 Oct 2024
Beyond Exact Match: Semantically Reassessing Event Extraction by Large Language Models Yi-Fan Lu Xian-Ling Mao Tian Lan Heyan Huang Heyan Huang Xiaoyan Gao 85 0 0 12 Oct 2024