Title
LLM-Rubric: A Multidimensional, Calibrated Approach to Automated Evaluation of Natural Language Texts Helia Hashemi J. Eisner Corby Rosset Benjamin Van Durme Chris Kedzie 145 6 0 03 Jan 2025
A 2-step Framework for Automated Literary Translation Evaluation: Its Promises and Pitfalls Sheikh Shafayat Dongkeun Yoon Woori Jang Jiwoo Choi Alice Oh Seohyon Jung 208 1 0 03 Jan 2025
A review of faithfulness metrics for hallucination assessment in Large Language Models Ben Malin Tatiana Kalganova Nikoloas Boulgouris HILM 136 2 0 03 Jan 2025
Fine-grained and Explainable Factuality Evaluation for Multimodal Summarization Liqiang Jing Jingxuan Zuo Yue Zhang 123 9 0 31 Dec 2024
Evaluate Summarization in Fine-Granularity: Auto Evaluation with LLM Dong Yuan Eti Rastogi Fen Zhao Sagar Goyal Gautam Naik Sree Prasanna Rajagopal 63 0 0 31 Dec 2024
ComparisonQA: Evaluating Factuality Robustness of LLMs Through Knowledge Frequency Control and Uncertainty Qing Zong Zhaoxiang Wang Tianshi Zheng Xiyu Ren Yangqiu Song 155 3 0 28 Dec 2024
A Survey of Calibration Process for Black-Box LLMs Liangru Xie Hui Liu Jingying Zeng Xianfeng Tang Yan Han Chen Luo Jing Huang Zhen Li Suhang Wang Qi He 142 4 0 17 Dec 2024
Attention with Dependency Parsing Augmentation for Fine-Grained Attribution Qiang Ding Lvzhou Luo Yixuan Cao Ping Luo 150 0 0 16 Dec 2024
Coverage-based Fairness in Multi-document Summarization Haoyuan Li Yusen Zhang Rui Zhang Snigdha Chaturvedi 183 0 0 11 Dec 2024
HalluCana: Fixing LLM Hallucination with A Canary Lookahead Tianyi Li Erenay Dayanik Shubhi Tyagi Andrea Pierleoni HILM 122 0 0 10 Dec 2024
QAPyramid: Fine-grained Evaluation of Content Selection for Text Summarization Shiyue Zhang David Wan Arie Cattan Ayal Klein Ido Dagan Joey Tianyi Zhou 128 0 0 10 Dec 2024
LLM-as-an-Interviewer: Beyond Static Testing Through Dynamic LLM Evaluation Eunsu Kim Juyoung Suk Seungone Kim Niklas Muennighoff Dongkwan Kim Alice Oh ELM 196 1 0 10 Dec 2024
Enhancing Trust in Large Language Models with Uncertainty-Aware Fine-Tuning R. Krishnan Piyush Khanna Omesh Tickoo HILM 116 1 0 03 Dec 2024
FactCheXcker: Mitigating Measurement Hallucinations in Chest X-ray Report Generation Models Alice Heiman Xiaoman Zhang E. Chen Sung Eun Kim Pranav Rajpurkar HILM MedIm 159 0 0 27 Nov 2024
From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judge Dawei Li Bohan Jiang Liangjie Huang Alimohammad Beigi Chengshuai Zhao ... Canyu Chen Tianhao Wu Kai Shu Lu Cheng Huan Liu ELM AILaw 377 112 0 25 Nov 2024
Investigating Factuality in Long-Form Text Generation: The Roles of Self-Known and Self-Unknown Lifu Tu Rui Meng Shafiq Joty Yingbo Zhou Semih Yavuz HILM 130 1 0 24 Nov 2024
Probing LLM Hallucination from Within: Perturbation-Driven Approach via Internal Knowledge Seongmin Lee Hsiang Hsu Chun-Fu Chen Duen Horng Chau LRM 101 2 0 14 Nov 2024
Beyond the Safety Bundle: Auditing the Helpful and Harmless Dataset Khaoula Chehbouni Jonathan Colaço-Carr Yash More Jackie CK Cheung G. Farnadi 175 1 0 12 Nov 2024
FactLens: Benchmarking Fine-Grained Fact Verification Kushan Mitra Dan Zhang Sajjadur Rahman Estevam R. Hruschka HILM 143 1 0 08 Nov 2024
Measuring short-form factuality in large language models Jason W. Wei Nguyen Karina Hyung Won Chung Yunxin Joy Jiao Spencer Papay Amelia Glaese John Schulman W. Fedus ELM KELM HILM 76 78 0 07 Nov 2024
Culinary Class Wars: Evaluating LLMs using ASH in Cuisine Transfer Task Hoonick Lee Mogan Gim Donghyeon Park Donghee Choi Jaewoo Kang 54 0 0 04 Nov 2024
Rate, Explain and Cite (REC): Enhanced Explanation and Attribution in Automatic Evaluation by Large Language Models Aliyah R. Hsu James Zhu Zhichao Wang Bin Bi Shubham Mehrotra ... Sougata Chaudhuri Regunathan Radhakrishnan S. Asur Claire Na Cheng Bin Yu ALM LRM 184 0 0 03 Nov 2024
Human-inspired Perspectives: A Survey on AI Long-term Memory Zihong He Weizhe Lin Hao Zheng Fan Zhang Matt Jones Laurence Aitchison X. Xu Miao Liu Per Ola Kristensson Junxiao Shen 246 3 0 01 Nov 2024
The Automated Verification of Textual Claims (AVeriTeC) Shared Task Michael Schlichtkrull Yulong Chen Chenxi Whitehouse Zhenyun Deng Mubashara Akhtar ... Christos Christodoulopoulos O. Cocarascu Arpit Mittal James Thorne Andreas Vlachos 100 9 0 31 Oct 2024
Improving Uncertainty Quantification in Large Language Models via Semantic Embeddings Yashvir S. Grewal Edwin V. Bonilla Thang D. Bui UQCV 84 9 0 30 Oct 2024
Unified Triplet-Level Hallucination Evaluation for Large Vision-Language Models J. Wu Tsz Ting Chung Kai Chen Dit-Yan Yeung LRM VLM 131 4 0 30 Oct 2024
FactBench: A Dynamic Benchmark for In-the-Wild Language Model Factuality Evaluation Farima Fatahi Bayat Lechen Zhang Sheza Munir Lu Wang HILM 116 4 0 29 Oct 2024
LongReward: Improving Long-context Large Language Models with AI Feedback Jing Zhang Zhongni Hou Xin Lv S. Cao Zhenyu Hou Yilin Niu Lei Hou Yuxiao Dong Ling Feng Juanzi Li OffRL LRM 86 10 0 28 Oct 2024
Graph-based Uncertainty Metrics for Long-form Language Model Outputs Mingjian Jiang Yangjun Ruan Prasanna Sattigeri Salim Roukos Tatsunori Hashimoto 67 5 0 28 Oct 2024
Fictitious Synthetic Data Can Improve LLM Factuality via Prerequisite Learning Yujian Liu Shiyu Chang Tommi Jaakkola Yang Zhang 74 1 0 25 Oct 2024
ChunkRAG: Novel LLM-Chunk Filtering Method for RAG Systems Ishneet Sukhvinder Singh Ritvik Aggarwal Ibrahim Allahverdiyev Muhammad Taha Aslihan Akalin Kevin Zhu Sean O'Brien 126 10 0 25 Oct 2024
Improving Model Factuality with Fine-grained Critique-based Evaluator Yiqing Xie Wenxuan Zhou Pradyot Prakash Di Jin Yuning Mao ... Sinong Wang Han Fang Carolyn Rose Daniel Fried Hejia Zhang HILM 167 8 0 24 Oct 2024
Multilingual Hallucination Gaps in Large Language Models Cléa Chataigner Afaf Taik G. Farnadi HILM LRM 68 5 0 23 Oct 2024
Leveraging the Domain Adaptation of Retrieval Augmented Generation Models for Question Answering and Reducing Hallucination Salman Rakin Md. A. R. Shibly Zahin M. Hossain Zeeshan Khan Md. Mostofa Akbar 70 3 0 23 Oct 2024
Enhancing Answer Attribution for Faithful Text Generation with Large Language Models Juraj Vladika Luca Mülln Florian Matthes 65 0 0 22 Oct 2024
Trustworthy Alignment of Retrieval-Augmented Large Language Models via Reinforcement Learning Zongmeng Zhang Yufeng Shi Jinhua Zhu Wengang Zhou Xiang Qi Peng Zhang Haoyang Li RALM HILM 39 0 0 22 Oct 2024
Self-Explained Keywords Empower Large Language Models for Code Generation Lishui Fan Mouxiang Chen Zhongxin Liu 78 1 0 21 Oct 2024
RAC: Efficient LLM Factuality Correction with Retrieval Augmentation Changmao Li Jeffrey Flanigan KELM LRM 57 3 0 21 Oct 2024
BRIEF: Bridging Retrieval and Inference for Multi-hop Reasoning via Compression Yuankai Li Jia-Chen Gu Di Wu Kai-Wei Chang Nanyun Peng RALM MQ 72 0 0 20 Oct 2024
Cross-Document Event-Keyed Summarization William Walden Pavlo Kuchmiichuk Alexander Martin Chihsheng Jin Angela Cao Claire Sun Curisia Allen Aaron Steven White RALM 53 0 0 18 Oct 2024
Tell me what I need to know: Exploring LLM-based (Personalized) Abstractive Multi-Source Meeting Summarization Frederic Kirstein Terry Ruas Robert Kratel Bela Gipp 44 3 0 18 Oct 2024
LoGU: Long-form Generation with Uncertainty Expressions Ruihan Yang Caiqi Zhang Zhisong Zhang Xinting Huang Sen Yang Nigel Collier Dong Yu Deqing Yang HILM 270 9 0 18 Oct 2024
Cross-Lingual Auto Evaluation for Assessing Multilingual LLMs Sumanth Doddapaneni Mohammed Safi Ur Rahman Khan Dilip Venkatesh Raj Dabre Anoop Kunchukuttan Mitesh M. Khapra ELM 72 1 0 17 Oct 2024
Probing-RAG: Self-Probing to Guide Language Models in Selective Document Retrieval Ingeol Baek Hwan Chang Byeongjeong Kim Jimin Lee Hwanhee Lee RALM 160 5 0 17 Oct 2024
From Single to Multi: How LLMs Hallucinate in Multi-Document Summarization Catarina G. Belem Pouya Pezeskhpour Hayate Iso Seiji Maekawa Nikita Bhutani Estevam R. Hruschka HILM 138 3 0 17 Oct 2024
FIRE: Fact-checking with Iterative Retrieval and Verification Zhuohan Xie Rui Xing Yuxia Wang Jiahui Geng Hasan Iqbal Dhruv Sahnan Iryna Gurevych Preslav Nakov HILM 173 7 0 17 Oct 2024
Decomposition Dilemmas: Does Claim Decomposition Boost or Burden Fact-Checking Performance? Qisheng Hu Quanyu Long Wenya Wang 405 9 0 17 Oct 2024
A Claim Decomposition Benchmark for Long-form Answer Verification Zhihao Zhang Yixing Fan Ruqing Zhang Jiafeng Guo HILM 54 0 0 16 Oct 2024
An Automatic and Cost-Efficient Peer-Review Framework for Language Generation Evaluation Junjie Chen Weihang Su Zhumin Chu Haitao Li Qinyao Ai Yiqun Liu Min Zhang Shaoping Ma 63 3 0 16 Oct 2024
Varying Shades of Wrong: Aligning LLMs with Wrong Answers Only Jihan Yao Wenxuan Ding Shangbin Feng Lucy Lu Wang Yulia Tsvetkov 73 2 0 14 Oct 2024