Contrastive Decoding: Open-ended Text Generation as Optimization

27 October 2022

Xiang Lisa Li

Ari Holtzman

Daniel Fried

Percy Liang

Jason Eisner

Tatsunori Hashimoto

Luke Zettlemoyer

M. Lewis

ArXiv PDF HTML

Papers citing "Contrastive Decoding: Open-ended Text Generation as Optimization"

50 / 264 papers shown

Title
Towards Better Open-Ended Text Generation: A Multicriteria Evaluation Framework Esteban Garces Arias Hannah Blocher Julian Rodemann Meimingwei Li Christian Heumann Matthias Aßenmacher 28 1 0 24 Oct 2024
CogSteer: Cognition-Inspired Selective Layer Intervention for Efficient Semantic Steering in Large Language Models Xintong Wang Jingheng Pan Longqin Jiang Liang Ding Xingshan Li Chris Biemann LLMSV 29 0 0 23 Oct 2024
Mitigating Hallucinations of Large Language Models in Medical Information Extraction via Contrastive Decoding Derong Xu Ziheng Zhang Zhihong Zhu Zhenxi Lin Qiang Liu X. Wu Tong Xu Xiangyu Zhao Yefeng Zheng Enhong Chen 20 2 0 21 Oct 2024
What's New in My Data? Novelty Exploration via Contrastive Generation Masaru Isonuma Ivan Titov 28 0 0 18 Oct 2024
LargePiG: Your Large Language Model is Secretly a Pointer Generator ZhongXiang Sun Zihua Si Xiaoxue Zang Kai Zheng Yang Song Xiao Zhang Jun Xu HILM RALM 42 0 0 15 Oct 2024
MLLM can see? Dynamic Correction Decoding for Hallucination Mitigation Chenxi Wang Xiang Chen N. Zhang Bozhong Tian Haoming Xu Shumin Deng H. Chen MLLM LRM 37 4 0 15 Oct 2024
Divide, Reweight, and Conquer: A Logit Arithmetic Approach for In-Context Learning Chengsong Huang Langlin Huang Jiaxin Huang MoMe 38 1 0 14 Oct 2024
COrAL: Order-Agnostic Language Modeling for Efficient Iterative Refinement Yuxi Xie Anirudh Goyal Xiaobao Wu Xunjian Yin Xiao Xu Min-Yen Kan Liangming Pan William Yang Wang LRM 86 1 0 12 Oct 2024
Mitigating Modality Prior-Induced Hallucinations in Multimodal Large Language Models via Deciphering Attention Causality Guanyu Zhou Yibo Yan Xin Zou Kun Wang Aiwei Liu Xuming Hu 48 7 0 07 Oct 2024
DAMRO: Dive into the Attention Mechanism of LVLM to Reduce Object Hallucination Xuan Gong Tianshi Ming Xinpeng Wang Zhihua Wei MLLM 42 10 0 06 Oct 2024
CS4: Measuring the Creativity of Large Language Models Automatically by Controlling the Number of Story-Writing Constraints Anirudh Atmakuru Jatin Nainani Rohith Siddhartha Reddy Bheemreddy Anirudh Lakkaraju Zonghai Yao Hamed Zamani Haw-Shiuan Chang 116 2 0 05 Oct 2024
A Simple yet Effective Training-free Prompt-free Approach to Chinese Spelling Correction Based on Large Language Models Houquan Zhou Zhenghua Li Bo Zhang Chen Li Shaopeng Lai Ji Zhang Fei Huang M. Zhang LRM 33 2 0 05 Oct 2024
Look Twice Before You Answer: Memory-Space Visual Retracing for Hallucination Mitigation in Multimodal Large Language Models Xin Zou Yizhou Wang Yibo Yan Yuanhuiyi Lyu Kening Zheng ... Junkai Chen Peijie Jiang Jiaheng Liu Chang Tang Xuming Hu 86 7 0 04 Oct 2024
Interpretable Contrastive Monte Carlo Tree Search Reasoning Zitian Gao Boye Niu Xuzheng He Haotian Xu Hongzhang Liu Aiwei Liu Xuming Hu Lijie Wen LRM 57 26 0 02 Oct 2024
Truth or Deceit? A Bayesian Decoding Game Enhances Consistency and Reliability Weitong Zhang Chengqi Zang Bernhard Kainz 31 0 0 01 Oct 2024
Style-Specific Neurons for Steering LLMs in Text Style Transfer Wen Lai Viktor Hangya Alexander Fraser 36 7 0 01 Oct 2024
FedPT: Federated Proxy-Tuning of Large Language Models on Resource-Constrained Edge Devices Zhidong Gao Yu Zhang Zhenxiao Zhang Yanmin Gong Yuanxiong Guo 23 0 0 01 Oct 2024
HELPD: Mitigating Hallucination of LVLMs by Hierarchical Feedback Learning with Vision-enhanced Penalty Decoding Fan Yuan Chi Qin Xiaogang Xu Piji Li VLM MLLM 33 4 0 30 Sep 2024
A Survey on the Honesty of Large Language Models Siheng Li Cheng Yang Taiqiang Wu Chufan Shi Yuji Zhang ... Jie Zhou Yujiu Yang Ngai Wong Xixin Wu Wai Lam HILM 32 4 0 27 Sep 2024
CoCA: Regaining Safety-awareness of Multimodal Large Language Models with Constitutional Calibration Jiahui Gao Renjie Pi Tianyang Han Han Wu Lanqing Hong Lingpeng Kong Xin Jiang Zhenguo Li 41 5 0 17 Sep 2024
Householder Pseudo-Rotation: A Novel Approach to Activation Editing in LLMs with Direction-Magnitude Perspective Van-Cuong Pham Thien Huu Nguyen LLMSV 40 3 0 16 Sep 2024
AdaCAD: Adaptively Decoding to Balance Conflicts between Contextual and Parametric Knowledge Han Wang Archiki Prasad Elias Stengel-Eskin Joey Tianyi Zhou 82 5 0 11 Sep 2024
Mitigating Hallucination in Visual-Language Models via Re-Balancing Contrastive Decoding Xiaoyu Liang Jiayuan Yu Lianrui Mu Jiedong Zhuang Jiaqi Hu Yuchen Yang Jiangnan Ye Lu Lu Jian Chen Haoji Hu VLM 45 2 0 10 Sep 2024
What is the Role of Small Models in the LLM Era: A Survey Lihu Chen Gaël Varoquaux ALM 63 23 0 10 Sep 2024
$$\mathbb{USCD}$: Improving Code Generation of LLMs by Uncertainty-Aware Selective Contrastive Decoding$ $\mathbb{USCD}$ : Improving Code Generation of LLMs by Uncertainty-Aware Selective Contrastive Decoding Shuai Wang Liang Ding Li Shen Yong Luo Zheng He Wei Yu Dacheng Tao 33 0 0 09 Sep 2024
Bidirectional Decoding: Improving Action Chunking via Guided Test-Time Sampling Yuejiang Liu Jubayer Ibn Hamid Annie Xie Yoonho Lee Maximilian Du Chelsea Finn OffRL 65 5 0 30 Aug 2024
ConVis: Contrastive Decoding with Hallucination Visualization for Mitigating Hallucinations in Multimodal Large Language Models Yeji Park Deokyeong Lee Junsuk Choe Buru Chang MLLM VLM 64 4 0 25 Aug 2024
Towards Analyzing and Mitigating Sycophancy in Large Vision-Language Models Yunpu Zhao Rui Zhang Junbin Xiao Changxin Ke Ruibo Hou Yifan Hao Qi Guo Yunji Chen 29 4 0 21 Aug 2024
Continual Dialogue State Tracking via Reason-of-Select Distillation Yujie Feng Bo Liu Xiaoyu Dong Zexin Lu Li-Ming Zhan Xiao-Ming Wu Albert Y. S. Lam CLL LRM 42 5 0 19 Aug 2024
Reefknot: A Comprehensive Benchmark for Relation Hallucination Evaluation, Analysis and Mitigation in Multimodal Large Language Models Kening Zheng Junkai Chen Yibo Yan Xin Zou Xuming Hu 113 6 0 18 Aug 2024
Automatic Metrics in Natural Language Generation: A Survey of Current Evaluation Practices Patrícia Schmidtová Saad Mahamood Simone Balloccu Ondřej Dušek Albert Gatt Dimitra Gkatzia David M. Howcroft Ondřej Plátek Adarsa Sivaprasad 45 3 0 17 Aug 2024
Lower Layer Matters: Alleviating Hallucination via Multi-Layer Fusion Contrastive Decoding with Truthfulness Refocused Dingwei Chen Feiteng Fang Shiwen Ni Feng Liang Ruifeng Xu Min Yang Chengming Li HILM 28 1 0 16 Aug 2024
Unlocking Decoding-time Controllability: Gradient-Free Multi-Objective Alignment with Contrastive Prompts Tingchen Fu Yupeng Hou Julian McAuley Rui Yan 38 3 0 09 Aug 2024
Get Confused Cautiously: Textual Sequence Memorization Erasure with Selective Entropy Maximization Zhaohan Zhang Ziquan Liu Ioannis Patras 39 2 0 09 Aug 2024
Mitigating Hallucinations in Large Vision-Language Models (LVLMs) via Language-Contrastive Decoding (LCD) Avshalom Manevich Reut Tsarfaty MLLM 21 4 0 06 Aug 2024
Self-Introspective Decoding: Alleviating Hallucinations for Large Vision-Language Models Fushuo Huo Wenchao Xu Zhong Zhang Yining Qi Zhicheng Chen Peilin Zhao VLM MLLM 66 19 0 04 Aug 2024
Adaptive Contrastive Decoding in Retrieval-Augmented Generation for Handling Noisy Contexts Youna Kim Hyuhng Joon Kim Cheonbok Park Choonghyun Park Hyunsoo Cho Junyeob Kim Kang Min Yoo Sang-goo Lee Taeuk Kim 31 5 0 02 Aug 2024
Cool-Fusion: Fuse Large Language Models without Training Cong Liu Xiaojun Quan Yan Pan Liangzhi Li Weigang Wu Xu Chen VLM MoMe 53 3 0 29 Jul 2024
Learn from the Learnt: Source-Free Active Domain Adaptation via Contrastive Sampling and Visual Persistence Mengyao Lyu Tianxiang Hao Xinhao Xu Hui Chen Zijia Lin Jungong Han Guiguang Ding VLM TTA 31 5 0 26 Jul 2024
VACoDe: Visual Augmented Contrastive Decoding Sihyeon Kim Boryeong Cho Sangmin Bae Sumyeong Ahn SeYoung Yun 36 3 0 26 Jul 2024
Adaptive Contrastive Search: Uncertainty-Guided Decoding for Open-Ended Text Generation Esteban Garces Arias Julian Rodemann Meimingwei Li Christian Heumann Matthias Aßenmacher 43 3 0 26 Jul 2024
Internal Consistency and Self-Feedback in Large Language Models: A Survey Xun Liang Shichao Song Zifan Zheng Hanyu Wang Qingchen Yu ... Rong-Hua Li Peng Cheng Zhonghao Wang Feiyu Xiong Zhiyu Li HILM LRM 68 25 0 19 Jul 2024
Watermark Smoothing Attacks against Language Models Hongyan Chang Hamed Hassani Reza Shokri WaLM 65 2 0 19 Jul 2024
Multilingual Contrastive Decoding via Language-Agnostic Layers Skipping Wenhao Zhu Sizhe Liu Shujian Huang Shuaijie She Chris Wendler Jiajun Chen 30 3 0 15 Jul 2024
Knowledge Overshadowing Causes Amalgamated Hallucination in Large Language Models Yuji Zhang Sha Li Jiateng Liu Pengfei Yu Yi Ren Fung Jing Li Manling Li Heng Ji 37 11 0 10 Jul 2024
Pseudo-RIS: Distinctive Pseudo-supervision Generation for Referring Image Segmentation Seonghoon Yu Paul Hongsuck Seo Jeany Son DiffM 57 4 0 10 Jul 2024
Merge, Ensemble, and Cooperate! A Survey on Collaborative Strategies in the Era of Large Language Models Jinliang Lu Ziliang Pang Min Xiao Yaochen Zhu Rui Xia Jiajun Zhang MoMe 49 18 0 08 Jul 2024
Orchestrating LLMs with Different Personalizations Jin Peng Zhou Katie Z Luo Jingwen Gu Jason Yuan Kilian Q. Weinberger Wen Sun 54 2 0 04 Jul 2024
Securing Multi-turn Conversational Language Models Against Distributed Backdoor Triggers Terry Tong Lyne Tchapmi Qin Liu Muhao Chen AAML SILM 47 1 0 04 Jul 2024
Towards a Scalable Reference-Free Evaluation of Generative Models Azim Ospanov Jingwei Zhang Mohammad Jalali Xuenan Cao Andrej Bogdanov Farzan Farnia EGVM 40 8 0 03 Jul 2024