Lexicon-based constrained decoding approaches aim to control the meaning or style of the generated text through certain target concepts. Existing approaches over-focus the targets themselves, leading to a lack of high-level reasoning about how to achieve them. However, human usually tackles tasks by following certain rules that not only focuses on the targets but also on semantically relevant concepts that induce the occurrence of targets. In this work, we present DECIDER, a rule-controllable decoding strategy for constrained language generation inspired by dual-system cognitive theory. Specifically, in DECIDER, a pre-trained language model (PLM) is equiped with a logic reasoner that takes high-level rules as input. Then, the DECIDER allows rule signals to flow into the PLM at each decoding step. Extensive experimental results demonstrate that DECIDER can effectively follow given rules to guide generation direction toward the targets in a more human-like manner.
View on arXiv@article{xu2025_2403.01954,
title={ DECIDER: A Dual-System Rule-Controllable Decoding Framework for Language Generation },
author={ Chen Xu and Tian Lan and Yu Ji and Changlong Yu and Wei Wang and Jun Gao and Qunxi Dong and Kun Qian and Piji Li and Wei Bi and Bin Hu },
journal={arXiv preprint arXiv:2403.01954},
year={ 2025 }
}