Test-Time Alignment of Discrete Diffusion Models with Sequential Monte Carlo

Discrete diffusion models have become highly effective across various domains. However, real-world applications often require the generative process to adhere to certain constraints but without task-specific fine-tuning. To this end, we propose a training-free method based on Sequential Monte Carlo (SMC) to sample from the reward-aligned target distribution at the test time. Our approach leverages twisted SMC with an approximate locally optimal proposal, obtained via a first-order Taylor expansion of the reward function. To address the challenge of ill-defined gradients in discrete spaces, we incorporate a Gumbel-Softmax relaxation, enabling efficient gradient-based approximation within the discrete generative framework. Empirical results on both synthetic datasets and image modelling validate the effectiveness of our approach.

@article{pani2025_2505.22524,
  title={ Test-Time Alignment of Discrete Diffusion Models with Sequential Monte Carlo },
  author={ Chinmay Pani and Zijing Ou and Yingzhen Li },
  journal={arXiv preprint arXiv:2505.22524},
  year={ 2025 }
}