Methodology

---

Temporal Self-Consistency Voting

Relying solely on the final prediction thus risks discarding better intermediate outputs.To address this, we propose a method that aggregates predictions across timesteps through a weighted voting mechanism. Formally, given a diffusion sampling trajectory \(\left\{x_0^t\right\}_{t=1}^{T}\), our method selects the final answer \(a^*\) according to a weighted vote over all timesteps: $$ a^* = \arg \max _a \sum_{t=1}^T f(t) \cdot \mathbb{1} \left(\text { meaning }\left(x_0^t\right) = a\right). $$ Here, \(\mathbb{1}(\cdot)\) is the indicator function that returns 1 if the decoded meaning of \(x_0^t\) matches the candidate answer \(a\), and \(f(t)\) is a weighting function over timesteps. Since the accuracy of predictions generally tends to improve as the sampling step increases (or diffusion step decreases), we design \(f(t)\) to be a monotonically decreasing function of diffusion step \(t\). In experiments, we explore three types of weighting schemes: constant, linear decay, and exponential decay.

Temporal Consistency Reinforcement

We introduce a metric called Temporal Semantic Entropy, which captures the distribution of semantic variations in answers generated at each step of the iterative denoising process. During decoding, we obtain a sequence of \(T\) intermediate answers, denoted by \(\{x_0^t\}_{t=1}^{T}\). We group these answers into clusters based on their semantic meaning, forming a set of semantic clusters \(\mathcal{C} = \{C_1, C_2, \ldots, C_K\}\), where each cluster \(C_k = \{x_0^t : \text{meaning}(x_0^t) = k\}\) contains all answers with equivalent semantics (outcome equals \(k\)). We then define the temporal semantic entropy (TSE) of a trajectory as: $$ \operatorname{TSE}(\left\{x_0^t\right\}_{t=1}^{T})=-\sum_{C_k}\left(\left[\sum_{x_0^t \in C_k} p\left(x_0^t\right)\right] \log \left[\sum_{x_0^t \in C_k} p\left(x_0^t\right)\right]\right), $$ which quantifies the uncertainty in the semantic content of answers over the decoding steps. In our analysis we observe that high temporal semantic entropy may serve as a signal for model uncertainty. Thus we propose a post-training approach designed to encourage temporal consistency in model outputs. we define the reward \(r_i\) as the negative Temporal Semantic Entropy of response \(o_i\), i.e., \(r_i = -\text{TSE}(o_i)\).This reward encourages outputs whose intermediate token representations maintain semantic stability throughout the generation process, thereby promoting temporal semantic consistency.

What's more, we combine TSE with the accuracy reward using scoring rule. Specifically, we consider the following four forms: $$ \begin{aligned} \textbf{Entropy scoring:} \quad & r^{\text{ent}}_{i} = \mathbb{1}_{o_i = o^*} \cdot c(o_i) \\ \textbf{Quadratic scoring:} \quad & r^{\text{quad}}_{i}= \mathbb{1}_{o_i = o^*} -(c(o_i) - \mathbb{1}_{o_i = o^*})^2 \\ \textbf{Logistic scoring:} \quad & r^{\text{log}}_{i} = \mathbb{1}_{o_i = o^*} + \mathbb{1}_{o_i = o^*} \log (c(o_i)) + (1 - \mathbb{1}_{o_i = o^*}) \log (1 - c(o_i)) \\ \textbf{Spherical scoring:} \quad & r^{\text{sphe}}_{i} = \mathbb{1}_{o_i = o^*} + \frac{c(o_i)}{\sqrt{(c(o_i))^2 + (1 - c(o_i))^2}} \end{aligned} $$ All four proposed reward functions aim to jointly encourage correctness and temporal self-consistency. Notably, the \(r^{\text{ent}}_{i}\) function gives a reward of 0 for incorrect answers, while for correct answers, the reward is given by \(c(o_i) = \frac{\mathcal{H}_{\max} - \operatorname{TSE}(o_i)}{\mathcal{H}_{\max}}\), where \(\mathcal{H}_{\max} = \log T\). This reward reaches a maximum value of 1 when all sampling steps yield the correct answer. By default, we use the spherical scoring rule because it demonstrates more superior results compared to the alternatives.

Experimental Results

---

Key Insights

---

Citation

@article{wang2025temporaldynamics,
  title={Time Is a Feature: Exploiting Temporal Dynamics in Diffusion Language Models},
  author={Wen, Wang and Bozhen, Fang and Chenchen, Jing and Yongliang, Shen and Yangyi, Shen and Qiuyu, Wang and Hao, Ouyang and Hao, Chen and Chunhua, Shen},
  journal={arXiv preprint arXiv:2508.09138},
  year={2025}
}

Copy BibTex

Acknowledgements

We would like to thank Muzhi Zhu, Canyu Zhao, and Linhao Zhong at Zhejiang University for their valuable discussions and insightful feedback. We also acknowledge the template from Ximing Xing that helped us build this project homepage.