- For scalar discrete random variables with K categories
Forward process
- Transition matrix [Qt]ij=p(xt=j∣xt−1=i)
- Forward process
- q(xt∣xt−1)=Cat(xt;p=xt−1Qt)
- where Cat(x;p) is a categorical distribution over the one-hot row vector x with probabilites given by the row vector p.
- xt−1Qt is a row vector-matrix product
- It is assumed that Qt is applied to each pixel or token independently, so q(xt∣xt−1) represents the forward process for the entire object
- Closed forms
- Forward: q(xt∣x0)=Cat(xt;p=x0Qˉt) with Qˉt=Q1...Qt
- Posterior: q(xt−1∣xt,x0)=q(xt∣x0)q(xt∣xt−1,x0)q(xt−1∣x0)=Cat(xt−1;p=x0QˉtxtTxtQtT⊙x0Qˉt−1)
Choice of transition matrix Qt
- must converge to a know stationary distribution