Kavish constructed an autoregressive diffusion model to synthesize US equity market‑event streams using four years of timestamped data containing price, event kind (trade, order, cancel, etc.) and other fields. A causally masked transformer encoder produced a latent representation for each prior event; a lightweight event‑kind head predicted a two‑category distribution (trade vs. best‑bid‑offer update) while a diffusion head, conditioned on the latent and the true event kind via AdaLN, generated continuous targets such as price and elapsed time. During inference the model sampled an event kind from the encoder‑conditioned distribution and then generated the continuous features of the next event with the diffusion head. Initial experiments with a DDPM using a 1,000‑step cosine schedule and DDIM sampling proved unstable, yielding denoising trajectories in which 88‑95% of the seven continuous variables exceeded eight standard deviations from the data mean. Switching to a flow‑matching formulation eliminated this divergence and provided stable sampling with few steps. To address the inherent discreteness of market data, Kavish expanded the categorical head to 20 fine‑grained classes (e.g., size‑only, ask‑up, bid‑down) to capture point masses and applied an “atom smoothing” procedure that smoothed spiky distributions before reinjecting the sharp spikes. This hybrid approach improved marginal distributions of both event kinds and continuous features, as measured by total‑variational divergence, and a classifier trained to distinguish model‑generated from real events showed markedly higher realism for the atom‑smoothed flow‑matching model, although autoregressive rollouts still displayed widening spreads in synthetic CME US price paths, indicating remaining gaps in temporal fidelity.

Read original