Temporal foundation models such as Chronos and TimesFM achieve cross‑domain generalization by sharing a numerical observation space. In contrast, discrete event sequences (DES) – e.g., operational logs or patient clinical trajectories – have domain‑specific event vocabularies with no consistent label correspondence across domains. Real‑world settings often contain a large number of distinct events, which challenges classical temporal point‑process methods and current in‑context inference models like FIM‑PP. Existing DES foundation models are limited to a small, fixed set of event types, restricting their use on high‑dimensional streams.
To overcome these limitations, we introduce two complementary modules: (i) DES‑JEPA, a joint‑embedding predictive architecture that learns a shared discrete vocabulary from synthetic stochastic processes with unaligned vocabularies; (ii) DES‑PFN, a prior‑fitted network that reuses the learned vocabulary for in‑context next‑event prediction. Without task‑specific fine‑tuning, the framework attains competitive zero‑shot performance on eight unseen real‑world datasets and scales to up to 700 distinct event types.
Review