NeFut Logo NeFut
中 Admin Login

[CS.AI] ATEMPO: Amortized Transfer Entropy Model Per‑lag Over‑Time

Published at: 2026-10-28 22:00 Last updated: 2026-10-29 00:58
#algorithm #AI #Machine Learning

When one time series informs another, the delay at which the information arrives determines whether a model can exploit the relationship. A source may be highly predictive at a specific lag but essentially uninformative at neighboring lags. Fixed‑length input windows therefore bundle useful and irrelevant history together, obscuring the true structure. Transfer entropy (TE) measures how much the past of a source adds explanatory power about a target beyond the target’s own history, thus recovering these delays. Existing TE estimators require separate fits over source and conditioning depths and repeated local‑window estimation to track time‑varying lags. We introduce ATEMPO, a diffusion‑based TE estimator amortized over lags and time. During training we randomly sample source depth $k$ and conditioning depth $l$ and explicitly condition on temporal position, allowing a single score network to output the full TE surface in one go without refitting. Differencing this surface yields the arrival times of information. On synthetic datasets with ground‑truth TE, ATEMPO attains the lowest mean relative error; on datasets with known lags, its lag‑detection accuracy reaches $0.98\pm0.04$. ATEMPO requires only one fit for source depths $K=6$ and target depths $L=3$, whereas traditional estimators need 18 separate fits for the same $K\\times L$ grid. Across three systems with time‑varying lag, ATEMPO is the only estimator that maintains perfect lag detection for all window lengths. The recovered delays improve downstream forecasting: on WEATHER-5K, ATEMPO‑selected inputs reduce MSE by 7.25%, and using 120 selected inputs outperforms the benchmark’s 240‑feature default by 4.6%. In three brain‑computer interface tasks, ATEMPO‑chosen lags achieve the highest or tied‑best $R^2$, improving mean $R^2$ by 4.1% relative to the strongest baseline.

Review: ATEMPO’s ability to learn the entire transfer‑entropy surface in a single model dramatically cuts computational overhead while delivering robust lag detection, offering a practical tool for causal time‑series analysis and predictive modeling.

Original Source: https://arxiv.org/abs/2610.36536

[h] Back to Home