Abstract
Scientific Machine Learning (SciML) methods, such as Neural Ordinary Differential Equations (NODEs), Physics-Informed Neural Networks (PINNs), and Universal Differential Equations (UDEs), are most effective when structural priors reflect reliable governing dynamics. This study investigates what occurs when this assumption is violated.
Using macroeconomic forecasting as a stress-test domain, we evaluate five model families: ARIMA, LSTM, NODE, PINN, and UDE across 23 countries with sparse annual data, multiple temporal splits, and five random seeds.
Our results indicate that none of the evaluated models achieve consistently strong forecasting performance, underscoring the difficulty of low-frequency macroeconomic prediction. However, a clear relative hierarchy emerges: less-constrained models, particularly ARIMA and NODE, consistently outperform more-constrained heuristic-prior models such as PINN and UDE.
Rather than viewing this as a rejection of SciML, we interpret it as a diagnostic finding: structural priors can act as misregularizers when they do not match the data-generating process.
We identify failure modes including prior misalignment, regime shifts, structural breaks, and optimization instability, arguing that SciML practitioners should test whether structure helps before assuming that more structure is beneficial.
Blogger's Review: This study reveals the complexities of structural priors in scientific machine learning, emphasizing the importance of thorough testing in practical applications. For researchers looking to apply SciML, understanding the relationship between models and data is crucial. Over-reliance on structural priors may lead to performance degradation, ultimately affecting the reliability of predictions.