SAMPAT (Smooth Approximation via Multivariate Polynomials and Analytic Transformations) is a three-layer neural architecture aimed at addressing the common issue of interpretability in deep learning models. Interpretability is crucial for analyzing experimental data, as mere quantitative predictions may not suffice for scientists. SAMPAT can provably learn a continuous, everywhere differentiable function, able to approximate any smooth function arbitrarily closely. Its approximant is expressed as a closed and compact algebraic analytic expression, providing complete interpretability.
Experiments on synthetic and benchmark datasets indicate that SAMPAT yields competitive performance with simpler representations. For many tasks, a two-layer SAMPAT suffices. By imposing restrictions on the connectivity between neurons, SAMPAT can provide a range of approximants, including regular and trigonometric polynomials, rational expressions, Gaussians, mixtures of Gaussians, as well as arbitrary combinations of these; without restrictions, it learns a suitable structure.
Moreover, SAMPAT can be used to factorize polynomials and model nonlinear systems. With the addition of skip connections, a 4 to 6 layer SAMPAT is adequate to represent a substantive range of methods widely used in AI/ML, allowing the choice of a model's family, not just its parameters, to also be optimized as part of the learning process.
Blogger's Review: The introduction of the SAMPAT architecture is a proactive response to the demand for interpretability in deep learning, especially in scientific research where trust in results is crucial. By incorporating multivariate polynomials, SAMPAT not only enhances model performance but also provides scientists with a more intuitive understanding, which could spark broader applications and interest in future research.