Abstract
Long-sequence memory tracking places two opposing demands on a recurrent state: near-lossless retention of stored bindings over long horizons, and active overwriting of stale ones. In our diagnostic suite, the strongest efficient baselines tend to solve only one side well. Continuous-time-parameterized state-space models (SSMs) such as Mamba obtain their discrete recurrence by zero-order-hold discretization of a continuous-time system; we argue that this detour is unnecessary for memory tracking and parameterize the discrete transition directly. Naju (Native Adaptive Junction Unit) factorizes the recurrent update, schematically: $$ x_n = f_n \odot x_{n-1} + i_n \odot(B_n u_n) $$ where $f_n$ is a learned forget gate, $i_n$ is an independent write gain, and input-dependent write/read maps. Since the sigmoid pole satisfies $0 < f_n < 1$, this model can effectively manage information forgetting and updating.
Blogger's Review: The Naju model simplifies the handling of long-sequence memory by directly parameterizing the discrete transition, successfully addressing the conflict between information retention and updating. The introduction of a forget gate and independent gain provides new insights for the design of memory systems, warranting further research and application.