NeFut Logo NeFut
中 Admin Login

[CS.AI] SeqFold: Timestamp-Addressable Compressed States for User Action Sequence Backbones

Published at: 2026-11-12 22:00 Last updated: 2026-11-13 06:56
#algorithm #AI #Machine Learning

Existing sequential user‑modeling approaches mainly aim to improve the backbone encoder or to learn task‑specific user representations for a single prediction objective. This work explores a different angle: exporting the multi‑layer states of a frozen, long‑history backbone as compact, timestamp‑addressable representations for downstream training backfill and serving.\n\nIn this setting, the user state at a given timestamp summarizes all actions before that moment, so downstream systems only need the state aligned with each target timestamp instead of the full hidden‑state timeline. To achieve this, we introduce SeqFold, which combines three techniques to produce compressed states:\n1. Learned multi‑channel fusion that projects features from different layers into a unified multi‑dimensional space;\n2. Channel‑specific temporal decay that naturally reduces the influence of older actions;\n3. Lookup‑Free Quantization (LFQ) that directly quantizes the fused continuous vectors into discrete codes.\n\nTask‑specific decoders then map the shared discrete codes back into flexible embedding spaces, preserving model expressiveness. To make timestamp‑addressable export practical at scale, we further develop a fused streaming implementation: the fusion recurrence is updated after every user action, but states are materialized only at predefined target timestamps.\n\nExperiments on several large‑scale proprietary datasets show that SeqFold reduces per‑state storage by $3{,}400\times$ while retaining most of the downstream ranking performance of the uncompressed model. These results demonstrate that SeqFold is an effective interface for reusing long‑history sequence backbones in downstream models.\n\nReview: SeqFold’s structured fusion and quantization dramatically shrink the otherwise massive time‑series states, achieving both storage and inference gains. The approach offers a practical pathway for deploying rich user‑behavior backbones in real‑world systems.

Original Source: https://arxiv.org/abs/2611.13704

[h] Back to Home