NeFut Logo NeFut
中 Admin Login

[CS.AI] Spatio-Temporal Multi-Order Interaction for Video Recognition

Published at: 2026-11-06 22:00 Last updated: 2026-11-07 12:32
#Machine Learning #optimization #Neural

Many temporal approaches for video recognition rely on explicit cross‑channel aggregation projections, whose computational cost grows quadratically with the number of channels, causing heavy overhead. At the same time, existing spatio‑temporal representations are biased either toward local order interactions (CNNs) or toward global order interactions (Transformers), making it hard to capture both aspects. Building on MogaNet’s multi‑order spatial representation, we introduce two complementary improvements. First, for temporal modeling we employ parallel multi‑order processing using depthwise‑only convolutions and an implicit gating‑based aggregation, achieving linear complexity with respect to channel count and greatly reducing computation. Second, we design a dual interaction between spatial and temporal dimensions, allowing mutual reinforcement without extra cost. Combined, VideoMogaNet attains or surpasses state‑of‑the‑art action recognition performance on benchmarks such as UCF101, HMDB51, Kinetics‑400, and Diving48, while substantially cutting the temporal branch’s computational burden.

Review: This work delivers a practical and efficient solution by coupling a linear‑complexity temporal module with a spatial‑temporal dual interaction, achieving accuracy gains without added cost—a notable advance for video recognition.

Original Source: https://arxiv.org/abs/2611.05633

[h] Back to Home