E‑commerce live streaming often contains noisy, long‑duration streams where product information is scattered across the host's speech, video frames, product images, overlaid text, and user queries. To address this, we introduce TLive‑Omni, an omni‑modal understanding model tailored for live‑commerce scenarios. The model projects image, video, audio, and text inputs into a shared representation space, enabling cross‑modal fusion. For analyzing extended live streams, we propose Per‑vGrid, a timestamped token organization that groups each video grid with its temporally aligned audio inside explicit boundary tokens, facilitating precise temporal alignment. Training follows a three‑stage supervised recipe: first learning omni‑modal perception, then instruction‑following response generation, and finally applying Faithful‑RFT reinforcement fine‑tuning to improve answer faithfulness and expression quality while meeting real‑time constraints. Faithful‑RFT scores final responses directly with task‑verifiable feedback instead of optimizing exploratory reasoning during rollout. The system is supported by a scenario‑oriented atomic capability taxonomy and a compact data production engine that converts live‑commerce audio‑video streams into training signals for speech recognition, speaker analysis, product visual grounding, text recognition, temporal grounding, video dense captioning, and omni‑modal QA. For scalable training, a synchronized length‑grouped sampler reduces padding while keeping workloads comparable across workers, and a lightweight dynamic sampling strategy regenerates rollout groups with near‑zero reward variance to preserve meaningful relative advantages for GRPO. Experiments on e‑commerce live streaming benchmarks show strong performance across domain tasks and good generalization on general benchmarks.
Blogger's Review: TLive‑Omni’s novel temporal alignment and staged training pipeline markedly boost omni‑modal comprehension in live‑commerce, offering a practical solution for real‑world deployment.