NeFut Logo NeFut
中 Admin Login

[CS.AI] From Waste to Discovery: Recycling High-Scored Unexposed Candidates in Taobao Feed Recommendation

Published at: 2026-11-05 22:00 Last updated: 2026-11-07 12:32
#algorithm #Machine Learning #optimization

Helping users discover new interests is essential for the long‑term health of industrial recommender systems, yet the usual trick of boosting under‑explored (discovery) items often backfires in production. We deployed URED (Unexposed Recycling for Efficient Discovery) in Taobao feed recommendation at Alibaba. It is a training‑free recall‑stage system that has run stably for over six months serving tens of millions of daily active users. The key insight is that naive boosting injects cold, information‑poor candidates that rankers systematically over‑estimate, yielding no discovery gain and risking engagement. In contrast, items that receive high scores but are never shown (commonly discarded by industrial pipelines) form a pool that is both relevant and reliably predictable. URED aggregates such items over a 30‑day window, stores them in distributed offline storage, and re‑injects them at recall time through a lightweight online path that performs a single O(1) lookup per request, adding less than 1 % latency and requiring no model retraining. Online A/B tests show a 3.47 % increase in discovered categories and an 8.47 % boost in content richness while preserving engagement; completed video views rise 2.62 % and total views move slightly positive (+0.36 %, not significant). Ablation studies confirm both pillars are necessary: shrinking the window to one day collapses the discovery gain to 0.55 % (not significant), and random injection without score filtering yields no gain and a slight engagement drop (‑0.17 %). Across all recall channels, URED achieves a discovery rate 13.9× the platform average and 6.3× the strongest near‑line baseline, with engagement quality on par with the platform. We share deployment details of offline aggregation, distributed storage, and low‑latency online serving, and provide practical lessons for tackling the exploration‑exploitation dilemma under tight resource constraints.

Review: Leveraging high‑scored yet unexposed candidates offers a low‑cost path to richer discovery, highlighting the value of data recycling in large‑scale recommendation pipelines.

Original Source: https://arxiv.org/abs/2611.05997

[h] Back to Home