We introduce a framework that learns cross‑task relationships in multi‑task models by approximating the joint distribution of task labels through targeted pairwise relationships. This sidesteps the intractable complexity of modeling the full joint space while leveraging transfer learning to boost overall performance.
The key idea is to decompose the high‑dimensional joint distribution into a set of manageable binary relations. For each pair of tasks we capture the dependency via conditional probabilities, mutual information, or similar statistics, preserving collaborative signals without sacrificing scalability.
We validate the approach in YouTube’s production recommendation pipelines, covering the Notifications, Homepage, and Watch Next surfaces. Across click‑through rate, watch time, and user satisfaction metrics, the inclusion of cross‑task relationships yields consistent improvements, typically a 3%–5% relative gain, especially helping sparse tasks benefit from dense ones.
To ease adoption, we provide a workflow template that outlines data preparation, relationship extraction, model integration, and online A/B testing steps. The template is designed to be portable to other multi‑task scenarios, lowering engineering overhead.
Review: The framework demonstrates notable gains in a real‑world production setting and is a strong candidate for broader adoption in other multi‑task systems.