NeFut Logo NeFut
中 Admin Login

[CS.AI] QUARTET: Quad-branch Cross-Attention and Random-walk Traces for Enhancing Transformers on Relational Graphs

Published at: 2026-09-25 22:00 Last updated: 2026-09-28 00:49
#algorithm #Machine Learning #Graph

Relational Deep Learning (RDL) treats multi‑table databases as heterogeneous temporal graphs, and graph Transformers now dominate benchmarks such as RelBench. The current top model, RelGT, suffers from two drawbacks: its random local sampler yields loosely connected subgraphs that impede message passing, and its global attention relies on a single seed‑feature memory, ignoring broader macro‑level dynamics. QUARTET addresses these issues with a graph Transformer that applies full self‑attention on local subgraphs while enriching global context via four cross‑attention branches. A Causal Random Walk (CRW) sampler, built on recency‑truncated Personalized PageRank (PPR), extracts compact, hub‑robust, densely connected local subgraphs without temporal leakage. The quad‑branch cross‑attention injects global information from four complementary perspectives—seed feature, seed topology, temporal dynamics, and collaborative dynamics. Across RelBench v1 classification tasks, QUARTET consistently matches or surpasses state‑of‑the‑art baselines such as HGT and RelGT. Ablation studies confirm that the CRW sampler markedly improves local neighborhood quality, while the global branches deliver essential task‑specific predictive gains.

Review: QUARTET’s combination of high‑quality local sampling and multi‑view global cross‑attention effectively remedies RelGT’s limitations, achieving notable performance gains on real‑world benchmarks and highlighting the value of finer‑grained attention designs for heterogeneous temporal graphs.

Original Source: https://arxiv.org/abs/2609.26855

[h] Back to Home