Abstract
Graph Neural Networks (GNNs) and Large Language Models (LLMs) have both played significant roles in recommendation systems by modeling structural and semantic signals, respectively. However, integrating their complementary strengths remains a challenge, especially in sparse settings where semantic precision is crucial. We propose TRWH (Text-driven Random Walk Heterogeneous Graph Neural Network), a novel framework that merges LLM-generated textual profiles with heterogeneous graph structures through strategic random walk augmentation.
TRWH consists of three core components:
- Embedding Creation: Produces user and item representations using both Word2Vec and LLM-based profiling.
- Heterogeneous Graph Neural Network (HeteroGNN): Propagates information across multi-relational edges.
- Random Walk-based Path Construction: Enriches sparse graphs with second-order user-user and item-item links.
Experiments on the Amazon-2023 Fashion (2M users, 825K items) and Beauty (631K users, 112K items) datasets demonstrate that TRWH achieves substantial performance gains over state-of-the-art methods, including 80.0% RMSE and 52.6% MAE reductions on Fashion, and 25.7% and 10.8% improvements on Beauty. Notably, while random walks improve performance with traditional embeddings, they can dilute the nuanced representations learned by LLMs, underscoring the importance of adaptive integration strategies.
Blogger's Review: The TRWH framework innovatively combines the strengths of GNNs and LLMs, showcasing substantial performance improvements in sparse recommendation scenarios. Future research could further explore optimizing the integration of random walks with LLMs to retain more semantic information.