This paper introduces a reinforcement learning approach enhanced by semantic stop embedding to mitigate bus bunching. Traditional reinforcement learning controllers primarily rely on instantaneous operational variables or route-specific stop identifiers, which provide limited information about the functional and operational context of individual stops. This study uses a large language model (LLM) to transform heterogeneous stop information into fixed semantic embeddings that are incorporated into a deep Q-learning controller without requiring real-time LLM inference. The results show that the semantic controller reduces headway variability, bunching events, and passenger waiting time by 32.0%, 69.2%, and 24.0%, respectively, compared to the best calibrated Daganzo baseline. Furthermore, semantic stop information improves headway regularity, waiting time, and holding effort, providing a more favorable overall trade-off across control objectives. The findings suggest that semantic state representations can complement conventional operational states and support adaptation-based policy reuse across related transit routes. Blogger's Review: This paper proposes an innovative approach to solving the bus bunching problem by utilizing LLM for semantic stop embedding and applying it to a reinforcement learning controller. The experimental results demonstrate that this method can effectively reduce bunching events and passenger waiting time, with good practical value and research significance.