Language models (LMs) are often trained with Reinforcement Learning using Verifiable Rewards (RLVR) to boost their reasoning abilities. However, RLVR does not explicitly address calibration during training, which can cause severe calibration degradation, especially over‑confidence. Recent calibration‑aware training methods incorporate uncertainty estimation objectives, improving calibration but still suffering from over‑confidence under distribution shift and sacrificing reasoning performance.\ \ To tackle these issues, we introduce RL-ARC, a calibration‑aware training framework that jointly leverages reasoning confidence and answer confidence. Specifically, RL-ARC uses reasoning confidence as an auxiliary signal to calibrate answer confidence: for correct answers it acts as a regularizer that reinforces reliable confidence estimates, while for incorrect answers it serves as an over‑confidence penalty to suppress unwarranted high confidence.\ \ Extensive experiments on both in‑distribution (ID) and out‑of‑distribution (OOD) settings show that RL-ARC not only markedly improves calibration but also enables models to adaptively estimate confidence based on each question, with negligible impact on reasoning performance. The results highlight reasoning confidence as a crucial factor for training trustworthy reasoning models.\ \ Review: RL-ARC effectively integrates reasoning‑guided confidence into answer calibration, achieving better calibrated models without compromising reasoning strength, making it a promising approach for reliable large‑scale reasoning systems.