This work investigates whether auxiliary trajectory preferences can boost self‑improvement and transfer of small language models under limited supervision. We compare two reward families: confidence‑based self‑confidence (SC) rewards and entropy‑based UID‑Conj. rewards, which combine global entropy variation with local smoothness. Experiments consist of online reward augmentation and offline training on correct responses.\n\nIn the online setting, Qwen3‑1.7B is adapted to 30 AIME problems without reference answers; the offline setting uses historical AIME problems. UID‑Conj. improves the macro‑average across the adaptation benchmark and three transfer benchmarks by 2.08 percentage points over the unmodified TTRL. In weighted SFT, UID‑Conj. outperforms uniform weighting on the same response pool by 1.93 points, with a paired prompt‑bootstrap 95% confidence interval of [0.44, 3.44].\n\nThe benefits are task‑dependent: SC rewards better assess trace quality and instruction following, while UID‑Conj. rewards yield stronger training outcomes in most scenarios. The study shows that auxiliary preferences can expose learning value among equally correct traces that inference alone cannot capture, motivating evaluation through downstream gains.\n\nReview: Auxiliary rewards provide fine‑grained learning signals for small models, especially when full answers are unavailable, and merit further exploration across more reasoning‑intensive tasks.