Large language models (LLMs) are increasingly used as automatic judges for chatbot responses. Proprietary models achieve high agreement with human annotators but are expensive and hard to deploy. Lightweight open‑source LLM judges are cheaper yet often diverge from human ratings. This work investigates whether soft‑prompt tuning can bring open‑source judges closer to human annotation policies. We train soft prompts on the FairBotBench benchmark and evaluate the adapted judges on bias, toxicity, and personalization. Results show that hybrid soft‑prompt tuning, which blends textual prompts with learned continuous embeddings, consistently improves agreement with human annotations compared to untuned baselines. For Qwen 3.5 9B, the tuned judges outperform hosted reference judges across all three dimensions.\n\nReview: Soft‑prompt tuning offers an effective way to boost the reliability of open‑source LLM judges, especially valuable for cost‑constrained real‑world deployments.