Real-time speech-to-speech translation (S2ST) must juggle translation quality, latency, naturalness of the generated speech, and speaker consistency. While publicly documented S2ST systems have pushed forward direct, multilingual, streaming, and expressive modeling, commercial products and APIs are increasingly offering real-time translation to end users. However, deploying reproducible systems for long-form and multi‑speaker conversations remains difficult because partial ASR hypotheses are unstable, turn boundaries are ambiguous, and the target speech needs a suitable speaker prompt.
We introduce X-Translator, a low‑cost modular cascaded S2ST pipeline that stitches streaming ASR, machine translation, and prompt‑conditioned TTS together via a session‑level runtime controller. The system uses incremental segment commitment to turn unstable ASR streams into translation‑ready units, and an online speaker prompt manager to bind source speech spans to speaker‑specific voice prompts for synthesis.
Using OpenSTBench, we evaluate translation accuracy, speech quality, and latency, comparing against proprietary speech‑translation APIs as behavioral baselines. Additional experiments measure voice stability over long utterances, speaker preservation in multi‑speaker dialogs, and multilingual translation quality. The results show that X-Translator achieves acceptable translation and naturalness while keeping latency low, and it effectively maintains speaker characteristics.
X-Translator provides an open platform for studying the practical trade‑offs of deployment‑oriented S2ST. The code and demo are released at https://github.com/zhaoyx239/X-Translator.
Review