Large Language Models (LLMs) have shown strong performance in various language tasks, yet their capabilities in accessibility-critical modalities like Braille remain unclear. We evaluated state-of-the-art LLMs on bidirectional Korean-Braille translation using a human-annotated dataset. Contrary to expectations that multilingual, instruction-tuned models could generalize to Braille through text representations, we found consistently poor and unstable outputs with substantial disagreement from human judgments. These results highlight a lack of Braille-aware tokenization and weak alignment between Korean and Braille patterns. In contrast, supervised fine-tuning of a smaller model (T5-small) on the same dataset achieved significant and stable improvements over zero-shot and prompted LLM baselines across standard metrics (SacreBLEU, ChrF++, CER, BLEU, ROUGE-L, METEOR, CIDEr). Our findings reveal a systematic limitation of current LLMs and demonstrate the effectiveness of modest task-specific supervision.
Blogger's Review: This study highlights the shortcomings of current large language models in the application of accessibility technologies, particularly in Braille translation, underscoring the importance of task-specific fine-tuning. It provides significant insights for future model improvements and the development of accessibility technologies.