General-purpose code embeddings power tools such as code search, classification, and retrieval. Compact transformer encoders usually rely on human‑written docstrings (expensive and inconsistent) or on mined structural signals like execution traces (environment‑specific and costly). This work empirically investigates an alternative: contrastive pre‑training of small encoders within a dual‑encoder framework, using synthetically generated natural‑language descriptions that highlight a program’s functionality and intent. During training the description and the code are paired; at inference only the code encoder is kept. We evaluate the approach on eight retrieval, classification, and generation tasks across C, C++, and Java, comparing it with pre‑training baselines, generalist LLMs, and embedding‑specific models. Synthetic semantic supervision yields statistically significant improvements over same‑size baselines on five of eight tasks and parity on two more. After fine‑tuning it matches or exceeds zero‑shot models that are two orders of magnitude larger on classification, and it remains on par with execution‑aware supervision when pre‑training data is matched, suggesting a scalable and effective alternative to existing code‑representation paradigms.
Review