Manifold learning methods such as t‑SNE and UMAP preserve local neighborhoods but often ignore the relative order of neighbors. Two embeddings may share the same neighbor set while disagreeing on which point is closest, which harms downstream tasks that depend on ordering. We therefore introduce Smooth Rank t‑SNE (SR‑tSNE), which augments the standard t‑SNE loss with a differentiable rank term. This term uses a sigmoid rank approximation to explicitly force true high‑dimensional neighbors to outrank unrelated intruders in the low‑dimensional embedding while retaining their original order. We also extend the Degree of Locality Preservation (DLP) metric to Ordinal DLP (O‑DLP), awarding a neighbor only when it appears at its original rank. Experiments across eight benchmarks covering images, text, speech, remote sensing and single‑cell data show that SR‑tSNE outperforms 12 baselines in both rank preservation and neighborhood overlap. Ablation studies reveal that the rank term alone boosts O‑DLP by 5%–28%, a gain unattainable with standard pairwise losses. This advantage transfers to downstream tasks: SR‑tSNE achieves the best label‑transfer performance on seven vision datasets and ranks first in 17 of 18 dataset‑metric combinations for cell‑type discovery across six single‑cell datasets, regardless of whether clustering resolution is selected with or without cell‑type labels. In intraday trading, SR‑tSNE yields higher returns, lower risk, and lower turnover than baselines. To our knowledge, SR‑tSNE is the first t‑SNE‑family method that explicitly preserves order.\n\nReview: By introducing a differentiable rank approximation, SR‑tSNE provides fine‑grained control over embedding order, delivering consistent gains across diverse modalities and real‑world applications, and opening a new evaluation dimension for manifold learning.