Distributed model training must cope with two major issues: straggler delays and Byzantine attacks.
To ensure timely and reliable gradient aggregation across multiple workers, we introduce dSTAR, a lightweight and efficient distributed SGD approach. dSTAR gathers updates only from the first $k$ workers that respond and uses an ensemble median to compute deviations for each gradient. Updates whose deviation exceeds a preset threshold are filtered out. This strategy mitigates the impact of slow workers while defending against malicious gradient manipulation.
From a theoretical standpoint, dSTAR is proven to be $(\alpha, f)$‑Byzantine resilient, meaning it tolerates up to $f$ Byzantine nodes and still guarantees reliable gradients, achieving a linear convergence rate. Empirical tests across diverse attack models and network latency conditions show that dSTAR consistently retains high accuracy, with accuracy drops under attack limited to less than 5%, whereas competing Byzantine‑resilient methods often suffer 40%‑50% degradation.
In summary, dSTAR offers a robust and efficient solution for training models in distributed settings prone to both straggler delays and Byzantine faults.
Review