Complex algorithms, such as deep neural networks, are increasingly deployed on embedded, resource-constrained platforms. However, existing hardware and software schemes for implementing these models on the edge fall short, particularly in safety-critical applications like medical devices.
First, hardware such as GPUs, NPUs, and TPUs is designed for throughput rather than computational correctness, making them susceptible to fault injection attacks. Second, software schemes for porting algorithms onto edge devices—like quantization schemes—are either static and sound (non-optimal power consumption) or dynamic yet unsound (not suitable for safety-critical applications).
To address these issues, we propose a novel approach to real-time, dynamic, and sound quantization, along with hardware support. We developed a sound, real-time adaptive-precision quantization approach that utilizes left-to-right arithmetic to pass the most significant bits (MSB) first and dynamically adjusts precision online while conducting sensitivity analysis to quantify and manage the risk of decision-boundary crossings.
Next, we introduce a novel hardware approach using systolic arrays to perform left-to-right arithmetic to generate MSB first. This scheme enables resource-efficient neural networks and AI at the edge while ensuring sound and resource-efficient high-precision mathematics on hardware, providing resilience against bit flip attacks on the most critical bits. This work is currently in progress, with software implementations completed and hardware development underway.
Blogger's Review: This article presents an innovative dynamic quantization and hardware architecture addressing safety and performance challenges in embedded systems, showcasing the potential for high-precision computing in resource-constrained environments, particularly in safety-critical applications like medical devices.