Recent FPGA designs have significantly improved deep learning inference efficiency due to dedicated tensor blocks and in-BRAM computation. ReRAM-based analog in-memory computing (IMC) pushes this efficiency further, offering an order-of-magnitude improvement in compute density and energy efficiency by performing vector-matrix multiplication (VMM) directly within the ReRAM crossbar.
However, conventional IMC designs support only static-weight VMM, leaving nonlinear operations and dynamic matrix-matrix multiplication (DIMM) to the FPGA fabric. Consequently, the benefits of IMC are mainly confined to static-weight models, while Transformer-based models, which rely on frequent nonlinear and DIMM operations, gain only limited improvements.
Additionally, the ADCs within each IMC block consume over 70% of its area and power, further limiting system efficiency and scalability.
To address these limitations, we propose a novel FPGA architecture that integrates an ADC-free IMC block, replacing the conventional ADC with analog content-addressable memories (ACAMs) that natively perform nonlinear operations inside the block.
To fully exploit this block, we conduct an FPGA-aware design-space exploration to determine optimal crossbar dimensions while balancing FPGA area, flexibility, and deep learning performance. We develop an efficient mapping that leverages ACAMs to carry out DIMM operations, extending the applicability of IMC to attention computation.
On CNN and Transformer-based benchmarks, the proposed architecture achieves up to 40x and 1.9x higher energy efficiency and 4.1x and 2.5x higher area efficiency, respectively.
Overall, it significantly improves FPGA deep learning inference efficiency and sustains robust gains on Transformer-based workloads across long input sequences, advancing domain-specialized FPGA design.
Blogger's Review: This study highlights the potential of FPGA in deep learning inference, particularly in Transformer model applications. By introducing ACAMs, it overcomes the bottleneck of traditional ADCs, greatly enhancing energy and area efficiency. This marks a new direction in FPGA design, warranting further exploration and application.