A Reconfigurable Floating-Point Division and Square Root Architecture for High-Precision Softmax

With the advancement of deep learning models, the Softmax function with self-attention has become pervasive in everyday applications. As components of the Softmax function and its inputs, both division and square root operations impact its accuracy. However, these two non-linear operations bring significant area and power consumption for hardware implementation. To address these challenges, this paper proposes a reconfigurable floating-point division and square root (FDSR) architecture that achieves low resource consumption and high accuracy for general-purpose computation. The FDSR enhances the traditional non-restoring algorithm by using shift-registers and optimizing the leading-one detection and shift operations, reducing hardware resource usage while maintaining high accuracy (0.5 ULP). In the mantissa calculation, the division can be converted to a square root operation by simply switching the input to the subtractor through multiplexers. Additionally, a triple-mode reconfigurable iteration unit is introduced, featuring a multi-layer variable pipeline architecture to improve adaptability for different applications. By redesigning the pipeline depth and reusing logical units, the FDSR effectively addresses the issue of lengthy iteration cycles in the non-restoring method. Implementation results using 40nm CMOS technology demonstrate that the proposed design achieves a 76.49% power reduction and a 14.69% area reduction for floating-point division compared to Synopsys Design Ware and an 88.05% power reduction and a 90.57% area reduction for floating-point square root. With 28 nm CMOS technology, the FDSR reduces power consumption by 91.55% and reduces area by 64.39% for floating-point division compared to Synopsys Design Ware. On the FPGA platform, the FDSR significantly reduces hardware resource consumption, achieving an 85.23% reduction for floating-point division and 87.81% for floating-point square root, outperforming state-of-the-art designs.