/ˌɛf ˈpiː ˈθɜr ti ˈtu/
n. "IEEE 754 single-precision 32-bit floating point format balancing range and accuracy for graphics/ML workloads."
FP32 is the ubiquitous single-precision floating-point format using 1 sign bit, 8 exponent bits, and 23 mantissa bits to represent numbers from ±1.18×10⁻³⁸ to ±3.4×10³⁸ with ~7 decimal digits precision—standard for GPU shaders, SIMD vector math, and RNN inference where FP64 precision wasteful. Normalized values store as ±1.m × 2^(e-127) with denormals extending tiny values near zero.
Key characteristics of FP32 include:
- IEEE 754 Layout: 1 sign + 8 biased exponent (127) + 23 fraction bits = 32 total.
- Dynamic Range: ±10⁻³⁸ to ±10³⁸; gradual underflow via denormals to 1.4×10⁻⁴⁵.
- Precision: ~7.2 decimal digits; machine epsilon 1.19×10⁻⁷ between 1.0-2.0.
- Tensor Core Native: NVIDIA A100/H100 FP32 accumulation from FP16/BF16 inputs.
- Memory Efficiency: 4 bytes/value vs FP64 8 bytes; 2x HBM capacity.
A conceptual example of FP32 matrix multiply flow:
1. Load FP32 A + B from HBM @1.2TB/s
2. Tile 16x16 blocks to SM registers (256KB/core)
3. SIMD FMA: 16x FP32 MAC/cycle × 64 CUDA cores = 1TFLOP/clock
4. Accumulate to FP32 C with 24-bit precision
5. Store result HBM; 2.8 TFLOPS achieved @1.4GHz
6. 33ms total for 67T ops (A100 spec)Conceptually, FP32 is like a digital slide rule with 7-digit readout—trades half the precision of FP64 for 4x HBM throughput and 2x SIMD register density, perfect when 0.0001% error tolerable in RNN inference or ray tracing.
In essence, FP32 powers modern computing from HPC CFD to FFT-accelerated SDR, feeding SerDes 400G networks while EMI-shielded ENIG GPUs crunch Bluetooth beamforming on LED-lit racks.