NVIDIA / TransformerEngine

A library for accelerating Transformer models on NVIDIA GPUs, including using 8-bit and 4-bit floating point (FP8 and FP4) precision on Hopper, Ada and Blackwell GPUs, to provide better performance with lower memory utilization in both training and inference.

3,219 665 Language: Python License: Apache-2.0 Updated: 8h ago

cuda deep-learning fp4 fp8 gpu jax machine-learning python pytorch