Nonuniform-Tensor-Parallelism: Mitigating GPU failure impact for Scaled-up LLM Training.

Tue, 01 Apr 2025 01:00:00 +0200

Daiyaan Arfeen, Dheevatsa Mudigere, Ankit More, Bhargava Gopireddy, Ahmet Inci, Gregory R. Ganger:
Nonuniform-Tensor-Parallelism: Mitigating GPU failure impact for Scaled-up LLM Training. CoRR abs/2504.06095 (2025)