Russian technology MNC Yandex has introduced YaFSDP, an open-source tool designed to improve the efficiency of training LLMs. This method improves GPU communication and reduces memory usage, offering a speedup of up to 26% over existing tools.
YaFSDP outperforms the traditional FSDP method, showing improvements in training speed, especially for large models. For example, YaFSDP achieved a 21% speedup on Llama 2 with 70 billion parameters and a 26% speedup on Llama 3 with the same number of parameters. These enhancements make YaFSDP a valuable tool for AI developers working with large, complex models.
By optimising GPU consumption, YaFSDP can save developers and companies significant amounts of money—potentially hundreds of thousands of dollars monthly.
“Currently, we’re actively experimenting with various model architectures and parameter sizes to expand YaFSDP’s versatility,” said Mikhail Khruschev, senior developer at Yandex and part of the team behind YaFSDP. “
The open-source model is available on GitHub.
Benefits and Implementation of YaFSDP
…