Home Original page

GitHub - volcengine/veScale: Byted PyTorch Distributed for Hyperscale Training of LLMs and RLs

Old veScale Has been Moved to legacy/; New veScale is Coming ...

veScale is an internal PyTorch Distributed library, enabling hyperscale distributed training of LLMs and RLs. This repo open-sources a small piece of veScale for a better community.

@misc{li2025vescale,
      title={veScale: Consistent and Efficient Tensor Programming with Eager-Mode SPMD}, 
      author={Youjie Li and Cheng Wan and Zhiqi Lin and Hongyu Zhu and Jiacheng Yang and Ziang Song and Xinyi Di and Jiawei Wu and Huiyao Shu and Wenlei Bao and Yanghua Peng and Haibin Lin and Li-Wen Chang},
      year={2025},
      eprint={2509.07003},
      archivePrefix={arXiv},
      primaryClass={cs.PL},
      url={https://arxiv.org/abs/2509.07003}, 
}

The veScale Project is under the Apache License v2.0.