add support for port 0 auto-selection in multi-GPU environments by hellobiondi · Pull Request #3501 · huggingface/accelerate

What does this PR do?

This PR implements support for port 0 auto-selection in multi-GPU environments (prepare_multi_gpu_env()). The documentation already mentions that setting port to 0 will automatically select the next available port, but this functionality wasn't actually implemented in the code.

When main_process_port is set to 0, the code now:

  • Automatically finds an available port through socket binding
  • Updates relevant arguments (master_port, rdzv_endpoint) with the selected port
  • Provides a more seamless experience for users working in environments where specific ports might be occupied

Before submitting

Who can review?

@SunMarc @zach-huggingface - This relates to the Command Line Interface and distributed training functionality.