accelerate_config_moe.yaml配置修改如下,其他没有修改,直接运行样例脚本lingbot-video-moe-30b-a3b_ti2v.sh,没有并行起来 compute_environment: LOCAL_MACHINE debug: false deepspeed_config: gradient_accumulation_steps: 1 offload_optimizer_device: none offload_param_device: none zero3_init_flag: true zero_stage: 3 zero3_save_16bit_model: true distributed_type: DEEPSPEED downcast_bf16: 'no' enable_cpu_affinity: true machine_rank: 0 main_training_function: main mixed_precision: bf16 num_machines: 4 num_processes: 32 main_process_ip: 192.168.16.46 main_process_port: 29500 rdzv_backend: static same_network: true tpu_env: [] tpu_use_cluster: false tpu_use_sudo: false use_cpu: false
Activity
Artiprocher commentedon Oct 14, 2026
@chenhangcal 多机训练的配置比较繁琐,请确认:
accelerate config进行配置Artiprocher commentedon Oct 14, 2026
@chenhangcal
chenhangcal commentedon Oct 14, 2026
用accelerate config配置了,感觉是accelerate的多机配置没有传递到DeepSpeed