o "j!@s<ddlZddlZddlZddlmZmZGdddeZdS)N) ControleMode Controllerc@s0eZdZeddZddZddZddZd S) PSControllercCsd|jjtjks|jjst|jjdks|jjst|jjdkr0|j |j dtj|j_dSdS)Nrz enabledTF) argsZrun_moderZPS server_numlenservers trainer_numtrainersloggerdebug__name__)clsctxri/var/www/html/Deteccion_Ine/venv/lib/python3.10/site-packages/paddle/distributed/launch/controllers/ps.pyenables zPSController.enablecCs,|jjjr|jjjr|dS|dS)N)rrr r _build_pod_with_args_build_pod_with_masterselfrrr build_pod&s  zPSController.build_podc sd|jjjvr dn|jjjt|jjjd}t|jjjd}fdd|jjjdD}fdd|jjjdD}t|}t|}|||j _ | ||gddl }| }tj|rit||jjj} d|ddd| } d || |jjjd } t|D]0} |jjj|jjj|| dd d d t||jjjd } | | d| }|j| |dqd}|D] }|rn|d 7}qt|D]4} d|d||| dd dd| |t||jjjd} | | d| }|j| |dqdS)Nz 127.0.0.1,cg|] }|r|qSr startswith.0shostrr 4  z5PSController._build_pod_with_args..crrrrr rrr"7r#r{}:{}:3ZPADDLE_GLOO_RENDEZVOUSZPADDLE_GLOO_FS_PATHZPADDLE_GLOO_HTTP_ENDPOINTZPADDLE_WITH_GLOOrPSERVER)PADDLE_PSERVERS_IP_PORT_LISTPADDLE_TRAINER_ENDPOINTS PADDLE_PORT PADDLE_ROLE TRAINING_ROLEPADDLE_TRAINERS_NUMPOD_IP serverlog.Zenvslog_fileTRAINER)r)r*r+r,r-PADDLE_TRAINER_IDr.r/ workerlog.)rrr nodeiplistsplitr rpodreplicas save_pod_logtempfilemkdtempospathexistsshutilrmtree gloo_portformat with_gloorangeupdate add_containerrjoin)rserver_endpointstrainer_endpointsr r rr r=gloo_rendezvous_dirrD gloo_http _gloo_envsier2trainer_rank_offsetrrr rr,sr             z!PSController._build_pod_with_argsc stjjjj_jjjpd}fddjj|D}jjjp$d}fddjj|D}t jj jj||jjj j jjd}jdjjdjj |jjjj\}}jjd|d d|D}|d d|D}d d|D} td d|d|D} td d|d|D} |j_||j_ddl} | } tj| rt| |dd}d|d dd|}d| |jjj!d}t"|D]6}jjd#|d#| |||  ddddt$| jjj%d}|&|d|}j'||dqt"|D];}jjd#|d#| | ||  dddd|| t$| jjj%d }|&|d|}j'||dqdS)Nrc g|] }jjjd|qSr%rr6r7rprrrr"yz7PSController._build_pod_with_master..crSrTrUrVrrrr"~rX)namerankr r dtyperD/z/infozsync peers done cSsg|]}t|qSr)jsonloadsrrPrrrr"scSg|] }|dD]}|qqSr rrrPjrrrr"cSr`r rrbrrrr"rdcSg|]}t|dqSrarr_rrrr"cSrfrergr_rrrr"rhrrDr$r%r&r'rr() PADDLE_NNODESr)r*r+r,r-r.r/r0r1r3) rir)r*r+r,r-r4r.r/r5)(intrrrZr:rr6Zget_free_portsr r]dumpsrYZdevicer[Z get_free_portZmasterZ sync_peersZjobidr;r r r<sumr=r>r?r@rArBrCrEr9rFrGrJrr7rHrI)rrr r r dataZ peer_listrZrKrLZserver_rank_offsetrRr=rMrDrNrOrPrQr2rrrrus                  z#PSController._build_pod_with_masterN)r __module__ __qualname__ classmethodrrrrrrrrrs   Ir)r]r?rB controllerrrrrrrrs