o "j'@sddlZddlZddlmZddlmZGdddZGdddZddZd d Z d d Z d dZ ddZ ddZ d"ddZd#ddZddZd#ddZd$ddZd"ddZd"d d!ZdS)%N) frameworkc@seZdZdZdddZeddZeddZed d Zed d Z ed dZ eddZ eddZ eddZ ddZddZddZdS)Groupz/ The abstract representation of group. NcCs8||_|dkr t|nd|_||_||_||_||_dS)Nr)_rank_in_grouplen _world_size_id_ranks_pg_name)selfZ rank_in_groupidranksZpgnamerg/var/www/html/Deteccion_Ine/venv/lib/python3.10/site-packages/paddle/distributed/communication/group.py__init__s  zGroup.__init__cC|jSN)rr rrrrank#z Group.rankcCrr)r rrrrr'rz Group.rankscCs t|jSr)rr rrrrnranks+ z Group.nrankscCrr)r rrrrr/rz Group.namecCrr)r rrrr process_group3rzGroup.process_groupcCrr)rrrrr world_size7rzGroup.world_sizecCs |jSr)r rrrrrbackend;rz Group.backendcCrr)rrrrrr ?rzGroup.idcCs |jdkrdS|jdkrdSdS)NrFT)rrrrrr is_memberCs  zGroup.is_membercCs|r |j|SdS)Nr)rrindex)r rrrrget_group_rankJs zGroup.get_group_rankcCsZd|jd|jd|jd}|dtt|j7}|d7}||jr(|j7}|Sd7}|S)Nzrank: z , nranks: z, id: z , ranks: z, z; name: None)rrr joinmapstrrr)r Z debug_strrrr__repr__PszGroup.__repr__)NN)__name__ __module__ __qualname____doc__rpropertyrrrrrrrr rr r%rrrrrs,          rc@seZdZdZiZdS) _GroupManagerrN)r&r'r(global_group_idgroup_map_by_idrrrrr+Zsr+cCs tjtjvr tdtjtjS)Nz$The global group is not initialized.)r+r,r- RuntimeErrorrrrr_get_global_group_s  r/cCs.|jtjvrtd|jd|tj|j<dS)NzThe group with id z already exist.)r r+r-r.grouprrr_add_new_groupes r2cCs |jtjkSr)r r+r,r0rrr_is_global_groupks r3cCs4t}|r|std|d|jdSdS)NzCurrent global rank z is not in group TF)distZget_rankrwarningswarnr)r1 global_rankrrr_warn_cur_rank_not_in_groupos r8cCs,||}|dksJd|d|j|S)NrzThe input rank z# can not be found inside the group )r r)r7r1Z group_rankrrr_get_or_throw_group_rankys  r9cCs tjtjvS)a@ Check whether the distributed environment has been initialized Returns: `True` if distributed environment has been initialized, otherwise `False`. Warning: This API only supports the dygraph mode. Examples: .. code-block:: python >>> # doctest: +REQUIRES(env: DISTRIBUTED) >>> import paddle >>> print(paddle.distributed.is_initialized()) False >>> paddle.distributed.init_parallel_env() >>> print(paddle.distributed.is_initialized()) True )r+r,r-rrrris_initializeds r:cCsT|durtn|}|jtjvsJd|jdt|r#tjdStj|j=dS)a Destroy a given group for communication Args: group (Group, optional): The group to be destroyed. All of process groups, including the default group, will be destroyed and the distributed environment will be deinitialized. Returns : None Warning: This API only supports the dygraph mode. Examples: .. code-block:: python >>> # doctest: +REQUIRES(env: DISTRIBUTED) >>> import paddle >>> import paddle.distributed as dist >>> dist.init_parallel_env() >>> group = dist.new_group([0, 1]) >>> dist.destroy_process_group(group) >>> print(dist.is_initialized()) True >>> dist.destroy_process_group() >>> print(dist.is_initialized()) False NzDestroy group with id z is invalid.)r/r r+r-r3clearr0rrrdestroy_process_groups  r<cCs*|tjvr tj|Std|ddS)a Get group instance by group id. Args: id (int): the group id. Default value is 0. Returns: Group: the group instance. Examples: .. code-block:: python >>> # doctest: +REQUIRES(env: DISTRIBUTED) >>> import paddle >>> import paddle.distributed as dist >>> dist.init_parallel_env() >>> gid = paddle.distributed.new_group([2,4,6]) >>> paddle.distributed.get_group(gid.id) zGroup z is not initialized.N)r+r-r5r6)r rrr get_groups  r=cCsNtr tj||Sd}tj|fit}|j|d|gid|giddS)Nc_sync_calc_streamXOut)typeinputsoutputs)rin_dynamic_modepaddle _legacy_C_opsr> LayerHelperlocals append_op)tensorop_typehelperrrr_sync_calc_streams rMcCs\trtj|g|gd|Sd}tj|fit}|j|d|gid|gid|iddS)Nring_idc_sync_comm_streamr?r@rArBrCattrs)rrDrErFrOrGrHrI)rJrNrKrLrrr_sync_comm_streams  rRTcCsD|dur |s dS|rt|dS|durdn|j}t||dS)a wait to sync stream for group. Args: tensor (Tensor): The Tensor used before sync. group (Group): The Group instance to perform sync. use_calc_stream (bool): Wether to use calculation stream (True) or communication stream (False). Default to True. Returns: None. Examples: .. code-block:: python >>> # doctest: +REQUIRES(env: DISTRIBUTED) >>> import paddle >>> paddle.distributed.init_parallel_env() >>> tindata = paddle.randn(shape=[2, 3]) >>> paddle.distributed.all_reduce(tindata, sync_op=True) >>> paddle.distributed.wait(tindata) Nr)rrMr rR)rJr1Zuse_calc_streamrNrrrwaits  rScCs|dur |s dStr7|durtn|}t}t|tjr'|j}n | }|j|}| dS|dur=dn|j }t j dgddd}trVt j||d|Sd}t|tsatdtj|fit}|j|d |gid |gid|id dS) a* Barrier among all participators in the group. Args: group (Group): The group instance return by new_group or None for global default group. Returns: None. Examples: .. code-block:: python >>> # doctest: +REQUIRES(env: DISTRIBUTED) >>> import paddle >>> from paddle.distributed import init_parallel_env >>> paddle.set_device('gpu:%d'%paddle.distributed.ParallelEnv().dev_id) >>> init_parallel_env() >>> paddle.distributed.barrier() NrZint32)ZdtyperNbarrierz,The type of 'group' for barrier must be int.r?r@rP)rrrDr/Z_current_expected_place isinstanceZCPUPlacerrUZ get_device_idrSr rEfullrFint ValueErrorrGrHrI)r1ZplacetaskZ device_idrNZbarrier_tensorrKrLrrrrU&s6     rUcCs.t|rtd|durt}|jS|}|jS)a Get the backend of given group. Args: group (Group): The group to work on. Use the global group as default. Returns: Returns the name of the given group backend. Examples: .. code-block:: python >>> # doctest: +REQUIRES(env: DISTRIBUTED) >>> import paddle >>> paddle.distributed.init_parallel_env() >>> paddle.distributed.get_backend() NCCL zInvalid group specifiedN)r8r.r/rr0rrr get_backend^s r[r)r)NT)r5rEZpaddle.distributed distributedr4rrr+r/r2r3r8r9r:r<r=rMrRrSrUr[rrrrs&  D   *  $8