o "j#@sdddlZddlmZddlmZddlmZddlmZddZ ddZ d d Z  dd dZ dS)N) framework) data_feeder)_get_global_groupcCsD|durtn|}|r|j||S|j|||}|r ||S)N)r process_groupZ%all_gather_into_tensor_on_calc_streamZall_gather_into_tensorwait)Z out_tensorZ in_tensorgroupsync_opuse_calc_streamtaskr s/var/www/html/Deteccion_Ine/venv/lib/python3.10/site-packages/paddle/distributed/communication/stream/all_gather.py"_all_gather_into_tensor_in_dygraphsr csl|durtn|}t|dkr|fddt|jD7}|r&|j|S|j||}|r4||S)Nrcsg|]}tqSr )paddleZ empty_like).0_tensorr r 0sz*_all_gather_in_dygraph..)rlenrangenranksrZall_gather_on_calc_stream all_gatherr) tensor_listrrrr r r rr _all_gather_in_dygraph*s rc Csd}tj|fit}|j|jd}|D] }t|dgddqt|dgdd|dur2dn|j}t } |j |d|gid |gi||| d d | t |j dkrb|t|ddS|t|| ddS) NZ c_allgather)dtyper)Zfloat16Zfloat32Zfloat64Zint32Zint64boolZint8Zuint8rrrXZOut)ring_idr r)typeZinputsZoutputsattrs)rZ LayerHelperlocalsZ"create_variable_for_type_inferencerrZcheck_variable_and_dtypeiddistZget_world_sizeZ append_opclearrshapeextendrZunstacksplit) rrrrZop_typehelperoutelemrrr r r _all_gather_in_static_mode>s>   r*TFcCs|dur |s td|s|rtdtr-t|r%t|||||St|||||S|dus5Jdt|r>tdt||||S)aA Gather tensors across devices to a correctly-sized tensor or a tensor list. Args: tensor_or_tensor_list (Union[Tensor, List[Tensor]]): The output. If it is a tensor, it should be correctly-sized. If it is a list, it should be empty or contain correctly-sized tensors. tensor (Tensor): The input tensor on each rank. The result will overwrite this tenor after communication. Support float16, float32, float64, int32, int64, int8, uint8 or bool as the input data type. group (Group, optional): Communicate in which group. If none is given, use the global group as default. sync_op (bool, optional): Indicate whether the communication is sync or not. If none is given, use true as default. use_calc_stream (bool, optional): Indicate whether the communication is done on calculation stream. If none is given, use false as default. This option is designed for high performance demand, be careful to turn it on except you are clearly know its meaning. Returns: Return a task object. Warning: This API only supports the dygraph mode now. Examples: .. code-block:: python >>> # doctest: +REQUIRES(env: DISTRIBUTED) >>> import paddle >>> import paddle.distributed as dist >>> dist.init_parallel_env() >>> local_rank = dist.get_rank() >>> tensor_list = [] >>> if local_rank == 0: ... data = paddle.to_tensor([[4, 5, 6], [4, 5, 6]]) >>> else: ... data = paddle.to_tensor([[1, 2, 3], [1, 2, 3]]) >>> task = dist.stream.all_gather(tensor_list, data, sync_op=False) >>> task.wait() >>> print(tensor_list) [[[4, 5, 6], [4, 5, 6]], [[1, 2, 3], [1, 2, 3]]] (2 GPUs) NzjThe group should not be None and all ranks which invoke this operation should be the member of this group.z5use_calc_stream can only be true in sync op behavior.z3Group can not be used in static graph mode for now.zLOnly support passing a tensor list to `all_gather` in static graph mode now.) Z is_member RuntimeErrorrZin_dynamic_moderZ is_tensorr rr*)Ztensor_or_tensor_listrrrr r r r rvs4.     r)NTF) rZpaddle.distributed distributedr"rZ paddle.baserZ&paddle.distributed.communication.grouprr rr*rr r r r s    ;