o "j!@sddlZddlZddlZddlmZddlmZddlmZddl m Z ddl m Z m Z gZdZGdd d ZeZd d Zdd d ZddZ      dddZdS)N) _legacy_C_ops)core)check_variable_and_dtype)Variable) LayerHelperin_dynamic_modeZmodel_parallel_rngc@sJeZdZdZddZddZddZdd Zd d Ze j e fd d Z dS)RNGStatesTrackerz! Tracker the RNG states. cCi|_t|_dSNstates_setseeds_selfrk/var/www/html/Deteccion_Ine/venv/lib/python3.10/site-packages/paddle/distributed/fleet/layers/mpu/random.py__init__'s zRNGStatesTracker.__init__cCr r r rrrrreset,s zRNGStatesTracker.resetcCsn||jvr td|d|j|||jvr td|dt}t|t|j|<t|dS)Nzseed z already existsstate )r ValueErroraddr paddle get_rng_stateseed set_rng_state)rnamerorig_rng_staterrrr0s    zRNGStatesTracker.addcCs"i}|jD] }|j|||<q|Sr r )rstatesrrrrget_states_tracker;s z#RNGStatesTracker.get_states_trackercCs ||_dSr r)rrrrrset_states_trackerAs z#RNGStatesTracker.set_states_trackerc cst||jvrtd|dt}t|j|zdVWt|j|<t|dSt|j|<t|w)Nrz does not exist)r rrrr)rrrrrr rng_stateDs  zRNGStatesTracker.rng_stateN) __name__ __module__ __qualname____doc__rrrr r! contextlibcontextmanagerMODEL_PARALLEL_RNGr"rrrrr"s rcCstSr )RNG_STATE_TRACKERrrrrget_rng_state_trackerTsr+c Csddlm}|}|}|}|}|}|r)|}|d|||}ntj dd}|d|||}t t t |t|dS)Nr)fleeti')Zpaddle.distributedr,Zget_hybrid_communicate_groupZget_model_parallel_rankZget_model_parallel_world_sizeZ get_stage_idZget_pipe_parallel_world_sizenprandomrandintr*rrr)rr) rr,ZhcgZmp_rankZmp_sizeZpp_rankZpp_sizeZ global_seedZ local_seedrrrmodel_parallel_random_seedXs  r1cCsR|dur|dks Jtdit}|jtjd}|jdd|id|ddd|S) NrdtypeOutT)Z deterministicrng_nameZ force_cpu)typeoutputsattrs)r)rlocals"create_variable_for_type_inferencerZint32 append_op)r6helperoutrrrdeterminate_seedps r??Tupscale_in_trainc Cs~|durtjj||||||St|tttfstdt|ttfr)|dkr)|Sd|kr3dks:nJt d|dvsDJt d|dusNJtd|d krTd n|}t rmt |d |d | d dddd| \}}|St |} t|tr|j dgkstd|j tdit} t|dgdd| j|jd}| jtjjjdd}| jd|g| d|g|gd|| |dd|S)a Dropout is a regularization technique for reducing overfitting by preventing neuron co-adaption during training. The dropout operator randomly sets the outputs of some units to zero, while upscale others according to the given dropout probability. Args: x (Tensor): The input tensor. The data type is float32 or float64. p (float|int): Probability of setting units to zero. Default 0.5. axis (int|list|tuple): The axis along which the dropout is performed. Default None. rng_name (str): The random seed generator name, which used to obtain deterministic results. training (bool): A flag indicating whether it is in train phrase or not. Default True. mode(str): ['upscale_in_train'(default) | 'downscale_in_infer']. 1. upscale_in_train(default), upscale the output at training time - train: out = input * mask / ( 1.0 - dropout_prob ) - inference: out = input 2. downscale_in_infer, downscale the output at inference - train: out = input * mask - inference: out = input * (1.0 - dropout_prob) name (str, optional): Name for the operation (optional, default is None). For more information, please refer to :ref:`api_guide_Name`. Returns: A Tensor representing the dropout, has same shape and data type as `x` . Examples: We use ``p=0.5`` in the following description for simplicity. 1. When ``axis=None`` , this is commonly used dropout, which dropout each element of x randomly. .. code-block:: text Let's see a simple case when x is a 2d tensor with shape 2*3: [[1 2 3] [4 5 6]] we generate mask with the same shape as x, which is 2*3. The value of mask is sampled from a Bernoulli distribution randomly. For example, we may get such mask: [[0 1 0] [1 0 1]] So the output is obtained from elementwise multiply of x and mask: [[0 2 0] [4 0 6]] Using default setting, i.e. ``mode='upscale_in_train'`` , if in training phase, the final upscale output is: [[0 4 0 ] [8 0 12]] if in test phase, the output is the same as input: [[1 2 3] [4 5 6]] we can also set ``mode='downscale_in_infer'`` , then if in training phase, the final output is: [[0 2 0] [4 0 6]] if in test phase, the scale output is: [[0.5 1. 1.5] [2. 2.5 3. ]] Nz4p argument should be a number(int|float) or Variablerr-z!p argument should between 0 and 1)downscale_in_inferrAzBmode argument should be 'downscale_in_infer' or 'upscale_in_train'z/unsupport axis when using random seed generatorrBZdowngrade_in_infer dropout_probis_testZfix_seedFrdropout_implementationzGRequired p.shape == [1] if type(p) is Variable, but received p.shape = dropoutx)Zfloat16Zfloat32Zfloat64r3T)r4Z stop_gradient)XZSeed)r5Mask)rCrDrE)r7Zinputsr8r9)rF)rnnZ functionalrF isinstancefloatintr TypeErrorrrrr?shaperr:rr;r4rZVarDescZVarTypeZUINT8r<) rGpZaxisr6Ztrainingmoderr>maskrr=rrrrF}sjG"         rFr )r@NNTrAN)r'numpyr.rrZ paddle.baserZpaddle.base.data_feederrZpaddle.common_ops_importrZpaddle.frameworkrr__all__r)rr*r+r1r?rFrrrrs,    /