o #j>.@sddlZddlZddlmZddlmZmZmZddlmZm Z m Z ddl m Z m Z gZGdd d e ZGd d d eZGd d d eZdS)N)_C_ops)core framework unique_name)_current_expected_placein_dygraph_mode in_pir_mode) Initializercalculate_gaincs6eZdZdZ     d fdd Zd dd ZZS) MSRAInitializera]Implements the MSRA initializer a.k.a. Kaiming Initializer This class implements the weight initialization from the paper `Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification `_ by Kaiming He, Xiangyu Zhang, Shaoqing Ren and Jian Sun. This is a robust initialization method that particularly considers the rectifier nonlinearities. In case of Uniform distribution, the range is [-x, x], where .. math:: x = gain \times \sqrt{\frac{3}{fan\_in}} In case of Normal distribution, the mean is 0 and the standard deviation is .. math:: \frac{gain}{\sqrt{{fan\_in}}} Args: uniform (bool, optional): whether to use uniform or normal distribution. Default is True. fan_in (float32|None, optional): fan_in (in_features) of trainable Tensor, If None, it will be infered automaticly. If you don't want to use in_features of the Tensor, you can set the value of 'fan_in' smartly by yourself. Default is None. seed (int32, optional): random seed. Default is 0. negative_slope (float, optional): negative_slope (only used with leaky_relu). Default is 0.0. nonlinearity(str, optional): the non-linear function. Default is relu. Note: It is recommended to set fan_in to None for most cases. TNrrelucsD|dusJ|dus Jt||_||_||_||_||_dS)zConstructor for MSRAInitializerN)super__init___uniform_fan_in_seed_negative_slope _nonlinearity)selfuniformfan_inseednegative_slope nonlinearity __class__^/var/www/html/Deteccion_Ine/venv/lib/python3.10/site-packages/paddle/nn/initializer/kaiming.pyrAs   zMSRAInitializer.__init__c Cs||}t|tjtjjjfsJt|tjtjjfsJ| |\}}|j dur,|n|j }|j dkr9|j j |_ |jtjjjksL|jtjjjkrj|jsjtjjj}|jtdd|jdg|j|tjjjdd}n|jtjjtjjfvr|jstjj}|}n|j}|}t r|jrt!|j"|j#}|t$%dt&|} t'(|j|| | |j t)}nt!|j"|j#}|t$%t&|} t)} t'*|jd | |j || }|jtjjjks|jtjjjkr|jst'+||j} | ,|dS|,|dSt-rS|jrt!|j"|j#}|t$%dt&|} t'(|j|| | |j t)}nt!|j"|j#}|t$%t&|} t)} t'*|jd | |j || }|jtjjtjjfvrQ|jsQt'+||jS|S|jrt!|j"|j#}|t$%dt&|} |j.d id |i|jt/|| | |j d d d} n%t!|j"|j#}|t$%t&|} |j.dd |i|jt/|d | |j dd d} |jtjjjks|jtjjjkr|js|j.dd|id |i|j|jdd| |_0| S)aWInitialize the input tensor with MSRA initialization. Args: var(Tensor): Tensor that needs to be initialized. block(Block, optional): The block in which initialization ops should be added. Used in static graph only, default None. Returns: The initialization op. Nr.Z masra_inittmpF)nameshapedtypetypeZ persistableg@Zuniform_randomZOut)r#r$minmaxrT)r%inputsoutputsattrs stop_gradientZgaussian_random)r#r$meanstdr)r%r*r+r,castX)Zin_dtype out_dtype)r%r)r*r+)1Z _check_block isinstancerVariablepaddleZpirrZ ParameterMetaZBlockZ _compute_fansrrprogramZ random_seedr$ZVarDescZVarTypeZFP16ZBF16rZFP32Z create_varrgeneratejoinr"r#Z LOD_TENSORZDataTypeZFLOAT16ZBFLOAT16ZFLOAT32rr rrmathsqrtfloatrrrZgaussianr/Z_share_underline_tensor_tor Z append_opintop)rvarblockZf_inZf_outrr1Zout_varZgainlimitr.ZplaceZvar_tmpr<rrrforwardSs           zMSRAInitializer.forward)TNrrr)N)__name__ __module__ __qualname____doc__rr@ __classcell__rrrrr s"r c"eZdZdZdfdd ZZS) KaimingNormala_Implements the Kaiming Normal initializer This class implements the weight initialization from the paper `Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification `_ by Kaiming He, Xiangyu Zhang, Shaoqing Ren and Jian Sun. This is a robust initialization method that particularly considers the rectifier nonlinearities. In case of Normal distribution, the mean is 0 and the standard deviation is .. math:: \frac{gain}{\sqrt{{fan\_in}}} Args: fan_in (float32|None, optional): fan_in (in_features) of trainable Tensor, If None, it will be infered automaticly. If you don't want to use in_features of the Tensor, you can set the value of 'fan_in' smartly by yourself. Default is None. negative_slope (float, optional): negative_slope (only used with leaky_relu). Default is 0.0. nonlinearity(str, optional): the non-linear function. Default is relu. Note: It is recommended to set fan_in to None for most cases. Examples: .. code-block:: python >>> import paddle >>> import paddle.nn as nn >>> linear = nn.Linear(2, 4, weight_attr=nn.initializer.KaimingNormal()) >>> data = paddle.rand([30, 10, 2], dtype='float32') >>> res = linear(data) Nr&rctjd|d||ddS)NFrrrrrrrrrrrrrrrr zKaimingNormal.__init__Nr&rrArBrCrDrrErrrrrGs$rGcrF)KaimingUniformaYImplements the Kaiming Uniform initializer This class implements the weight initialization from the paper `Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification `_ by Kaiming He, Xiangyu Zhang, Shaoqing Ren and Jian Sun. This is a robust initialization method that particularly considers the rectifier nonlinearities. In case of Uniform distribution, the range is [-x, x], where .. math:: x = gain \times \sqrt{\frac{3}{fan\_in}} Args: fan_in (float32|None, optional): fan_in (in_features) of trainable Tensor, If None, it will be infered automaticly. If you don't want to use in_features of the Tensor, you can set the value of 'fan_in' smartly by yourself. Default is None. negative_slope (float, optional): negative_slope (only used with leaky_relu). Default is 0.0. nonlinearity(str, optional): the non-linear function. Default is relu. Note: It is recommended to set fan_in to None for most cases. Examples: .. code-block:: python >>> import paddle >>> import paddle.nn as nn >>> linear = nn.Linear(2, 4, weight_attr=nn.initializer.KaimingUniform()) >>> data = paddle.rand([30, 10, 2], dtype='float32') >>> res = linear(data) Nr&rcrH)NTrrIrJrKrrrr=rLzKaimingUniform.__init__rMrNrrrrrOs#rO)r8r4rbaserrrZbase.frameworkrrr Z initializerr r __all__r rGrOrrrrs K/