o #jv2@sddlZddlmZddlmZmZmZddlmZddl m Z m Z m Z ddl mZgZGd d d eZGd d d eZGd ddeZdS)N)_C_ops)core framework unique_name)check_variable_and_dtype)_current_expected_placein_dygraph_mode in_pir_mode) Initializercs,eZdZdZd fdd Zd ddZZS) XavierInitializera This class implements the Xavier weight initializer from the paper `Understanding the difficulty of training deep feedforward neural networks `_ by Xavier Glorot and Yoshua Bengio. This initializer is designed to keep the scale of the gradients approximately same in all the layers. In case of Uniform distribution, the range is [-x, x], where .. math:: x = \sqrt{\\frac{6.0}{fan\_in + fan\_out}} In case of Normal distribution, the mean is 0 and the standard deviation is .. math:: \sqrt{\\frac{2.0}{fan\_in + fan\_out}} Args: uniform (bool, optional): whether to use uniform ,if False use normal distribution. Default is True. fan_in (float, optional): fan_in for Xavier initialization. If None, it is inferred from the variable. Default is None. fan_out (float, optional): fan_out for Xavier initialization. If None, it is inferred from the variable. Default is None. seed (int, optional): Random seed. Default is 0. Note: It is recommended to set fan_in and fan_out to None for most cases. TNrcs>|dusJ|dus Jt||_||_||_||_dSN)super__init___uniform_fan_in_fan_out_seed)selfuniformfan_infan_outseed __class__]/var/www/html/Deteccion_Ine/venv/lib/python3.10/site-packages/paddle/nn/initializer/xavier.pyrCs    zXavierInitializer.__init__c Cszddl}||}t|tj|jjfsJt||jjjs&t|dgdd| |\}}|j dur4|n|j }|j dur>|n|j }|j dkrK|j j|_ |jtjjjks^|jtjjjkr||js|tjjj}|jtdd|jdg|j|tjjjdd } n|jtjjtjjfvr|jstjj }|} n|j}|} t!r|jrt"#d t$||} t%&| j|| | |j t'} nt"#d t$||} t'} t%(| jd | |j || } |jtjjjks|jtjjjkr|jst%)| |j} | *|dS| *|dSt+rL|jrt"#d t$||} |j,&| j|| | |j t'} nt"#d t$||} t%(| jd | |j |t'} |jtjjtjjfvrJ|jsJt%)| |jS| S|jrqt"#d t$||} |j-d id| i| j|| | |j ddd}nt"#d t$||} |j-dd| i| j| jd | |j ddd}|jtjjjks|jtjjjkr|js|j-dd| id|i| j|jdd||_.|S)aXInitialize the input tensor with Xavier initialization. Args: var(Tensor): Tensor that needs to be initialized. block(Block, optional): The block in which initialization ops should be added. Used in static graph only, default None. Returns: The initialization op rNZOut)Zuint16Zfloat16Zfloat32Zfloat64Z xavier_init.tmpF)nameshapedtypetypeZ persistableg@g@gZuniform_random)r!r"minmaxrT)r#inputsoutputsattrs stop_gradientZgaussian_random)r!r"meanstdr)r#r'r(r)castX)Zin_dtype out_dtype)r#r&r'r()/paddleZ _check_block isinstancerZBlockZpirrZ ParameterMetarZ _compute_fansrrrprogramZ random_seedr"ZVarDescZVarTypeZFP16ZBF16rZFP32Z create_varrgeneratejoinr r!Z LOD_TENSORZDataTypeZFLOAT16ZBFLOAT16ZFLOAT32r mathsqrtfloatrrrZgaussianr,Z_share_underline_tensor_tor Z_pir_opsZ append_opop)rvarblockr/Zf_inZf_outrrr.Zout_varlimitr+ZplaceZvar_tmpr7rrrforwardLs            zXavierInitializer.forward)TNNrr)__name__ __module__ __qualname____doc__rr; __classcell__rrrrr s# r c"eZdZdZdfdd ZZS) XavierNormala This class implements the Xavier weight initializer from the paper `Understanding the difficulty of training deep feedforward neural networks `_ by Xavier Glorot and Yoshua Bengio, using a normal distribution whose mean is :math:`0` and standard deviation is .. math:: \sqrt{\frac{2.0}{fan\_in + fan\_out}}. Args: fan_in (float, optional): fan_in for Xavier initialization, which is inferred from the Tensor. Default is None. fan_out (float, optional): fan_out for Xavier initialization, which is inferred from the Tensor. Default is None. name (str, optional): For details, please refer to :ref:`api_guide_Name`. Generally, no setting is required. Default: None. Returns: A parameter initialized by Xavier weight, using a normal distribution. Examples: .. code-block:: python >>> import paddle >>> paddle.seed(1) >>> data = paddle.ones(shape=[3, 1, 2], dtype='float32') >>> weight_attr = paddle.framework.ParamAttr( ... name="linear_weight", ... initializer=paddle.nn.initializer.XavierNormal()) >>> bias_attr = paddle.framework.ParamAttr( ... name="linear_bias", ... initializer=paddle.nn.initializer.XavierNormal()) >>> linear = paddle.nn.Linear(2, 2, weight_attr=weight_attr, bias_attr=bias_attr) >>> print(linear.weight) Parameter containing: Tensor(shape=[2, 2], dtype=float32, place=Place(cpu), stop_gradient=False, [[-0.21607460, 0.08382989], [ 0.29147008, -0.07049121]]) >>> print(linear.bias) Parameter containing: Tensor(shape=[2], dtype=float32, place=Place(cpu), stop_gradient=False, [1.06076419, 0.87684733]) >>> res = linear(data) >>> print(res) Tensor(shape=[3, 1, 2], dtype=float32, place=Place(cpu), stop_gradient=False, [[[1.13615966, 0.89018601]], [[1.13615966, 0.89018601]], [[1.13615966, 0.89018601]]]) Nctjd||dddS)NFrrrrrrrrrrr rrrrzXavierNormal.__init__NNNr<r=r>r?rr@rrrrrBs5rBcrA) XavierUniforma+ This class implements the Xavier weight initializer from the paper `Understanding the difficulty of training deep feedforward neural networks `_ by Xavier Glorot and Yoshua Bengio. This initializer is designed to keep the scale of the gradients approximately same in all the layers. In case of Uniform distribution, the range is :math:`[-x,x]`, where .. math:: x = \sqrt{\frac{6.0}{fan\_in + fan\_out}}. Args: fan_in (float, optional): fan_in for Xavier initialization, which is inferred from the Tensor. Default is None. fan_out (float, optional): fan_out for Xavier initialization, which is inferred from the Tensor. Default is None. name (str, optional): For details, please refer to :ref:`api_guide_Name`. Generally, no setting is required. Default: None. Returns: A parameter initialized by Xavier weight, using a uniform distribution. Examples: .. code-block:: python >>> import paddle >>> paddle.seed(1) >>> data = paddle.ones(shape=[3, 1, 2], dtype='float32') >>> weight_attr = paddle.framework.ParamAttr( ... name="linear_weight", ... initializer=paddle.nn.initializer.XavierUniform()) >>> bias_attr = paddle.framework.ParamAttr( ... name="linear_bias", ... initializer=paddle.nn.initializer.XavierUniform()) >>> linear = paddle.nn.Linear(2, 2, weight_attr=weight_attr, bias_attr=bias_attr) >>> print(linear.weight) Parameter containing: Tensor(shape=[2, 2], dtype=float32, place=Place(cpu), stop_gradient=False, [[-1.18095720, 0.64892638], [ 0.43125069, -1.11156428]]) >>> print(linear.bias) Parameter containing: Tensor(shape=[2], dtype=float32, place=Place(cpu), stop_gradient=False, [-0.27524316, 1.13808715]) >>> res = linear(data) >>> print(res) Tensor(shape=[3, 1, 2], dtype=float32, place=Place(cpu), stop_gradient=False, [[[-1.02494967, 0.67544925]], [[-1.02494967, 0.67544925]], [[-1.02494967, 0.67544925]]]) NcrC)NTrrDrErFrrrr[rGzXavierUniform.__init__rHrIrrrrrJ#s7rJ)r4r/rbaserrrZbase.data_feederrZbase.frameworkrr r Z initializerr __all__r rBrJrrrrs   K: