o )j*%@sddlZddlmZddlmmZddlmZdZGdddejZGdddej Z Gdd d ejZ Gd d d ejZ Gd d d ejZ GdddejZGdddejZdS)N)Tensor:0yE>cs*eZdZdZdfdd ZddZZS) GlobalLayerNorma?Calculate Global Layer Normalization. Args: dim : (int or list or torch.Size) Input shape from an expected input of size. eps : float A value added to the denominator for numerical stability. elementwise_affine : bool A boolean value that when set to True, this module has learnable per-element affine parameters initialized to ones (for weights) and zeros (for biases). Example: ------- >>> x = torch.randn(5, 10, 20) >>> GLN = GlobalLayerNorm(10, 3) >>> x_norm = GLN(x) rTcstt|||_||_||_|jrM|dkr-tt |jd|_ tt |jd|_ |dkrKtt |jdd|_ tt |jdd|_ dSdS| dd| dddS)Nweightbias)superr__init__dimepselementwise_affinenn ParametertorchZonesrZzerosr Zregister_parameter)selfr shaper r __class__r/var/www/html/Deteccion_Ine/venv/lib/python3.10/site-packages/modelscope/models/audio/separation/m2/conv_module.pyr s zGlobalLayerNorm.__init__cCs|dkr>> x = torch.randn(5, 10, 20) >>> CLN = CumulativeLayerNorm(10) >>> x_norm = CLN(x) Tcstt|j||dddS)Nr)rr )r r%r )rr rrrrr cs  zCumulativeLayerNorm.__init__csx|dkr |dddd}t|}|dddd}|dkr:t|dd}t|}t|dd}|S)zReturns the normalized tensor. Args: x : torch.Tensor Tensor size [N, C, K, S] or [N, C, L] rrrrr)r permute contiguousr rr transposerrrrrrgs    zCumulativeLayerNorm.forward)Trrrrrr%Ssr%cs8eZdZdZdeffdd ZdedefddZZS) Transposez; Wrapper class of torch.transpose() for Sequential module. rcstt|||_dSN)r r*r r)rrrrrr s zTranspose.__init__rreturncCs |j|jSr+)r(rr)rrrrs zTranspose.forward) r r!r"r#tupler rrr$rrrrr*sr*csXeZdZdZ   ddedededed ed ed d ffd d Zded efddZZ S)DepthwiseConv1daYWhen groups == in_channels and out_channels == K * in_channels, where K is a positive integer, this operation is termed in literature as depthwise convolution. Args: in_channels (int): Number of channels in the input out_channels (int): Number of channels produced by the convolution kernel_size (int or tuple): Size of the convolving kernel stride (int, optional): Stride of the convolution. Default: 1 padding (int or tuple, optional): Zero-padding added to both sides of the input. Default: 0 bias (bool, optional): If True, adds a learnable bias to the output. Default: True Inputs: inputs - **inputs** (batch, in_channels, time): Tensor containing input vector Returns: outputs - **outputs** (batch, out_channels, time): Tensor produces by depthwise 1-D convolution. rrF in_channels out_channels kernel_sizestridepaddingr r,Nc s@tt|||dksJdtj|||||||d|_dS)Nrz7out_channels should be constant multiple of in_channels)r/r0r1groupsr2r3r )r r.r rZConv1dconv)rr/r0r1r2r3r rrrr s  zDepthwiseConv1d.__init__inputscCs ||Sr+)r5rr6rrrrs zDepthwiseConv1d.forward)rrF) r r!r"r#intboolr rrr$rrrrr.s*r.c sPeZdZdZ   ddedededed d f fd d Zd ed efddZZ S) ConvModulea Conformer convolution module starts with a pointwise convolution and a gated linear unit (GLU). This is followed by a single 1-D depthwise convolution layer. Batchnorm is deployed just after the convolution to aid training deep models. Args: in_channels (int): Number of channels in the input kernel_size (int or tuple, optional): Size of the convolving kernel Default: 17 dropout_p (float, optional): probability of dropout Inputs: inputs inputs (batch, time, dim): Tensor contains input sequences Outputs: outputs outputs (batch, time, dim): Tensor produces by conformer convolution module. r皙?r/r1expansion_factor dropout_pr,Nc sdtt||dddksJd|dksJdttddt|||d|ddd|_dS) Nrrrz5kernel_size should be a odd number for 'SAME' paddingz+Currently, Only Supports expansion_factor 2r)r)r2r3)r r:r r Sequentialr*r. sequential)rr/r1r=r>rrrr s&  zConvModule.__init__r6cCs|||ddS)Nrr)r@r(r7rrrrszConvModule.forward)r;rr<) r r!r"r#r8floatr rrr$rrrrr:s"r:cs&eZdZdfdd ZddZZS) DilatedDenseNetr@c stt|||_||_tjddd|_|dd|_|jdf|_ t |jD]c}d|}||d|dd}t |d |dtjdd||fddt |d |dtj |j|d|j|j |df|jd d t |d |dtj|d d t |d |dt|jq'dS)N)rrrrg)valuerrpad{}rconv{}F)r1Zdilationr4r norm{}T)Zaffineprelu{})r rBr depthr/rZ ConstantPad2dpadZtwidthr1rangesetattrformatZConv2dZInstanceNorm2dZPReLU)rrJlorderr/iZdilZ pad_lengthrrrr s8    zDilatedDenseNet.__init__cCst|d}|dddd}|}t|jD];}t|d|d|}t|d|d|}t|d|d|}t|d|d|}tj||gdd }q|dddd}|dS) NrrrrrFrGrHrI)r ) rZ unsqueezer&rLrJgetattrrNcatZsqueeze)rrZx_perskiprPoutZout1rrrrs  zDilatedDenseNet.forward)rrCrD)r r!r"r rr$rrrrrBsrBcs,eZdZejdffdd ZddZZS)FFConvMDilatedr<c sBtt||t||ttdd|dt||_dS)Nrr;)rJrOr/) r r rr?ZLinearZSiLUrBZDropoutmdl)rZdim_inZdim_outZ norm_klassZdropoutrrrr s   zFFConvMDilated.__init__cCs||}|Sr+)rV)rroutputrrrrs zFFConvMDilated.forward)r r!r"r LayerNormr rr$rrrrrUsrU)rZtorch.nnrZ torch.nn.initinitrZEPSModulerrXr%r*r.r:rBrUrrrrs  H, *+)