o )j@sddlZddlZddlZddlmZmZmZddlZddlm Z ddl m Z ddl m Z ddlmZddlmZddlmZddlmZd d lmZmZd d lmZejeje jd Gd dde ZGddde jZ dS)N)DictOptionalTuple)Models) TorchModel)Tensor)MODELS) update_conf)Tasks) GlobalCMVNload_kaldi_cmvn)FSMN) module_namecspeZdZdZ     ddededed ed ed eef fd d Z ddZ de ee ffddZ ddZZS) FSMNDecoratorz? A decorator of FSMN for integrating into modelscope framework N' F model_dir cmvn_filebackbone input_dim output_dimtrainingc sZtj|g|Ri|d|_d|_|r ||||||_dS|tj|dd|_dS)ainitialize the fsmn model from the `model_dir` path. Args: model_dir (str): the model path. cmvn_file (str): cmvn file backbone (dict): params related to backbone input_dim (int): input dimension of network output_dim (int): output dimension of network training (bool): training or inference mode Nz config.yaml)Zmodel_workspaceZ config_path)super__init__model model_cfg init_modelospathjoin) selfrrrrrrargskwargs __class__l/var/www/html/Deteccion_Ine/venv/lib/python3.10/site-packages/modelscope/models/audio/kws/nearfield/model.pyrs    zFSMNDecorator.__init__cCst|dr |jdSdS)Ntmp_dir)hasattrr(cleanupr!r&r&r'__del__<s zFSMNDecorator.__del__returncCs$|jdur|dur|j|S|jS)zP Args: input (torch.Tensor): Input tensor (B, T, D) N)rforwardr)r!inputr&r&r'r.@s zFSMNDecorator.forwardc Cs|durt|\}}tt|t|}nd}d}d} |d} |d} |d} |d} |d}|d}|d}|d }|d }t|| | | | |||||| }d}d}t||||| |||}|S) Ninput_affine_dim num_layers linear_dimproj_dim left_order right_order left_stride right_strideoutput_affine_dim)r r torchZ from_numpyfloatrKWSModel)r!rrrrmeanZistd global_cmvnZ hidden_dim preprocessingr1r2r3r4r5r6r7r8r9 classifier activationZ kws_modelr&r&r'rJs8     zFSMNDecorator.init_model)NNrrF)__name__ __module__ __qualname____doc__strdictintrboolrr,rrr.r __classcell__r&r&r$r'rs.! rcseZdZdZdedededeejdeejdejdejd ejffd d Zd d Z ddZ e j ddde j dfde jde jdee je jffddZddZZS)r<aOur model consists of four parts: 1. global_cmvn: Optional, (idim, idim) 2. preprocessing: feature dimension projection, (idim, hdim) 3. backbone: backbone or feature extractor of the whole network, (hdim, hdim) 4. classifier: output layer or classifier of KWS model, (hdim, odim) 5. activation: nn.Sigmoid for wakeup word nn.Identity for speech command dataset idimodimhdimr>r?rr@rAc s>t||_||_||_||_||_||_||_||_ dS)ab Args: idim (int): input dimension of network odim (int): output dimension of network hdim (int): hidden dimension of network global_cmvn (nn.Module): cmvn for input feature, (idim, idim) preprocessing (nn.Module): feature dimension projection, (idim, hdim) backbone (nn.Module): backbone or feature extractor of the whole network, (hdim, hdim) classifier (nn.Module): output layer or classifier of KWS model, (hdim, odim) activation (nn.Module): nn.Identity for training, nn.Sigmoid for inference N) rrrKrLrMr>r?rr@rA) r!rKrLrMr>r?rr@rAr$r&r'rws  zKWSModel.__init__cCs |jSN)r to_kaldi_netr+r&r&r'rOs zKWSModel.to_kaldi_netcCs |j|SrN)rto_pytorch_net)r!Z kaldi_filer&r&r'rPs zKWSModel.to_pytorch_netr)Zdtypexin_cacher-cCsh|jdur ||}|jdur||}|||\}}|jdur&||}|jdur0||}||fSrN)r>r?rr@rA)r!rQrRZ out_cacher&r&r'r.s        zKWSModel.forwardcCs"|jdur |j|jdSrN)r? fuse_modulesrr+r&r&r'rSs  zKWSModel.fuse_modules)rBrCrDrErHrnnModulerrOrPr:Zzerosr;rrr.rSrJr&r&r$r'r<ls<    r<)!rsystempfiletypingrrrr:Ztorch.nnrTZmodelscope.metainforZmodelscope.modelsrZmodelscope.models.baserZmodelscope.models.builderrZ"modelscope.utils.audio.audio_utilsr Zmodelscope.utils.constantr Zcmvnr r ZfsmnrZregister_moduleZkeyword_spottingZ"speech_kws_fsmn_char_ctc_nearfieldrrUr<r&r&r&r's(        T