o *jU@slddlmZddlZddlmZ ddejdejdeefddZdd e d e fd d Z Gdddej Z dS))OptionalNmaskdtypetgt_lencCs^|\}}|dur |n|}|ddddddf|d|||}||t|jS)z_ Expands attention_mask from `[bsz, seq_len]` to `[bsz, 1, tgt_seq_len, src_seq_len]`. N)sizeexpandtoZ masked_fillbooltorchZfinfomin)rrrZbszZsrc_lenZ expanded_maskr n/var/www/html/Deteccion_Ine/venv/lib/python3.10/site-packages/modelscope/models/multi_modal/ofa/utils/utils.py expand_masks   rF drop_probtrainingcCsZ|dks|s|Sd|}d|jddf}|tj||j|jd}||||}|S)a Drop paths (Stochastic Depth) per sample (when applied in main path of residual blocks). Args: x (`nn.Modules`): input nn layers. drop_prob (`float`): drop path ratio. training (`bool`): whether is training or inference. rr)rdevice)shaper ZrandrrZfloor_div)xrrZ keep_probrZ random_tensoroutputr r r drop_paths  rcs8eZdZdZd fdd ZddZdefdd ZZS) DropPathz Drop paths (Stochastic Depth) per sample (when applied in main path of residual blocks). See more details about drop path from https://arxiv.org/pdf/1605.07648v4.pdf. Args: drop_prob: drop path ratio. Ncst||_dSN)super__init__r)selfr __class__r rr4s  zDropPath.__init__cCst||j|jSr)rrr)rrr r rforward8szDropPath.forwardreturncCs d|jS)Nzp={})formatr)rr r r extra_repr;s zDropPath.extra_reprr) __name__ __module__ __qualname____doc__rr strr# __classcell__r r rrr+s rr)rF)typingrr Ztorch.nnnnZTensorrintrfloatr rModulerr r r rs