o *j T@sddlZddlZddlmZmZmZddlZddlmZddlm Z ddl m Z m Z ddl mZddlmZddlmZmZmZdd lmZdd lmZd d lmZmZd d lmZeZdZej ej!ej"dGdddeZ#dS)N)OptionalTupleUnion)nn)CrossEntropyLoss)assert_device_mapget_device_map)Models)MODELS)AttentionBackboneModelOutputSeq2SeqLMOutputTokenGeneratorOutput)Tasks) get_logger)T5PreTrainedModelT5Stack)T5Configa_ The input argument `head_mask` was split into two arguments `head_mask` and `decoder_head_mask`. Currently, `decoder_head_mask` is set to copy `head_mask`, but this feature is deprecated and will be removed in future versions. If you do not want to use any `decoder_head_mask` now, please set `decoder_head_mask = torch.ones(num_layers, num_heads)`. )Z group_key module_namec%seZdZgdZdgZd2deffdd Zd2ddZd d Zd d Z d dZ ddZ ddZ ddZ ddZ                d3deejdeejdeejdeejdeejdeejdeejdeeeejdeeeejd eejd!eejd"eejd#eed$eed%eed&eed'eeejeff"d(d)Z       d4d*d+Zd"ejfd,d-Zfd.d/Zd0d1ZZS)5T5ForConditionalGeneration)zencoder\.embed_tokens\.weightzdecoder\.embed_tokens\.weightzlm_head\.weightzMdecoder\.block\.0\.layer\.1\.EncDecAttention\.relative_attention_bias\.weightNconfigc st||j|_t|j|j|_t |}d|_ d|_ d|_ t ||j|_t |}d|_ d|_ |j|_t ||j|_tj|j|jdd|_|d|_|dkrZ|dSdS)NFT)Zbiasauto)super__init__Zd_model model_dimrZ EmbeddingZ vocab_sizesharedcopydeepcopyZ is_decoder use_cacheZis_encoder_decoderrencodernum_decoder_layers num_layersdecoderZLinearlm_headZ post_initmodel_parallel parallelize)selfr device_mapkwargsZencoder_configZdecoder_config __class__n/var/www/html/Deteccion_Ine/venv/lib/python3.10/site-packages/modelscope/models/nlp/T5/text2text_generation.pyr<s&    z#T5ForConditionalGeneration.__init__cCsv|durtt|jjttjn||_t |jt|jj|j |j|j |j|j |j j|_ d|_dS)NT)rlenrblockrangetorchcudaZ device_countr'rr%r"r#to first_devicer$)r&r'r+r+r,r%Xs z&T5ForConditionalGeneration.parallelizecCsX|j|j|jd|_|jd|_|jd|_d|_d|_tj dS)NcpuF) r deparallelizer"r2r#r$r'r0r1Z empty_cacher&r+r+r,r5cs  z(T5ForConditionalGeneration.deparallelizecC|jSN)rr6r+r+r,get_input_embeddingsmz/T5ForConditionalGeneration.get_input_embeddingscCs"||_|j||j|dSr8)rrset_input_embeddingsr"r&Znew_embeddingsr+r+r,r;ps z/T5ForConditionalGeneration.set_input_embeddingscCs ||_dSr8r#r<r+r+r,set_output_embeddingsu z0T5ForConditionalGeneration.set_output_embeddingscCr7r8r=r6r+r+r,get_output_embeddingsxr:z0T5ForConditionalGeneration.get_output_embeddingscCr7r8)rr6r+r+r, get_encoder{r:z&T5ForConditionalGeneration.get_encodercCr7r8)r"r6r+r+r, get_decoder~r:z&T5ForConditionalGeneration.get_decoder input_idsattention_maskdecoder_input_idsdecoder_attention_mask head_maskdecoder_head_maskcross_attn_head_maskencoder_outputspast_key_values inputs_embedsdecoder_inputs_embedslabelsroutput_attentionsoutput_hidden_states return_dictreturncKs|| dur| n|jj} |dur|n|jj}|dur,|dur,|jj|jjkr,ttt|}|dur=|j ||| ||||d}n$|rat |t sat |dt |dkrR|dndt |dkr]|dndd}|d}|j rptj|jj| dur|dur| dur|| }|j rtj|jj||jj}|dur||jj}|dur||jj}|dur||jj}|j||| | ||||| |||d }|d}|j rtj|j j|j|j j|_||jjj}|jjr||jd}||}d}| durtd d }||d |d | d }|s*|f|dd|}|dur(|f|S|St|||j|j|j |j!|j"|j|j d S) aH Args: input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`): Indices of input sequence tokens in the vocabulary. T5 is a model with relative position embeddings so you should be able to pad the inputs on both the right and the left. Indices can be obtained using [`T5Tokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for detail. [What are input IDs?](../glossary#input-ids) To know more on how to prepare `input_ids` for pretraining take a look a [T5 Training](./t5#training). attention_mask (`torch.FloatTensor` of shape `(batch_size,sequence_length)`, *optional*): Mask to avoid performing attention on padding token indices. Mask values selected in `[0, 1]`: - 1 for tokens that are **not masked**, - 0 for tokens that are **masked**. [What are attention masks?](../glossary#attention-mask) decoder_input_ids (`torch.LongTensor` of shape `(batch_size, target_sequence_length)`, *optional*): Indices of decoder input sequence tokens in the vocabulary. Indices can be obtained using [`T5Tokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are decoder input IDs?](../glossary#decoder-input-ids) T5 uses the `pad_token_id` as the starting token for `decoder_input_ids` generation. If `past_key_values` is used, optionally only the last `decoder_input_ids` have to be input (see `past_key_values`). To know more on how to prepare `decoder_input_ids` for pretraining take a look at [T5 Training](./t5#training). decoder_attention_mask (`torch.BoolTensor` of shape `(batch_size, target_sequence_length)`, *optional*): Default behavior: generate a tensor that ignores pad tokens in `decoder_input_ids`. Causal mask will also be used by default. head_mask (`torch.FloatTensor` of shape `(num_heads,)` or `(num_layers, num_heads)`, *optional*): Mask to nullify selected heads of the self-attention modules in the encoder. Mask values selected in `[0, 1]`: - 1 indicates the head is **not masked**, - 0 indicates the head is **masked**. decoder_head_mask (`torch.FloatTensor` of shape `(num_heads,)` or `(num_layers, num_heads)`, *optional*): Mask to nullify selected heads of the self-attention modules in the decoder. Mask values selected in `[0, 1]`: - 1 indicates the head is **not masked**, - 0 indicates the head is **masked**. cross_attn_head_mask (`torch.Tensor` of shape `(num_heads,)` or `(num_layers, num_heads)`, *optional*): Mask to nullify selected heads of the cross-attention modules in the decoder. Mask values selected in `[0, 1]`: - 1 indicates the head is **not masked**, - 0 indicates the head is **masked**. encoder_outputs (`tuple(tuple(torch.FloatTensor)`, *optional*): Tuple consists of (`last_hidden_state`, `optional`: *hidden_states*, `optional`: *attentions*) `last_hidden_state` of shape `(batch_size, sequence_length, hidden_size)` is a sequence of hidden states at the output of the last layer of the encoder. Used in the cross-attention of the decoder. past_key_values (`tuple(tuple(torch.FloatTensor))` of length `config.n_layers` with each tuple having 4 tensors of shape `(batch_size, num_heads, sequence_length - 1, embed_size_per_head)`): Contains precomputed key and value hidden states of the attention blocks. Can be used to speed up decoding. If `past_key_values` are used, the user can optionally input only the last `decoder_input_ids` (those that don't have their past key value states given to this model) of shape `(batch_size, 1)` instead of all `decoder_input_ids` of shape `(batch_size, sequence_length)`. inputs_embeds (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`, *optional*): Optionally, instead of passing `input_ids` you can choose to directly pass an embedded representation. This is useful if you want more control over how to convert `input_ids` indices into associated vectors than the model's internal embedding lookup matrix. decoder_inputs_embeds (`torch.FloatTensor` of shape `(batch_size, target_sequence_length, hidden_size)`, *optional*): Optionally, instead of passing `decoder_input_ids` you can choose to directly pass an embedded representation. If `past_key_values` is used, optionally only the last `decoder_inputs_embeds` have to be input (see `past_key_values`). This is useful if you want more control over how to convert `decoder_input_ids` indices into associated vectors than the model's internal embedding lookup matrix. If `decoder_input_ids` and `decoder_inputs_embeds` are both unset, `decoder_inputs_embeds` takes the value of `inputs_embeds`. use_cache (`bool`, *optional*): If set to `True`, `past_key_values` key value states are returned and can be used to speed up decoding (see `past_key_values`). output_attentions (`bool`, *optional*): Whether or not to return the attentions tensors of all attention layers. See `attentions` under returned tensors for more detail. output_hidden_states (`bool`, *optional*): Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors for more detail. return_dict (`bool`, *optional*): Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple. labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the sequence classification/regression loss. Indices should be in `[-100, 0, ..., config.vocab_size - 1]`. All labels set to `-100` are ignored (masked), the loss is only computed for labels in `[0, ..., config.vocab_size]` Returns: Examples: >>> from transformers import T5Tokenizer, T5ForConditionalGeneration >>> tokenizer = T5Tokenizer.from_pretrained("t5-small") >>> model = T5ForConditionalGeneration.from_pretrained("t5-small") >>> # training >>> input_ids = tokenizer("The walks in park", return_tensors="pt").input_ids >>> labels = tokenizer(" cute dog the ", return_tensors="pt").input_ids >>> outputs = model(input_ids=input_ids, labels=labels) >>> loss = outputs.loss >>> logits = outputs.logits >>> # inference >>> input_ids = tokenizer( ... "summarize: studies have shown that owning a dog is good for you", return_tensors="pt" >>> ).input_ids # Batch size 1 >>> outputs = model.generate(input_ids) >>> print(tokenizer.decode(outputs[0], skip_special_tokens=True)) >>> # studies have shown that owning a dog is good for you. N)rCrDrLrGrOrPrQrr)last_hidden_state hidden_states attentions) rCrDrLrKencoder_hidden_statesZencoder_attention_maskrGrIrrOrPrQgi)Z ignore_index) lossZlogitsrKZdecoder_hidden_statesZdecoder_attentionscross_attentionsZencoder_last_hidden_staterWZencoder_attentions)#rrZuse_return_dictr!r warningswarnZ2_T5ForConditionalGeneration__HEAD_MASK_WARNING_MSG FutureWarningr isinstancer r-r$r0r1Z set_devicer"r3 _shift_rightr2r#weightdeviceZtie_word_embeddingsrrviewsizer rKrUrVrZrT)r&rCrDrErFrGrHrIrJrKrLrMrNrrOrPrQr(rUZdecoder_outputsZsequence_outputZ lm_logitsrYZloss_fctoutputr+r+r,forwards!        z"T5ForConditionalGeneration.forwardc Ks2|dur|ddddf}||||||||dS)NrX)rErKrJrDrGrHrIrr+) r&rCpastrDrGrHrIrrJr(r+r+r,prepare_inputs_for_generations z8T5ForConditionalGeneration.prepare_inputs_for_generationcCs ||Sr8)r_)r&rNr+r+r,%prepare_decoder_input_ids_from_labelsr?z@T5ForConditionalGeneration.prepare_decoder_input_ids_from_labelscs4tj|i|}tt|tjr|dS|ddS)Nr) sequences)rgenerater r^r0Tensor)r&argsr(rdr)r+r,rjs z#T5ForConditionalGeneration.generatec Cs|dur td|Sd}|D]1}d}|D]}||d||jf}q|dj|djks1Jt|t|ks;J||f}q|S)NzHYou might want to consider setting `use_cache=True` to speed up decodingr+r)loggerwarningZ index_selectr2rashaper-)r&rfZbeam_idxZreordered_decoder_pastZlayer_past_statesZreordered_layer_past_statesZlayer_past_stater+r+r,_reorder_caches0  z)T5ForConditionalGeneration._reorder_cacher8)NNNNNNNNNNNNNNNN)NNNNNNN)__name__ __module__ __qualname__Z_keys_to_ignore_on_load_missingZ"_keys_to_ignore_on_load_unexpectedrrr%r5r9r;r>r@rArBrr0Z LongTensorZ FloatTensorZ BoolTensorrkrboolrr rergrhrjrp __classcell__r+r+r)r,r.s            r)$rr[typingrrrr0rZtorch.nnrZ'transformers.utils.model_parallel_utilsrrZmodelscope.metainfor Zmodelscope.models.builderr Zmodelscope.outputsr r r Zmodelscope.utils.constantrZmodelscope.utils.loggerrZbackbonerr configurationrrmZ__HEAD_MASK_WARNING_MSGZregister_moduleZtext2text_generationZT5rr+r+r+r,s*