o *jL @s.dZddlZddlmZmZmZmZddlZddl Z ddl Z ddl mZddlmZedZdZd Zedddedddd Zd Zd Zd ZdZdeedeefddZdejdedejfddZdede jfddZde jdee e jffddZ!dee jdeee"fddZ#d eedee"ejffd!d"Z$d#ee"ejfdejfd$d%Z%d&d'd(ejd)e&dejfd*d+Z'd,ed-eed.e(defd/d0Z)deed1e"deefd2d3Z*deed.e(defd4d5Z+deedeefd6d7Z,d8edefd9d:Z-d-eed.e(d1e"defd;d<Z.d=eedeefd>d?Z/dS)@z Pairing logic for multimer data N)DictIterableListSequence) NumpyDict)restypes_with_x_and_gap-g?g?) msa_all_seqmsa_mask_all_seqdeletion_matrix_all_seqdeletion_matrix_int_all_seqmsamsa_maskdeletion_matrixdeletion_matrix_int)rrrr)Z residue_indexaatypeZall_atom_positionsZ all_atom_maskZseq_maskZbetween_segment_residuesZhas_alt_locationsZ has_hetatomsZasym_id entity_idZsym_idZ entity_maskZ deletion_meanZprediction_atom_maskZliterature_positionsZatom_indices_to_group_indicesZrigid_group_default_frameZnum_sym)Ztemplate_aatypeZtemplate_all_atom_positionsZtemplate_all_atom_mask)num_alignments seq_lengthchainsreturnc Cst|}|d}t|dkr|Sg}t|}t|}t|D]>\}}dd|D}|D]}|drIt|||} | |dd|f||<q/t t|dd|f|d<| |q |S)aReturns the original chains with paired NUM_SEQ features. Args: chains: A list of feature dictionaries for each chain. Returns: A list of feature dictionaries with sequence features including only rows to be paired. rcSsi|] \}}d|vr||qS)_all_seq).0kvrrs/var/www/html/Deteccion_Ine/venv/lib/python3.10/site-packages/modelscope/models/science/unifold/data/msa_pairing.py \z*create_paired_features..rNZnum_alignments_all_seq) listkeyslenpair_sequencesreorder_paired_rows enumerateitemsendswith pad_featuresnpasarrayappend) rZ chain_keysZupdated_chainsZ#paired_chains_to_paired_row_indices paired_rowsZ chain_numchainZ new_chain feature_name feats_paddedrrrcreate_paired_featuresGs.    r1featurer/cCsn|jttjks J|dvr"|jd}t|td|g|j}n |dkr*dg}n|Stj||gdd}|S)a~Add a 'padding' row at the end of the features list. The padding row will be selected as a 'paired' row in the case of partial alignment - for the chain that doesn't have paired alignment. Args: feature: The feature to be padded. feature_name: The name of the feature to be padded. Returns: The feature with an additional padding row. )r r r r rmsa_species_identifiers_all_seqraxis)dtyper*Zstr_shapeMSA_PAD_VALUESones concatenate)r2r/Znum_respaddingr0rrrr)is  r)chain_featurescCsz|d}|d}tj|d|kddtt|}tj|dkddtt|}t|dtt|d||d}|S) z9Makes dataframe with msa features needed for msa pairing.r rNr5r3)msa_species_identifiersmsa_rowmsa_similaritygap)r*sumfloatr#pd DataFrameZarange)r=Z chain_msaZ query_seqZper_seq_similarityZ per_seq_gapmsa_dfrrr _make_msa_dfs   rIrHcCs$i}|dD]\}}|||<q|S)z>Creates mapping from species to msa dataframe of that species.r@)groupby)rHZspecies_lookupspecies species_dfrrr_create_species_dicts rMthis_species_msa_dfscCs|g}dd|D}t|}dd}|D]}|dur(||}|jjd|j}ndg|}||qtt|}|S)a7Finds MSA sequence pairings across chains based on sequence similarity. Each chain's MSA sequences are first sorted by their sequence similarity to their respective target sequence. The sequences are then paired, starting from the sequences most similar to their target sequence. Args: this_species_msa_dfs: a list of dataframes containing MSA features for sequences for a specific species. Returns: A list of lists, each containing M indices corresponding to paired MSA rows, where M is the number of chains. cSsg|] }|durt|qSN)r#rrLrrr s z6_match_rows_by_sequence_similarity..cSs|jddddS)NrBrF)r6Z ascending)Z sort_values)xrrrsort_by_similaritysz>_match_rows_by_sequence_similarity..sort_by_similarityNr>) r*minrAZilocvaluesr,r!arrayZ transpose)rNall_paired_msa_rowsZnum_seqsZ take_num_seqsrSrLZspecies_df_sortedZmsa_rowsrrr"_match_rows_by_sequence_similaritys   rXexamplesc Cs<t|}g}t}|D]}t|}t|}|||t|q t|}|dt t|t g}ddt |D}t t|t g||<|D]H} | sOqJg} d} |D]}| |vrg| || | d7} qU| dqU| dkrrqJt t dd| Dd krqJt| } || || | qJd d|D}|S) z7Returns indices for paired MSA sequences across chains.r4cSsi|]}|gqSrr)rrrrrrsz"pair_sequences..rrNcSs g|] }t|tjrt|qSr) isinstancerFrGr#rPrrrrQs  z"pair_sequences..iXcSsi|] \}}|t|qSr)r*rV)r num_examplespaired_msa_rowsrrrrs )r#setrIrMr,updatesortedremover*zerosintrangeanyrVrXextendr') rYr[Zall_chain_species_dictZcommon_speciesr=rHZ species_dictrWall_paired_msa_rows_dictrKrNZspecies_dfs_presentr\rrrr$sT      r$rfcCsZg}t|ddD]}||}ttdd|D}t|}|||qt|S)aCreates a list of indices of paired MSA rows across chains. Args: all_paired_msa_rows_dict: a mapping from the number of paired chains to the paired indices. Returns: a list of lists, each containing indices of paired MSA rows across chains. The paired-index lists are ordered by: 1) the number of chains in the paired alignment, i.e, all-chain pairings will come first. 2) e-values T)reversecSsg|] }t|tjqSr)r*prodastypeZfloat64)rrowsrrrrQr z'reorder_paired_rows..)r_r*absrVZargsortre)rfrWZ num_pairingsr-Zpaired_rows_productZpaired_rows_sort_indexrrrr%s   r%g) pad_valuearrsrlcGsBdd|D}dtjj|}tjj|}||||j7}|S)z@Like scipy.linalg.block_diag but with an optional padding value.cSsg|]}t|qSr)r*Z ones_likerrRrrrrQ!szblock_diag..r)scipyZlinalg block_diagrir7)rlrmZ ones_arrsZ off_diag_maskZdiagrrrrps  rp np_examplenp_chains_listpair_msa_sequencesc Cstj|djdtjd|d<tj|djdtjd|d<|sSg}|D]}t|djd}d|d<||q$t||d<d d |D}t|d di|d <|St|djd|d<d|dd<d d |D}dd |D}t|d di}tj|dd} tj| |gdd|d <|S)z?Adds features that need to be computed/recomputed post merging.rrr7rrrrZcluster_bias_maskcS"g|] }tj|djtjdqSrrtr*r:r8Zint8rnrrrrQ?z._correct_post_merged_feats..rlZ bert_maskcSrurvrwrnrrrrQIrxcSru)r rtrwrnrrrrQLsr5)r*r+r8int32rar,r;rp) rqrrrsZcluster_bias_masksr.maskZ msa_masksZmsa_masks_all_seqZmsa_mask_block_diagr rrr_correct_post_merged_feats(s@     r{ max_templatescCsh|D]/}|D](\}}|tvr0t|j}||jd|d<dd|D}tj||dd||<qq|S)a0For each chain pad the number of templates to a fixed size. Args: chains: A list of protein chains. max_templates: Each chain will be padded to have this many templates. Returns: The list of chains, updated to have template features padded to max_templates. rcSsg|]}d|fqS)rr)rprrrrQiz"_pad_templates..Zconstant)mode)r'TEMPLATE_FEATURESr*Z zeros_liker8pad)rr|r.rrr<rrr_pad_templatesXs  rc s\i}|dD]fdd|D}dd}|tvry|s"dvr@tj|dd|<|dkr?t|jddd|d <qt|d ti|<|dkrxg}t |D]\}}|jd}t||d } | | qUt|dd|d <q|t vrtj|dd|<q|t vrtj|dd|<q|t vrt|tj|<q|d|<q|S) aYMerge features from multiple chains. Args: chains: A list of feature dictionaries that we want to merge. pair_msa_sequences: Whether to concatenate MSA features along the num_res dimension (if True), or to block diagonalize them (if False). Returns: A feature dictionary for the merged example. rcsg|]}|qSrrrnr/rrrQ|r~z8_merge_features_from_multiple_chains..rrr5rr>Zmsa_chains_all_seqrlr msa_chains)split MSA_FEATURESr*r;r:r8Zreshaperpr9r&r, SEQ_FEATURESrCHAIN_FEATURESrDriry) rrsZmerged_exampleZfeatsZfeature_name_splitrifeatZ cur_shapevalsrrr$_merge_features_from_multiple_chainsnsN      rcCsdtt}|D]}|dd}|||qg}t|D] }||}||qdd|D}|S)aUMerge all identical chains, making the resulting MSA dense. Args: chains: An iterable of features for each chain. Returns: A list of feature dictionaries. All features with the same entity_id will be merged - MSA features will be concatenated along the num_res dimension - making them dense. rrcSsg|]}t|ddqS)Trs)r)rrrrrrQs z-_merge_homomers_dense_msa..) collections defaultdictr!r,r_)rZ entity_chainsr.rZgrouped_chainsrrr_merge_homomers_dense_msas    rexamplec Cstd}|D]8}||vr>||}||d}z tj||gdd}Wnty9}z td||j|j|j|d}~ww|||<qtj|djdtjd|d <|S) z.Merges paired and block-diagonalised features.)rrrr5zconcat failed.Nrrtr)rr*r; Exceptionr8 __class__rVry)rfeaturesr/rZ feat_all_seqZ merged_featexrrr)_concatenate_paired_and_unpaired_featuress0   rcCs>t||d}t|}t|dd}|rt|}t|||d}|S)aAMerges features for multiple chains to single FeatureDict. Args: np_chains_list: List of FeatureDicts for each chain. pair_msa_sequences: Whether to merge paired MSAs. max_templates: The maximum number of templates to include. Returns: Single FeatureDict for entire complex. )r|Fr)rqrrrs)rrrrr{)rrrsr|rqrrrmerge_chain_featuress r np_chainsc Cs|d}t}i}|D]}t|dd}||vrntdd|dD}g}t|dD]\}} | |vr<||q-i} |D](} | |vri|rR|| || | <qAt|| j} d| d<t j | || j d| | <qA| ||<||D] } ||| || <qrt j |djdt j d|d<q |S) z=Removes unpaired sequences which duplicate a paired sequence.rrcss|]}|VqdSrO)tobytes)rsrrr sz1deduplicate_unpaired_sequences..r rrtr)r"rrbr]r&rr,r!r8r*rar7rVry) rZ feature_namesZ msa_featuresZcache_msa_featuresr.rZ sequence_setZ keep_rowsZrow_numseqZnew_msa_featuresr/Z new_shaperrrdeduplicate_unpaired_sequencess@        r)0__doc__rtypingrrrrnumpyr*ZpandasrFZ scipy.linalgroZdata_opsrZresidue_constantsrindexZ MSA_GAP_IDXZSEQUENCE_GAP_CUTOFFZSEQUENCE_SIMILARITY_CUTOFFr9rrrrr1Zndarraystrr)rGrIbytesrMrbrXr$r%rErpboolr{rrrrrrrrrrs     "  )6    0  ,