o #j/@s UddlmZddlZddlZddlZddlmZddlmZddl m Z ddl m Z ddl mZddlmZdd lmZd d lmZmZmZmZmZmZmZmZmZmZmZmZm Z m!Z!m"Z"m#Z#m$Z$e ed duddZ%e ed dvddZ&e ed dwddZ'e(dde)DZ*de+d<dde*DZ,de+d<e ed dxd!d"Z-e ed dvd#d$Z.e ed dvd%d&Z/e ed dvd'd(Z0e ed dvd)d*Z1e ed dvd+d,Z2e ed dvd-d.Z3e ed dvd/d0Z4e ed dvd1d2Z5e ed dvd3d4Z6e ed dvd5d6Z7e ed dvd7d8Z8e ed dvd9d:Z9e ed dvd;d<Z:e ed dvd=d>Z;e e Ddzd{dHdIZ?e dJd d|dLdMZ@d}dOdPZAd~dRdSZBdddWdXZCdd\d]ZDdd^d_ZEd`ejFdafddedfZG dddsdtZHdS)) annotationsN) bisect_right)IncrementalDecoder)aliases) lru_cache)findall) Generator)MultibyteIncrementalDecoder)ENCODING_MARKSIANA_SUPPORTED_SIMILARRE_POSSIBLE_ENCODING_INDICATIONUNICODE_RANGES_COMBINEDUNICODE_SECONDARY_RANGE_KEYWORDUTF8_MAXIMAL_ALLOCATIONCOMMON_CJK_CHARACTERS_LATIN_CJK_HANGUL _KATAKANA _HIRAGANA_THAI_ARABIC_ARABIC_ISOLATED_FORM_ACCENT_KEYWORDS _ACCENTUATED)maxsize characterstrreturnintcCszt|}Wn tyYdSwd}d|vr|tO}d|vr$|tO}d|vr,|tO}d|vr4|tO}d|vr<|tO}d|vrD|tO}d|vrT|t O}d |vrT|t O}t D] }||vrc|t O}|SqV|S) zRCompute all name-based classification flags with a single unicodedata.name() call.rZLATINZCJKZHANGULZKATAKANAZHIRAGANAZTHAIZARABICz ISOLATED FORM) unicodedataname ValueErrorrrrrrrrrrr)rdescflagskwr'Y/var/www/html/Deteccion_Ine/venv/lib/python3.10/site-packages/charset_normalizer/utils.py_character_flags&s: r)boolcCtt|t@SN)r*r)rrr'r'r(is_accentuatedIr.cCs.t|}|s |S|d}tt|ddS)N r)r! decompositionsplitchrr )rZ decomposedcodesr'r'r( remove_accentNs  r6ccs"|] \}}|j|j|fVqdSr,)startstop).0r"Z ord_ranger'r'r( [s  r:zlist[tuple[int, int, str]]_UNICODE_RANGES_SORTEDcCsg|]}|dqS)rr')r9er'r'r( _sr=z list[int]_UNICODE_RANGE_STARTS str | NonecCs<t|}tt|d}|dkrt|\}}}||kr|SdS)zK Retrieve the Unicode range official name from a single character. r rN)ordrr>r;)rZ character_ordidxr7r8r"r'r'r( unicode_rangebsrBcCr+r,)r*r)rr-r'r'r(is_latinsr/rCcCs2t|}d|vr dSt|}|durdSd|vS)NPTF Punctuationr!categoryrBrcharacter_categorycharacter_ranger'r'r(is_punctuationxs rKcCsBt|}d|vs d|vrdSt|}|durdSd|vo |dkS)NSNTFZFormsZLorFrHr'r'r( is_symbols rNcCs$t|}|dur dSd|vpd|vS)NFZ EmoticonsZ Pictographs)rB)rrJr'r'r( is_emoticonsrOcCs.|s|dvr dSt|}d|vp|dvS)N><+>u｜TZ>ZPdZPcZPo)isspacer!rG)rrIr'r'r( is_separators rUcCs||kSr,)islowerisupperr-r'r'r(is_case_variabler/rXcCr+r,)r*r)rr-r'r'r(is_cjkr/rYcCr+r,)r*r)rr-r'r'r( is_hiraganar/rZcCr+r,)r*r)rr-r'r'r( is_katakanar/r[cCr+r,)r*r)rr-r'r'r( is_hangulr/r\cCr+r,)r*r)rr-r'r'r(is_thair/r]cCr+r,)r*r)rr-r'r'r( is_arabicr/r^cCr+r,)r*r)rr-r'r'r(is_arabic_isolated_formr/r_cCs|tvSr,)rr-r'r'r(is_cjk_uncommonsr` range_namecstfddtDS)Nc3s|]}|vVqdSr,r')r9keywordrar'r(r:sz-is_unicode_range_secondary..)anyrrcr'rcr(is_unicode_range_secondarysrecCs(|duo|duo|dko|dkS)NFu)rT isprintabler-r'r'r(is_unprintables  rh sequencebytes | bytearray search_zonecCst|ttfs tt|}tt|dt||jddd}t|dkr&dS|D]'}| dd}t D]\}}||krD|S||krN|Sq6q(dS)zW Extract using ASCII-only decoder any specified encoding in the first n-bytes. Nasciiignoreerrorsr-_) isinstancebytes bytearray TypeErrorlenrr mindecodelowerreplaceritems)rjrlZseq_lenresultsZspecified_encodingencoding_alias encoding_ianar'r'r(any_specified_encodings&   rr"cCs |dvpttd|jtS)zQ Verify is a specific encoding is a multi byte one based on it IANA name > utf_16 utf_16_beutf_32 utf_16_leutf_8 utf_32_beZ utf_8_sigutf_7 utf_32_le encodings.) issubclass importlib import_modulerr )r"r'r'r(is_multi_byte_encoding s  rtuple[str | None, bytes]cCsJtD] }t|}t|tr|g}|D]}||r!||fSqqdS)z9 Identify and extract SIG/BOM in given sequence. )N)r rsrt startswith)rj iana_encodingZmarksmarkr'r'r(identify_sig_or_boms  rrcCs|dvS)N>rrr')rr'r'r(should_strip_sig_or_bom0srTcp_namestrictcCsN|dd}tD]\}}|||fvr|Sq |r%td|d|S)zIReturns the Python normalized encoding name (Not the IANA official name).rqrrzUnable to retrieve IANA for '')rzr{rr|r#)rrr~rr'r'r( iana_name4s r iana_name_a iana_name_bfloatc Cst|st|r dStd|j}td|j}|dd}|dd}d}tdD]}t|g}||||krA|d7}q,|dS)Ngrrnrorr )rrrrrangertry) rrZ decoder_aZ decoder_bZid_aZid_bZcharacter_match_countiZ to_be_decodedr'r'r( cp_similarityEs    rcCs|tvo |t|vS)z Determine if two code page are at least 80% similar. IANA_SUPPORTED_SIMILAR dict was generated using the function cp_similarity. )r )rrr'r'r( is_cp_similarYs rZcharset_normalizerz)%(asctime)s | %(levelname)s | %(message)slevel format_stringNonecCs:t|}||t}|t|||dSr,)logging getLoggersetLevel StreamHandler setFormatter Formatter addHandler)r"rrloggerhandlerr'r'r(set_logging_handlerds  r sequencesroffsetsr chunk_sizebom_or_sig_availablestrip_sig_or_bom sig_payloadrtis_multi_byte_decoderdecoded_payloadGenerator[str, None, None]c cs&|r|dur|D]} || | |} | sdS| Vq dS|D]p} | |} | t|dkr/q || | |} |rA|durA|| } | j||rHdndd} |r| dkrt|d} |r| d| |vrt| | dd D]#}||| } |r{|dur{|| } | j|dd} | d| |vrnqi| Vq dS) NFrnrrorr1)rwryrxr)rrrrrrrrrrchunkZ chunk_endZ cut_sequenceZchunk_partial_size_chkjr'r'r(cut_sequence_chunksqsD       r)rrrr )rrrr*)rrrr)rrrr?)rarrr*)ri)rjrkrlr rr?)r"rrr*)rjrkrr)rrrr*)T)rrrr*rr)rrrrrr)rrrrrr*)r"rrr rrrrr,)rrkrrrrrr rr*rr*rrtrr*rr?rr)I __future__rrrr!bisectrcodecsrZencodings.aliasesr functoolsrrertypingrZ_multibytecodecr Zconstantr r r rrrrrrrrrrrrrrr)r.r6sortedr|r;__annotations__r>rBrCrKrNrOrUrXrYrZr[r\r]r^r_r`rwrerhrrrrrrrINFOrrr'r'r'r(s       L "                      "