o #jm@sdZddlmZddlmZddlmZmZddlmZm Z m Z ddl m Z ddl Z ddlZddlZddlZddlZddlZddlZzddlmZWneyYddlmZYnwdd lmZddl Z dd lmZddlZddlZzddlZd ZWn eyd ZdZYnwddlZd gZ dZ!dZ"dfddZ#dfddZ$dfddZ%ddZ&ddZ'ddZ(dfddZ)d Z*d!d"Z+e+Z,d#d$Z-Gd%d&d&e.Z/dfd'd(Z0dgd)d*Z1d+d,Z2d-d.Z3d/d0Z4d1d2Z5d3d4Z6d5d6Z7e7Z8dhd7d8Z9d9d:Z:d;d<Z;d=d>Ze=d@Z?e=dAZ@dBdCZAdDdEZBdFdGZCGdHdIdIe.ZDdJdKZEdLdMZFdNdOZG dPdQZHdRdSZIdTdUZJdVdWZKdXdYZLdZd[ZMd\d]ZNd^d_ZOdid`daZPdZQGdbdcdce.ZRdddeZSdS)jz0 This module provide some tools for bce client. )print_function)absolute_import)strbytes) iteritemsiterkeys itervalues)compatN)urlparse)AES) http_headersTFs.cdn.bcebos.coms bcebos.comshttp cCs|}|r ||t} |}||krdkr nn|}||}|s(n|||dkr7|t|8}|dkrz*_generate_crc64_tables..r:r,r)range CRC64_POLY)tableir*r<tr r r!_generate_crc64_tabless    "rFcCs2t|tr |d}t|}t}|d@}t|}d}|d|kr||}||d}||d}||d} ||d} ||d } ||d } ||d } t|tstt|}t|}t|}t| } t| } t| } t| } t| } |||d>B|d >B| d >B| d>B| d>B| d>B| d>BN}|d |d@|d |d?d@A|d |d ?d@A|d|d ?d@A|d|d?d@A|d|d?d@A|d|d?d@A|d|d?d@A}|d7}|d|ks!||kr||}t|tst|}td||Ad@|d?A}|d7}||ks|d@S)z, Fast CRC64 update using slice-by-8 utf-8lrr?r,r1(08r@) isinstancerencode memoryview CRC64_TABLErintord)r*datamvrCrrDZb0b1b2Zb3Zb4Zb5Zb6Zb7br r r! _crc64_update)s|                !  r^c@s*eZdZdZd ddZddZddZd S) CRC64ECMAz To generate CRC64ECMA rcCs ||_dSNr*)selfr*r r r!__init__cs zCRC64ECMA.__init__cCst|j||_dS)z2 Update CRC64 with specified data N)r^r*)rbrYr r r!rfszCRC64ECMA.updatecC|jS)z& Get CRC64 ECMA value rarbr r r!valuelszCRC64ECMA.valueNr)__name__ __module__ __qualname____doc__rcrrfr r r r!r_^s   r_cCs|}|r ||td} |}||krdkr nn|}||}|s(n|||dkr7|t|8}|dkr.r:z.~-_rcSsg|]}|dqS)rG)rT)r;r{r r r!r=r>)rAstring ascii_lettersdigitsrXrSr)retchr r r!_get_normalized_char_lists rcCstg}t|D].}d}d}t|trt|d}|}n|}t|}|dkr-|s-|dq|t|qd|S)a Encode in_str. When encoding_slash is True, don't encode skip_chars, vice versa. :type in_str: string :param in_str: None :type encoding_slash: Bool :param encoding_slash: None =============================== :return: **ASCII string** rr rG/) r~rSrWchrrTrXappend_NORMALIZED_CHAR_LISTjoin)Zin_strZencoding_slashtmprsepindexr r r!normalize_strings     rcGs|g}|D] }|r|t|dqt|dkr?|dd|d<|dd|d<tdt|dD] }||d||<q3d|S)z Append path_components to the end of base_uri in order, and ignore all empty strings and None :param base_uri: None :type base_uri: string :param path_components: None :return: the final url :rtype: str Fr,rrr )rrrrstriplstriprArr)Zbase_uriZpath_componentsrpathrDr r r! append_uri4s   rcCsd}t|dkst|dkrdS|ddks|ddkrdSd|d kr*d ks;nd |d kr8d ks;dSdS|D] }||vrFdSq=d S)z Check bucket name whether is legal. :type bucket: string :param bucket: None ======================= :return: **Boolean** z%abcdefghijklmnopqrstuvwxyz0123456789-rI?Fr -r<rrz09T)r)ZbucketalphabetrDr r r!check_bucket_validLs 0rcCst}d|d<d|d<d|d<d|d<d |d <d |d <d |d<d|d<d|d<d|d<d|d<z2t|}tj|}|dd}|t|vrO||}nddl }| |j d|d}Wnd}Y|sld}t |S)z Get file type by filename. :type file_name: string :param file_name: None ======================= :return: **Type Value** zapplication/javascriptjszAapplication/vnd.openxmlformats-officedocument.spreadsheetml.sheetZxlsxzDapplication/vnd.openxmlformats-officedocument.spreadsheetml.templateZxltxzEapplication/vnd.openxmlformats-officedocument.presentationml.templateZpotxzFapplication/vnd.openxmlformats-officedocument.presentationml.slideshowZppsxzIapplication/vnd.openxmlformats-officedocument.presentationml.presentationZpptxzBapplication/vnd.openxmlformats-officedocument.presentationml.slideZsldxzGapplication/vnd.openxmlformats-officedocument.wordprocessingml.documentZdocxzGapplication/vnd.openxmlformats-officedocument.wordprocessingml.templateZdotxz.application/vnd.ms-excel.addin.macroEnabled.12Zxlamz5application/vnd.ms-excel.sheet.binary.macroEnabled.12Zxlsb.r rNzapplication/octet-stream)dictr convert_to_stringosrbasenamerrzr mimetypesinit types_mapgetr}) file_nameZmime_maprsuffixZ mime_typerr r r!guess_content_type_by_file_namecs6     rz(.)([A-Z][a-z]+)z([a-z])([0-9]{2,})z([a-z0-9])([A-Z])cCs4|dkrdStd|}td|}td|S)aConvert camel case to a "pythonic" name. Examples:: pythonize_name('CamelCase') -> 'camel_case' pythonize_name('already_pythonized') -> 'already_pythonized' pythonize_name('HTTPRequest') -> 'http_request' pythonize_name('HTTPStatus200Ok') -> 'http_status_200_ok' pythonize_name('UPPER') -> 'upper' pythonize_name('ContentMd5')->'content_md5' pythonize_name('') -> '' ZeTagetagz\1_\2)_first_cap_regexsub_number_cap_regex_end_cap_regexr)rs1s2r r r!pythonize_names    rcCsn|durdSg}t|D]!\}}|r|jtjkr-|dur d}|dt|t|fq |d|S)z@ :param params: :param for_signature: :return: Nrs%s=%s&)rrr Z AUTHORIZATIONrrsortr)paramsZ for_signatureresultrrr r r!get_canonical_querystrings rcCsg}t|jD]4\}}|ds;|dkr;t|tr#|d||fqt|tr2|d||fq|d||fqdd|S)z# :param obj: :return: __raw_dataz%s:'%s'z%s:u'%s'z%s:%sz{%s},)r__dict__ startswithrSrrrr)objrrrr r r! print_objects  rc@s*eZdZdZd ddZddZddZdS) Expandoz Expandable class NcCs|r |j|dSdSr`)rr)rbZ attr_dictr r r!rcszExpando.__init__cCs|drtdS)Nr)rAttributeError)rbitemr r r! __getattr__s zExpando.__getattr__cCst|Sr`)rrer r r!__repr__szExpando.__repr__r`)rhrirjrkrcrrr r r r!rs   rcCsDi}t|D]\}}t|tjst|}t|}|||<qt|S)z! :param d: :return: )rrSr string_typesrrr)dattrrrr r r!dict_to_python_objects   rc sfdd}|S)zG decorator of input param check :param types: :return: csfdd}j|_|S)Ncst|D]8\}}jj|vr<|durtdjj|t|jj|s._required.._decorated)rh)rrr)rr! _requiredszrequired.._requiredr )rrr rr!requireds rc Csd|vrd|}z t|t|j}Wnty*}z td|t|fd}~ww|jttj j jkr@tj j }tj j j }n|jttj j jkrUtj j }tj j j }ntd|j|j }|jdurg|j}|||fS)ac parse protocol, host, port from endpoint in config :type: string :param endpoint: endpoint in config :type: baidubce.protocol.HTTP or baidubce.protocol.HTTPS :param default_protocol: if there is no scheme in endpoint, we will use this protocol as default :return: tuple of protocol, host, port s//zInvalid endpoint:%s, error:%sNzUnsupported protocol %s)r r r}r ExceptionrrschemebaidubceprotocolHTTP default_portZHTTPShostnameport)ZendpointZdefault_protocolZ parse_resulterrhostr r r!parse_host_ports0      rc Cs`dd}|dd}tj}t||}||}|jdd}||}t|d} | jdd} | S)zc :param adminpass: adminpass :param secretkey: secretkey :return: cipheradminpass cSs(|dt|dtdt|dS)NrN)rr)r{r r r!Es(z+aes128_encrypt_16char_key..rrNrG)encoding hex_codec)r ZMODE_ECBnewrTZencryptcodecsdecode) Z adminpassZ secretkeyZpad_itkeymodeZcryptorZ pad_adminZbyte_pad_adminZcryptoradminpassZbyte_cipheradminpassZcipheradminpassr r r!aes128_encrypt_16char_key;s      rcCs.|durdStD] }||rdSqdS)zU :param host: custom domain :return: domain end with cdn endpoint or not NFT)DEFAULT_CNAME_LIKE_LISTrendswith)rrr r r!is_cname_like_hostXsrcCsJ|dus|dur dS|d}|t|o$t|dko$t|S)zQ custom host : xxx.region.bcebos.com : return: custom, domain or not NFryrJ)rzrrr r}ris_bos_suffixed_host)rZ bucket_nameZ host_splitr r r!is_custom_hostes  rcCs6|durdS|dr|dd}n|}|tS)zD :param host: bos endpoint :return: bos endpoint or not NFrr )rrDEFAULT_BOS_DOMAIN_SUFFIX)rZ check_hostr r r!rss  rcCstd}||S)z> :param ipAddr: ip address :return: true or false sL((2(5[0-5]|[0-4]\d))|[0-1]?\d{1,2})(.((2(5[0-5]|[0-4]\d))|[0-1]?\d{1,2})){3})recompilematch)ipAddrZ compile_ipr r r! check_ipv4s  rcCsBt|dr t|St|dr|jSt|drt|drt|SdS)N__len__rrr)hasattrrfile_object_remaining_bytes)rYr r r!_get_data_sizes  rcCs4|}|dtj|}||tj||SNr)rrrSEEK_ENDSEEK_SET)fileobjcurrentendr r r!rs rcCs|r |||dSdSr`r )progress_callbackconsumed_bytes total_bytesr r r!_invoke_progress_callbacksr cCs6|durt|}|durtd|jjt|||S)aHreturn a adapter,when reading 'data', that is, calling read or iterating over it Call the progress callback function :param data: bytes,file object or iterable :param progress_callback: callback function, ref:`_default_progress_callback` :param size: size of `data` :return: callback function adapter Nz{0} is not a file object)rrformat __class__rh_BytesAndFileAdapter)rYr sizer r r!make_progress_adapters   rc@s<eZdZdZd ddZeddZddZeZd d d Z dS) ra.With this adapter, you can add progress monitoring to 'data'. :param data: bytes or file object :param progress_callback: user-provided callback function. like callback(bytes_read, total_bytes) bytes_read is readed bytes;total_bytes is total bytes :param int size : data size NcCs||_||_||_d|_dSr)rYr rr)rbrYr rr r r!rcs z_BytesAndFileAdapter.__init__cCrdr`)rrer r r!rsz_BytesAndFileAdapter.lencCsdS)NTr rer r r!__bool__sz_BytesAndFileAdapter.__bool__cCs|j|jkr tdS|dus|dkr|j|j}n t||j|j}t|jtr5|j|j|j|}n|j|}|j|7_t |j t|j|j|j|S)Nrr) rrr r}minrSrYrrr r )rbamtrcontentr r r!rs    z_BytesAndFileAdapter.read)NNr`) rhrirjrkrcpropertyrr __nonzero__rr r r r!rs  rcCsx|r:tdt|t|}d|}dd|}|dkr(td|||ddn td|||ddtjdSdS) zProgress bar callback function that calculates the percentage of current completion :param consumed_bytes: Amount of data that has been uploaded/downloaded :param total_bytes: According to the total amount d*rz {}%[{}->{}] r)rz {}%[{}->{}]N)rWfloatprintr sysstdoutflush)r r ZrateZstart_progressZ end_progressr r r!default_progress_callbacks r)rr rrg)Tr`)Trk __future__rrbuiltinsrrZ future.utilsrrrrr rrrmrrrr urllib.parser r7Z Crypto.Cipherr Z baidubce.httpr rstructr8r6r'rrZHTTP_PROTOCOL_HEADr"r%r+r0r3r5r9rBrFrVr^objectr_rlrwr|r~rrrrrrrrrrrrrrrrrrrrrrrrrrrr r _CHUNK_SIZErrr r r r!s         $ (&  5'5   ! )  '    ;