o 0j,h$@sdZddlZddlZddlmZddlZddlmZddlm Z m Z m Z m Z m Z mZddlmZmZddlmZmZmZmZdd lmZmZdd lmZmZeeZed d Z d e!de"fddZ#de!de$e!e!ffddZ%e j&dgdd   ddee!ej'dd fdee"ej(ddd fdee"ej(ddd fd eddf d!dZ)de!de"fd"d#Z*e j&d$gd%d      ddee!dBej'd&d fd'ee"ej(d(d)d*d fd+ee"ej(d,d-d fd.ee"ej(d/d0d1d fd2ed eddfd3d4Z+d5e!dBd2e!dBd'e"d+e"d.e"d e!dBddfd6d7Z,de!d'e"d+e"d.e"d e!dBddf d8d9Z-e j&d:d;dd:Z.e j&d?gd@d   ddee!ej'd=d fdAee"ej(dBdCdDd fdEee"ej(dFdGd fd eddf dHd?Z/e j&dIgdJd      ddee!ej'dKd fd.ee"ej(d/d0dLd fdAee"ej(dBdCdDd fdMee"ej(dNdOd fdPee0e!dBej(dQd fdRee0e!dBej(dSd fd eddfdTdUZ1e j&dVgdWd ddXee!ej'dYd fdZee!ej'd[d fd eddfd\dVZ2e j&d]gd^d               dd_ee!dBej'd`d fdaee!dBej'dbd fd?ee"ej(dcd fddee"ej(dedfd fdgee"ej(dhdid fdjee!dBej(dkd fdlee!dBej(dmd fdMee"ej(dNdnd fdPee0e!dBej(dod fdRee0e!dBej(dpd fdqee!dBej(drd fdsee"ej(dtdud fdvee"ej(dwdxd fdyee"ej(dzd{d|d fd eddf d}d]Z3e j&d~gdd  ddee!ej'dd fdee!dBej'dd fd eddfdd~Z4dS)z7Contains commands to interact with buckets via the CLI.N) Annotated)logging) BUCKET_PREFIX BucketFile FilterMatcher_is_bucket_path_parse_bucket_path_split_bucket_id_and_prefix)SoftTemporaryDirectorydisable_progress_bars) SearchOptTokenOpt get_hf_api typer_factory) format_sizeprint_file_listing)OutputFormatWithAutooutz"Commands to interact with buckets.)helppathreturncCs |dS)Nzhf://) startswithrr\/var/www/html/Deteccion_Ine/venv/lib/python3.10/site-packages/huggingface_hub/cli/buckets.py _is_hf_handle4s rargumentc CsB|tr t|Szt|WSty td|dtdw)zParse a bucket argument accepting both 'namespace/name(/prefix)' and 'hf://buckets/namespace/name(/prefix)'. Returns: tuple: (bucket_id, prefix) where bucket_id is "namespace/bucket_name" and prefix may be empty string. zInvalid bucket argument: -. Must be in format namespace/bucket_name or znamespace/bucket_name)rrrr ValueError)rrrr_parse_bucket_argument8s   r create)zhf buckets create my-bucketz hf buckets create user/my-bucketz-hf buckets create hf://buckets/user/my-bucketz*hf buckets create user/my-bucket --privatez+hf buckets create user/my-bucket --exist-ok)nameZexamplesF bucket_idzTBucket ID: bucket_name, namespace/bucket_name, or hf://buckets/namespace/bucket_nameprivatez --privatezCreate a private bucket.exist_okz --exist-okz3Do not raise an error if the bucket already exists.tokenc Cst|d}|tr6zt|\}}Wnty%}ztt|d}~ww|r4td|dtd|}|j||r=|nd|d}t j d|j |j ddS) zCreate a new bucket.r&Nz-Cannot specify a prefix for bucket creation: . Use namespace/bucket_name or namespace/bucket_name.)r$r%zBucket created)handleurl) rrrr rtyper BadParameterstrZ create_bucketrresultr*r+) r#r$r%r&api parsed_idprefixeZ bucket_urlrrrr!Is* "  cCs.|tr|ttd}d|vS|}d|vS)zFCheck if argument is a bucket ID (namespace/name) vs just a namespace.N/)rrlen)rrrrr _is_bucket_ids r6z list | ls) zhf buckets listzhf buckets list huggingfacez$hf buckets list --search "my-prefix"zhf buckets list user/my-bucketz!hf buckets list user/my-bucket -Rz!hf buckets list user/my-bucket -hz%hf buckets list user/my-bucket --treez(hf buckets list user/my-bucket --tree -hz+hf buckets list hf://buckets/user/my-bucketz%hf buckets list user/my-bucket/sub -RzyNamespace (user or org) to list buckets, or bucket ID (namespace/bucket_name(/prefix) or hf://buckets/...) to list files.human_readablez--human-readablez-hz$Show sizes in human readable format.as_treez--treez3List files in tree format (only for listing files). recursivez --recursivez-Rz0List files recursively (only for listing files).searchcCsT|duot|}|r|durtdt|||||ddSt||||||ddS)zList buckets or files in a bucket. When called with no argument or a namespace, lists buckets. When called with a bucket ID (namespace/bucket_name), lists files in the bucket. Nz'Cannot use --search when listing files.)rr7r8r9r&) namespacer:r7r8r9r&)r6r,r- _list_files _list_buckets)rr7r8r9r:r&Z is_file_moderrrlist_cmds&9   r>r;cs|rtd|rtd|dur$|tr$|ttd}|d}t|d}fdd|j||dD}tj |d d d d dS) zList buckets in a namespace.z'Cannot use --tree when listing buckets.z,Cannot use --recursive when listing buckets.Nr4r'cs6g|]}|j|jrt|jn|j|j|jdqS))idr$size total_files created_at)r?r$rr@rArB).0bucketr7rr sz!_list_buckets..)r;r:right)r@rA)Z alignments) r,r-rrr5rstriprZ list_bucketsrtable)r;r:r7r8r9r&r0itemsrrErr=s       r=c Cs|r tjtjkr tdt|d}zt|\}}Wnty-}ztt |d}~wwt |j ||p5d|d} t | |||ddS)zList files in a bucket.z%Cannot use --tree with --format json.r'Nr2r9)r7r8r9) rmoderjsonr,r-rr rr.listlist_bucket_treer) rr7r8r9r&r0r#r2r3rJrrrr<s"  r<infozhf buckets info user/my-bucketz+hf buckets info hf://buckets/user/my-bucketzFBucket ID: namespace/bucket_name or hf://buckets/namespace/bucket_namec Cs^t|d}zt|\}}Wnty }ztt|d}~ww||}tj|dddS)zGet info about a bucket.r'Nr?)Zid_key) rr rr,r-r.Z bucket_infordict)r#r&r0r1_r3rDrrrrPs  delete)z hf buckets delete user/my-bucketz-hf buckets delete hf://buckets/user/my-bucketz&hf buckets delete user/my-bucket --yesz-hf buckets delete user/my-bucket --missing-okyesz--yesz-yzSkip confirmation prompt. missing_okz --missing-okz3Do not raise an error if the bucket does not exist.c Cs|tr2zt|\}}Wnty }ztt|d}~ww|r/td|dtd|}nd|vrBtd|dtdtjd|d |d t |d }|j ||d tj d |ddS)zDelete a bucket. This deletes the entire bucket and all its contents. Use `hf buckets rm` to remove individual files. Nz-Cannot specify a prefix for bucket deletion: r(r)r4zInvalid bucket ID: rz(Are you sure you want to delete bucket ''?rTr')rUzBucket deletedr#) rrr rr,r-r.rconfirmrZ delete_bucketr/)r#rTrUr&r1r2r3r0rrrrS6s0 % z remove | rm)z)hf buckets remove user/my-bucket/file.txtz2hf buckets rm hf://buckets/user/my-bucket/file.txtz.hf buckets rm user/my-bucket/logs/ --recursivez:hf buckets rm user/my-bucket --recursive --include "*.tmp"z8hf buckets rm user/my-bucket/data/ --recursive --dry-runzBucket path: namespace/bucket_name/path or hf://buckets/namespace/bucket_name/path. With --recursive, namespace/bucket_name is also accepted to target all files.z0Remove files recursively under the given prefix.dry_runz --dry-runz8Preview what would be deleted without actually deleting.includezQInclude only files matching pattern (can specify multiple). Requires --recursive.excludezLExclude files matching pattern (can specify multiple). Requires --recursive.c szt|\}Wnty}ztt|d}~ww|dkr.|s.tddd|s2|r9|s9tdt|d} |r td} g} | j| d pQdd d D]} t | t rl| | | d t| d qU| d t| d |s||rt||dfdd| D} n| } dd| D}tdd| D}t|d d}|stddSt|d|}|s|stddd|Dtjd|dddd|rtdfdd|Dtd|d dS| j|d!tjd"|dd#t||d$dS| d }|std%|r0td&td |td'dStjd(|d)d|d| j|gd!tjd*|d+dS),zbRemove files from a bucket. To delete an entire bucket, use `hf buckets delete` instead. Nz?No file path specified. To remove files, provide a path (e.g. 'zd/FILE') or use --recursive to remove all files. To delete the entire bucket, use `hf buckets delete z`.z,--include and --exclude require --recursive.r'zListing files from remoter4TrKzListing files from remote (z files))Zinclude_patternsZexclude_patternscsg|] }|jr|qSr)matchesrrCf)matcherrrrFszremove..cSsg|]}|jqSrrr_rrrrFscss|]}|jVqdSN)r@r_rrr szremove..rEzNo files to remove.z file(s) totaling  css|]}d|VqdS)z NrrCrrrrrcszRemove z from 'rVFrWc3s$|] }dtd|VqdS)delete: r4N)rrerXrrrcs"z (dry run) z would be removed.)rSzRemoved ')r#Z files_deletedr@zFile path cannot be empty.rfz"(dry run) 1 file would be removed.zRemove 'z' from 'z File removed)rr#)r rr,r-r.rrstatusrOrH isinstancerappendupdater5donersumrtextjoinrYbatch_bucket_filesr/r)rr9rTrZr[r\r&r2r3r0rh all_filesitemZ matched_files file_pathsZ total_sizesize_strZ count_label file_pathr)r#rarremovess=              rvmove)z/hf buckets move user/old-bucket user/new-bucketz/hf buckets move user/my-bucket my-org/my-bucketzIhf buckets move hf://buckets/user/old-bucket hf://buckets/user/new-bucketfrom_idzMSource bucket ID: namespace/bucket_name or hf://buckets/namespace/bucket_nameto_idzRDestination bucket ID: namespace/bucket_name or hf://buckets/namespace/bucket_namecCs|t|\}}|rtd|dtdt|\}}|r(td|dtdt|d}|j||dtjd||ddS)z2Move (rename) a bucket to a new name or namespace.z)Cannot specify a prefix for bucket move: r(r)r')rxryz Bucket movedN)r r,r-rrZ move_bucketrr/)rxryr&Zparsed_from_idZ from_prefixZ parsed_to_idZ to_prefixr0rrrrws"   sync)z2hf buckets sync ./data hf://buckets/user/my-bucketz2hf buckets sync hf://buckets/user/my-bucket ./dataz;hf buckets sync ./data hf://buckets/user/my-bucket --deletez^hf buckets sync hf://buckets/user/my-bucket ./data --include "*.safetensors" --exclude "*.tmp"zIhf buckets sync ./data hf://buckets/user/my-bucket --plan sync-plan.jsonlz'hf buckets sync --apply sync-plan.jsonlzhf buckets cp my-config.json hf://buckets/user/my-bucket/logs/zKhf buckets cp my-config.json hf://buckets/user/my-bucket/remote-config.jsonz7hf buckets cp - hf://buckets/user/my-bucket/config.jsonzchf buckets cp hf://buckets/user/my-bucket/logs hf://buckets/user/archive-bucket/ # nests logs/ dirzihf buckets cp hf://buckets/user/my-bucket/logs/ hf://buckets/user/archive-bucket/ # copies contents onlyzehf buckets cp hf://datasets/user/my-dataset/processed/ hf://buckets/user/my-bucket/dataset/processed/srczMSource: local file, any hf:// handle (model, dataset, bucket), or - for stdindstz@Destination: local path, bucket hf://... handle, or - for stdoutc Cst|d}t|}|duot|}t|}|duot|}|dk}|dk} |rO|rOz|||WntyD} ztt| d} ~ wwtj d||ddS|sc|sc|sc|dur^tdtd|rl|sltd|r|rt |\} } | d ks| d rtd | r|std |s|st j |rtd |rmt |\} } | d ks| d rtd| d dd}| r$tVtB}t j ||}|| | |fgt|d}|d}rtjj||d}sWdn1swYWdn 1s wYWdn 1swYdSdS|dur,|}nt j |s@| t js@| d rHt j ||}n|}t j |}|rZt j|dd|| | |fgtj d||ddS|rt |\} }tjj}|j| ||fgdtj dd|ddSt j |std|t |\} } | d krt j |}n| d r| t j |}n| }|j| ||fgdtj d|t!| d |ddS)zCopy files to or from buckets.r'N-ZCopied)rrz2Missing destination. Provide a bucket path as DST.z;One of SRC or DST must be a bucket path (hf://buckets/...).z+Stdin upload requires a bucket destination.r]r4zAStdin upload requires a full destination path including filename.z"Cannot pipe to stdout for uploads.zNSource must be a file, not a directory. Use `hf buckets sync` for directories.zJSource path must include a file name, not just a bucket or directory path.r rbiHT)r%Z Downloaded)addZUploadedstdinzSource file not found: )"rrrZ copy_filesrr,r-r.rr/rendswithosrisdirrsplitr r roZdownload_bucket_filesopenreadsysstdoutbufferwritesepdirnamemakedirsrrpisfilebasenamer)rrr&r0Z src_is_hfZ dst_is_hfZ src_is_bucketZ dst_is_bucketZ src_is_stdinZ dst_is_stdoutr3rRr2r#filenameZtmp_dirZtmp_pathr`chunkZ local_path parent_dirZ remote_pathdatarrrrs            & (      ")FFN)NFFFNNrb)FFFNNN)NNFFFNNFNNNFFFN)NN)5__doc__rrtypingrr,Zhuggingface_hubrZhuggingface_hub._bucketsrrrrrr Zhuggingface_hub.utilsr r Z _cli_utilsr rrrZ _file_listingrr_outputrrZ get_logger__name__loggerZ buckets_clir.boolrtupler commandZArgumentOptionr!r6r>r=r<rPrSrNrvrwrzrrrrrs       .  !"# A #   4 ! ' -.   * !'-4 : @FMT\] z