o )j"@s|ddlZddlZddlZddlZddlmZmZmZddl m Z ddl m Z m Z mZddlZddlmZddlmZddlmZddlmZeZejjd d ZeZeZeZda deddfd ee!e fd e!d e!dee!e"dfde e!de e!de e!fddZ#dddddefddZ$dddddddefddZ%ddddddefddZ&ddZ'Gdd d Z(e(j)fd!d"Z*d#d$Z+dS)%N)ManagerProcessValue)Path)ListOptionalUnionHubApi)ModelVisibility)DEFAULT_REPOSITORY_REVISION) get_logger) max_workerspath_or_fileobj path_in_reporepo_idtokenrevisioncommit_messagecommit_descriptioncCstj|sdSddlm}|}|||sd}|r"|d|}tN} ddlm } | | ||d} |r>tj | |} n| } tj | dd tj |r_tj | tj |} t|| ntj|| dd | |WddS1swwYdS) zPush files to model hub incrementally This function if used for patch_hub, user is not recommended to call this. This function will be merged to push_to_hub in later sprints. Nrr zUpdating files ) Repository)rT)exist_ok) dirs_exist_ok)ospathexistsZ modelscoper logintempfileTemporaryDirectoryZmodelscope.hub.repositoryrjoinmakedirsisfilebasenameshutilcopyfilecopytreepush)rrrrrrrr apiZtemp_cache_dirrrepoZ sub_folder dest_filer,[/var/www/html/Deteccion_Ine/venv/lib/python3.10/site-packages/modelscope/hub/push_to_hub.py_push_files_to_hubs0         "r.Tc Csz.t} | || j|||stjntj||||||d |p d}td|d|WdStyO} zt d|d|d| WYd} ~ dSd} ~ ww) N)Z visibilityZ chinese_namertagZoriginal_model_idignore_file_patternrzNo commit messagez!Successfully upload the model to z with message: Tz#Error happens when uploading model z: F) r rZ push_modelr ZPUBLICZPRIVATEloggerinfo Exceptionerror) repo_name output_dirrprivaterr0 source_repor1rr)er,r,r-_api_push_to_hubDs8   r;c Cs|dur tjd}|durtjd}|dusJ|dus"Jdtj|s*Jdt|vsAdt|vsAdt|vsAJtd|d |d |t|D]} t ||||||||| rdd SqSd S) a Args: repo_name: The repo name for the modelhub repo output_dir: The local output_dir for the checkpoint token: The user api token, function will check the `MODELSCOPE_API_TOKEN` variable if this argument is None private: If is a private repo, default True retry: Retry times if something error in uploading, default 3 commit_message: The commit message tag: The tag of this commit source_repo: The source repo (model id) which this model comes from ignore_file_pattern: The file pattern to be ignored in uploading. revision: The branch to commit to Returns: The boolean value to represent whether the model is uploaded. NMODELSCOPE_API_TOKENUPLOAD_IGNORE_FILE_PATTERNUEither pass in a token or to set `MODELSCOPE_API_TOKEN` in the environment variables.configuration.jsonconfiguration.yamlconfiguration.yml Uploading  to  with message TF) renvirongetrisdirlistdirr2r3ranger;) r6r7rr8retryrr0r9r1rir,r,r- push_to_hubgs(     rMc Cs|dur tjd}|durtjd}|dusJ|dus"Jdtj|s*Jdt|vsAdt|vsAdt|vsAJtd|d |d |t t ||||||||| S) az Args: repo_name: The repo name for the modelhub repo output_dir: The local output_dir for the checkpoint token: The user api token, function will check the `MODELSCOPE_API_TOKEN` variable if this argument is None private: If is a private repo, default True commit_message: The commit message tag: The tag of this commit source_repo: The source repo (model id) which this model comes from ignore_file_pattern: The file pattern to be ignored in uploading revision: The branch to commit to Returns: A handler to check the result and the status Nr=r>r?r@rArBrCrDrE) rrFrGrrHrIr2r3 _executorZsubmitr;) r6r7rr8rr0r9r1rr,r,r-push_to_hub_asyncs     rOc Cs d|_|}t|d|_|ddsO|dd}|d}ztdi||r6tj|r6t |Wnt yN}z t |WYd}~nd}~wwdSq)NTFdone delete_dirr7r,) valuerGr2r3poprMrrrr%rmtreer4r5)qbitemrQr7r:r,r,r- submit_tasks&     rXc@seZdZdZdZdS)UploadStrategycancelwaitN)__name__ __module__ __qualname__rZr[r,r,r,r-rYsrYcKs|dur t|dksJdtdurta|tvr:tt|<tddt|<ttt|t|fd}| |t |<t|}t|}| ddrO| |dS|j re|tjkretd| dd dS| |dS) Nrz"Please specify a valid queue name!rVF)targetargsrPz:Another uploading is running, this uploading with message rz will be canceled.)len_managerr_queuesQueuer_flagsrrXstart_tasksrGputrRrYrZr2r5) queue_nameZstrategykwargsprocessqueueflagr,r,r-push_to_hub_in_queues8   rncCs8t|d}|dur dS|t|t|dS)N)rgrSr!rcre)rirkr,r,r- wait_for_dones  ro),concurrent.futuresZ concurrentrr%rmultiprocessingrrrpathlibrtypingrrrjsonZmodelscope.hub.apir Zmodelscope.hub.constantsr Zmodelscope.utils.constantr Zmodelscope.utils.loggerr r2ZfuturesProcessPoolExecutorrNdictrcrergrbstrboolr.r;rMrOrXrYrZrnror,r,r,r-s        , % / (