o )j4@sddlZddlZddlmZddlmZmZmZddlm Z m Z m Z ddl m Z ddlmZddlmZe ZGd d d ZGd d d ZdS) N)Optional)GitErrorInvalidParameterNotLoginException)DEFAULT_DATASET_REVISIONDEFAULT_REPOSITORY_REVISIONMASTER_MODEL_BRANCH) get_logger)GitCommandWrapper) get_endpointc@seZdZdZedddfdededeedeedeedeef d d Zd d Zd dZ d&dedefddZ defddZ eedfdedeedeedee fddZ efded ed!eefd"d#Zefded ed!eefd$d%ZdS)' Repositoryz8A local representation of the model git repository. N model_dir clone_fromrevision auth_tokengit_pathendpointc Cs&||_||_tj||_tj||_|sd}t|ddl m }|r(||_ n| |_ t } | s9tdt ||_tj|jdd||} t|jrc|} |j| } | rc| | krcdS|j|j|j | |j|| rz| |j|j|j|j|j r|j|j|j dSdS)aInstantiate a Repository object by cloning the remote ModelScopeHub repo Args: model_dir (str): The model root directory. clone_from (str): model id in ModelScope-hub from which git clone revision (str, optional): revision of the model you want to clone from. Can be any of a branch, tag or commit hash auth_token (str, optional): token obtained when calling `HubApi.login()`. Usually you can safely ignore the parameter as the token is already saved when you login the first time, if None, we will use saved token. git_path (str, optional): The git command line path, if None, we use 'git' endpoint (str, optional): The ModelScope endpoint URL. If None, use default endpoint. Raises: InvalidParameter: revision is None. 0a non-default value of revision cannot be empty.rModelScopeConfigz)git lfs is not installed, please install.Texist_okN) _endpointrospathdirnamemodel_base_dirbasenamemodel_repo_namermodelscope.hub.apirr get_tokenr Zis_lfs_installedloggererror git_wrappermakedirs_get_model_id_urllistdir_get_remote_urlremove_token_from_urlcloneZgit_lfs_install add_user_infoconfig_auth_token) selfrrrrrrerr_msgrr$url remote_urlr1Z/var/www/html/Deteccion_Ine/venv/lib/python3.10/site-packages/modelscope/hub/repository.py__init__sB          zRepository.__init__cCs&|jr|jnt}|d|d}|S)N/.gitrr )r-Zmodel_idrr/r1r1r2r&TszRepository._get_model_id_urlcC.z |j|j}W|Styd}Y|SwN)r$get_repo_remote_urlrrr-remoter1r1r2r(Y zRepository._get_remote_urloriginmasterr;branchcCs|jj|j||ddS)zPull remote branch Args: remote (str, optional): The remote name. Defaults to 'origin'. branch (str, optional): The remote branch. Defaults to 'master'. )r;r?N)r$pullr)r-r;r?r1r1r2r@`szRepository.pullfile_name_suffixcCs"td|tj|jdfdS)zAdd file suffix to lfs list. Args: file_name_suffix (str): The file name suffix. examples '*.safetensors' z5printf ' %s filter=lfs diff=lfs merge=lfs -text '>>%sz.gitattributesN)rsystemrjoinr)r-rAr1r1r2 add_lfs_typeis zRepository.add_lfs_typeFcommit_message local_branch remote_branchforcecCstjdtdd |dust|tsd}t|t|ts!td|js(td|j |j |j|j |j |j|j |j }|j j|j dd|j |j ||j j|j |j|||d dS NzThis function is deprecated and will be removed in future versions. Please use git command directly or use HubApi().upload_folder instead) stacklevelz commit_message must be provided!zforce must be boolz'Must login to push, please login first.T) all_files)repo_dirtokenr/rFrG)warningswarnDeprecationWarning isinstancestrrboolrrr$r,rr+rrr9addcommitpush)r-rErFrGrHmsgr/r1r1r2rWts6    zRepository.pushtag_namemessagerefcCsR|dus|dkrd}t||dus|dkrd}t||jj|j|||ddS)aCreate a new tag. Args: tag_name (str): The name of the tag message (str): The tag message. ref (str, optional): The tag reference, can be commit id or branch. Raises: InvalidParameter: no commit message. NzFWe use tag-based revision, therefore tag_name cannot be None or empty.z=We use annotated tag, therefore message cannot None or empty.)rMrYrZr[)rr$tagr)r-rYrZr[rXr1r1r2r]s zRepository.tagcCs$|||||jj|j|ddS)zCreate tag and push to remote Args: tag_name (str): The name of the tag message (str): The tag message. ref (str, optional): The tag ref, can be commit id or branch. Defaults to MASTER_MODEL_BRANCH. )rMrYN)r]r$Zpush_tagr)r-rYrZr[r1r1r2 tag_and_pushs zRepository.tag_and_push)r=r>)__name__ __module__ __qualname____doc__rrSrr3r&r(r@rDrTrWrr]r^r1r1r1r2r sf >   2 r c@seZdZdZedddfdededeedeedeedeef d d Zd efd d Zedfdedeedee fddZ ddZ ddZ dS)DatasetRepositoryzEA local representation of the dataset (metadata) git repository. N repo_work_dir dataset_idrrrrc Cs||_||_|r t|tsd}t||d|_|js"d}t|tj |j|_ tj |j|_ |s:d}t|||_ ddlm}|rI||_n||_t||_tj|jdd|j|d |_d S) a Instantiate a Dataset Repository object by cloning the remote ModelScope dataset repo Args: repo_work_dir (str): The dataset repo root directory. dataset_id (str): dataset id in ModelScope from which git clone revision (str, optional): revision of the dataset you want to clone from. Can be any of a branch, tag or commit hash auth_token (str, optional): token obtained when calling `HubApi.login()`. Usually you can safely ignore the parameter as the token is already saved when you login the first time, if None, we will use saved token. git_path (str, optional): The git command line path, if None, we use 'git' endpoint (str, optional): The ModelScope endpoint URL. If None, use default endpoint. Raises: InvalidParameter: parameter invalid. z"dataset_work_dir must be provided!r4z%dataset_work_dir can not be root dir!rrrTr)reN)rrerRrSrrstriprdrrr repo_base_dirr repo_namerr rrr!r r$r% _get_repo_urlrepo_url) r-rdrerrrrr.rr1r1r2r3s,    zDatasetRepository.__init__returncCsht|jr|}|j|}|r||jkrdStd |j|j |j |j |j|j |j|jS)Nr\zCloning repo from {} )rr'rdr(r$r)rjr"infoformatr*rgrrhr)r-r0r1r1r2r*s   zDatasetRepository.cloneFrEr?rHcCstjdtdd |dust|tsd}t|t|ts!td|js(td|j |j |j|j |j |j|}|j |}|j |j |j j|j dd|j |j ||j j|j |j|||d dSrI)rOrPrQrRrSrrTrrr$r,rdr+rgrhr(r)r@rUrVrW)r-rEr?rHrXr0r1r1r2rWs6   zDatasetRepository.pushcCs"|jr|jnt}|d|dS)Nz /datasets/r5r6)r-rerr1r1r2riCszDatasetRepository._get_repo_urlcCr7r8)r$r9rdrr:r1r1r2r(Gr<z!DatasetRepository._get_remote_url) r_r`rarbrrSrr3r*rTrWrir(r1r1r1r2rcs> 2 2 rc)rrOtypingrZmodelscope.hub.errorsrrrZmodelscope.utils.constantrrrZmodelscope.utils.loggerr gitr Z utils.utilsr r"r rcr1r1r1r2s    <