o )jb @s<ddlZddlmZmZddlmZeZGdddZdS)N)ListTuple) get_loggerc@seZdZdZdedefddZedefddZd edefd d Zd ed e fd dZ d ede de fddZ d efddZdedeee ee ffddZdedefddZdS) BatchTrackerzThread-safe tracker for pre-assigned upload batches. Files are assigned to batches by sorted index (file_index // batch_size). Upload threads record results; main thread waits for batches in order. total_files batch_sizecCs||_|dkr|d|dnd|_ddt|jD|_ddt|jD|_g|_t|jD]}||}t|||}|j||q0ddt|jD|_t |_ dS)NrcSg|]}gqSr .0_r r _/var/www/html/Deteccion_Ine/venv/lib/python3.10/site-packages/modelscope/hub/upload_pipeline.py z)BatchTracker.__init__..cSr r r r r r rrrcSsg|]}tqSr ) threadingEventr r r rr!s) _batch_size _num_batchesrange_batch_results_batch_failures_batch_expectedminappend _batch_eventsrLock_lock)selfrristartendr r r__init__s4zBatchTracker.__init__returncCs|jSN)r)rr r r num_batches&szBatchTracker.num_batches file_indexcCs ||jSr$)r)rr&r r r batch_index*s zBatchTracker.batch_indexresultcCsn||}|j%|j||||r%|j|WddSWddS1s0wYdSr$)r'rrr_is_batch_completerset)rr&r(idxr r rrecord_success-s  "zBatchTracker.record_successitemerrorcCsr||}|j'|j|||f||r'|j|WddSWddS1s2wYdSr$)r'rrrr)rr*)rr&r-r.r+r r rrecord_failure4s  "zBatchTracker.record_failurecCsp||}|j&|j|d8<||r&|j|WddSWddS1s1wYdS)zMark a file as skipped (already committed). Decrements the batch's expected count so _is_batch_complete uses the correct target. When all files in a batch are skipped, the batch event is set automatically. rN)r'rrr)rr*)rr&r+r r rmark_file_skipped;s  "zBatchTracker.mark_file_skipped batch_idxcCsT|j||jt|j|t|j|fWdS1s#wYdS)axWait for a batch to complete. Blocks indefinitely until all files in the batch have reported success or failure. Per-blob timeouts (UPLOAD_BLOB_TIMEOUT) prevent individual uploads from hanging forever. Args: batch_idx: Index of the batch to wait for. Returns: Tuple of (successful_results, failures). N)rwaitrlistrr)rr1r r rwait_for_batchHs  $zBatchTracker.wait_for_batchcCs*t|j|t|j|}||j|kS)z Must be called under self._lock.)lenrrr)rr1countr r rr)ZszBatchTracker._is_batch_completeN)__name__ __module__ __qualname____doc__intr"propertyr%r'dictr,tuple Exceptionr/r0rrr4boolr)r r r rr s" r)rtypingrrZmodelscope.utils.loggerrloggerrr r r rs