o 0j@sddlmZddlZddlZddlZddlZddlmZddlm Z m Z m Z m Z m Z mZddlZddlmZddlmZddlmZd d lmZd d lmZd d lmZmZd d lmZddl m!Z!ddl"m#Z#ddl$m%Z%ddl&m'Z'ddl(m)Z)ddl*m+Z+m,Z,ddl-m.Z.m/Z/m0Z0ddl1m2Z2m3Z3m4Z4m5Z5m6Z6m7Z7m8Z8m9Z9m:Z:gdZ;ejedGddde>Z?edGd d!d!e>Z@dS)") annotationsN)chain)AnyDictListOptionalTupleUnion)Image)logging)pipeline_requires_extra)ImageBatchSampler) ReadImage) HPIConfigPaddlePredictorOption) benchmark)(AutoParallelImageSimpleInferencePipeline) BasePipeline) CropByBoxes)merge_tables_across_pages)assign_levels_to_parsing_res)construct_img_path gather_imgs) BaseResultPaddleOCRVLBlockPaddleOCRVLResult) convert_otsl_to_html crop_marginfilter_overlap_boxes merge_blockspost_process_for_spottingpre_process_for_spottingtokenize_figure_of_tabletruncate_repetitive_contentuntokenize_figure_of_table)image header_image footer_imagecseZdZdZdddddddddXfddZddZ dYdZd&d'Zd[d)d*Z      +d\d,d-Z           +            d]d^dFdGZ d_dKdLZ   d`dadQdRZ   d`dbdVdWZ Z S)c_PaddleOCRVLPipelinez_PaddleOCRVLPipeline PipelineNFT)deviceengine engine_config pp_optionuse_hpip hpi_configinitial_predictorconfigrr- Optional[str]r.r/Optional[Dict[str, Any]]r0Optional[PaddlePredictorOption]r1boolr2*Optional[Union[Dict[str, Any], HPIConfig]]r3returnNonec stjd%||||||d| |r|dd|_|jr.|didddi} || |_|dd|_|jr|d id d d i} | d d} | durR| dvsVJdi} | dd}durf|| d<| dd}durt|| d<| dd}dur|| d<| dd}dur|| d<|j| fi| |_|dd|_ |dd|_ |d idd di}|||_ |dd|_ |dd|_ t|ddd|_tdd |_t|_|d!d|_|d"d|_|d#gd$|_dSdS)&aInitializes the PaddleOCR-VL pipeline. Args: config (Dict): Configuration dictionary containing various settings. device (Optional[str], optional): The device to use for prediction. Defaults to `None`. engine (Optional[str], optional): Inference engine. Defaults to `None`. engine_config (Optional[Dict[str, Any]], optional): Engine-specific config. Defaults to `None`. pp_option (Optional[PaddlePredictorOption], optional): Paddle predictor options. Defaults to `None`. use_hpip (bool, optional): Whether to use HPIP. Defaults to `False`. hpi_config (Optional[Union[Dict[str, Any], HPIConfig]], optional): HPIP configuration. Defaults to `None`. initial_predictor (bool, optional): Whether to initialize predictors. Defaults to `True`. )r-r.r/r0r1r2use_doc_preprocessorTZ SubPipelinesZDocPreprocessorZpipeline_config_errorz+config error for doc_preprocessor_pipeline!use_layout_detectionZ SubModulesZLayoutDetectionZmodel_config_errorz"config error for layout_det_model! model_nameN)zPP-DocLayoutV2zPP-DocLayoutV3z3model_name must be PP-DocLayoutV2 or PP-DocLayoutV3 threshold layout_nmslayout_unclip_ratiolayout_merge_bboxes_modeuse_chart_recognitionFuse_seal_recognitionZ VLRecognitionzconfig error for vl_rec_model!format_block_contentuse_ocr_for_image_block batch_sizer)rGZBGR)format use_queuesmerge_layout_blocksmarkdown_ignore_labels)numberZfootnoteheaderr*footerr+Z aside_text)super__init__getr<Zcreate_pipelinedoc_preprocessor_pipeliner=Z create_modellayout_det_modelrCrD vl_rec_modelrErFr batch_samplerr img_readerr crop_by_boxesrIrJrK)selfr4r-r.r/r0r1r2r3kwargsZdoc_preprocessor_configZlayout_det_configr>Z layout_kwargsr?r@rArBZ vl_rec_config __class__rOr/var/www/html/Deteccion_Ine/venv/lib/python3.10/site-packages/paddlex/inference/pipelines/paddleocr_vl/pipeline.pyrQ:s         z_PaddleOCRVLPipeline.__init__cCst|dr |jdSdS)NrU)hasattrrUcloserYrOrOr]r_s z_PaddleOCRVLPipeline.closeuse_doc_orientation_classifyUnion[bool, None]use_doc_unwarpingr=rCrDrFrErJrKOptional[List[str]]dictc Cs|dur |dur |j} n |dus|durd} nd} |dur |j}|dur'|j}|dur.|j}|dur5|j}|dur<|j}|durC|j}| durJ|j} t| ||||||| dS)a Get the model settings based on the provided parameters or default values. Args: use_doc_orientation_classify (Union[bool, None]): Enables document orientation classification if True. Defaults to system setting if None. use_doc_unwarping (Union[bool, None]): Enables document unwarping if True. Defaults to system setting if None. Returns: dict: A dictionary containing the model settings. NTF)r<r=rCrDrFrErJrK) r<r=rCrDrFrErJrKre) rYrarcr=rCrDrFrErJrKr<rOrOr]get_model_settingss:z'_PaddleOCRVLPipeline.get_model_settings input_paramscCs |dr|jstddSdS)a4 Check if the input parameters are valid based on the initialized models. Args: input_params (Dict): A dictionary containing input parameters. Returns: bool: True if all required models are initialized according to input parameters, False otherwise. r<zRSet use_doc_preprocessor, but the models for doc preprocessor are not initialized.FT)r<r error)rYrgrOrOr]check_model_settings_valids  z/_PaddleOCRVLPipeline.check_model_settings_validautoc ? Csg} d} t} |dd} | dur| nd}|dd}|dur!|nd}i}i}i}tdg}|r2gnt}|sB|dg7}|dg7}|sI|dg7}tt|||D]#\}\}}}t|| }|d}|||| }~~|rvt||d gd }| |t|D]\}}|d }|d }||vrs|dursi} d }!|d|} |d|}g}"|d krd}!t ||d|\}} }"|d|} |d|}nj|dkr|rd}!|d|} |d|}nUd|vr|dkrd}!t |}#|#j \}$}%}&|$dkr|%dkr|#}|d|} |d|}n(|dkrd}!d} d} d }t |}n|dkr/|r/d!}!|d"|} |d#|}| |f}'|'|vrBggggd$d%||'<||'d& |||'d' |!||'d( | ||'d) ||f|'|||f<| |"q~qQ~~|duri}n |d*ddurd+|d*<|D]>}'|'\} }d| |d,|}(||'d&}||'d'})t|jjd-d.t||)Dfd/| rdndi|(}*~~)|*||'d0<qg}+g},g}-g}.t| D]r\}}g}/g}0i}1t|D]9\}}|d }|d}2|d }d1}3i} ||f|vr|||f}'||'}4|4d2}5|5t|4d)kr'|4d)|5||fks)J|4d0|5}6|4d&|5}7|4d(|5} |5d37}5|5|4d2<|7|6d4<|6d5d1}8|8durTd1}8|d kr[d6nd7}9t|8|9d8}8d9|8vrmd:|8vswd;|8vrd<|8vr|8d=d1}8|8d9d>d:d?d@d;dAd<d;dBd|>dH}|>dI} t |j!| ||_!q&|+ |/|, |0|- |1~/~0~1q|+|,|-|fS)JNF min_pixelsi max_pixelsiPsealchartboxestable)Znon_merge_labelsimglabelzOCR:Zocr_min_pixelsZocr_max_pixelszTable Recognition:boxZtable_min_pixelsZtable_max_pixelszChart Recognition:Zchart_min_pixelsZchart_max_pixelsformulaZformula_numberzFormula Recognition:rZformula_min_pixelsZformula_max_pixelsspottingz Spotting:TizSeal Recognition:Zseal_min_pixelsZseal_max_pixelsr)imagesqueriesfigure_token_maps vlm_block_idscurr_vlm_block_idxrvrwrxrymax_new_tokensi)Z use_cacherkrlcSsg|] \}}||dqS))r)queryrO).0r)r|rOrOr] s zC_PaddleOCRVLPipeline.get_layout_parsing_results..Zskip_special_tokensZ vlm_resultsrzrr)resulti2) min_countz\(z\)z\[z\]$z $ z $z\[\[z\]\]z $$ group_idpolygon_points)rrbboxcontentrr)figure_token_mapblockpathrqrr)"setpop IMAGE_LABELScopy enumeratezipr"rXr#appendr&r!shaper%updaterRlistrUpredictlenr'replacer r$rrcv2ZcvtColorZ COLOR_BGR2RGBr Z fromarrayr)r(r)?rYrvlayout_det_results imgs_in_docrCrDrF vlm_kwargsrJlayout_shape_modeblocksZ has_spottingZdrop_figures_setrkZdefault_min_pixelsrlZdefault_max_pixelsZbatch_dict_by_pixelZid2pixel_key_mapZimage_path_to_obj_mapZvis_image_labelsZ image_labelsir)layout_det_resimgs_in_doc_for_imgroZblocks_for_imgjrZ block_img block_labelrZ text_promptZ drop_figuresZcrop_imgwh_Z pixel_keyrZrwZ batch_resultsparsing_res_liststable_res_listsspotting_res_listZ table_blocksparsing_res_listtable_res_list spotting_res block_bbox block_contentZ pixel_inforzZ vl_rec_resultZ block_img4vlZ result_strrZhtml_strZ block_infoZimg_pathrZblk_inforOrOr]get_layout_parsing_resultss                                                   z/_PaddleOCRVLPipeline.get_layout_parsing_resultsinput3Union[str, List[str], np.ndarray, List[np.ndarray]]layout_thresholdOptional[Union[float, dict]]r@Optional[bool]rA1Optional[Union[float, Tuple[float, float], dict]]rBrrI prompt_labelOptional[Union[str, None]]repetition_penaltyOptional[float] temperaturetop_prk Optional[int]rlr{vlm_extra_argsOptional[dict]rc& +s||||||| dkrdndd<s"ddiV| dur)j} dur/ids\r7nd d krDdd <n d krNdd <dvs\Jddd* f dd  f dd| rd}tj|dtj|dtjjj|dt t t t fdd}fdd}fdd}tj ||fdd}| tj |dd}| tj |dd} | z| r3 r s2zjdd }!Wntjy rYn#Yqw|!d!s%td"|!d#d$|!d%|!d#V r rn/|D])}"t|"}#t|#d#ksMJt|#|#d!}$|$D]}%|%VqU~%~$~#~"q8W| r|jd&d |rztd'|jd&d |rtd(| jd&d | rtd)dSdSdS| r؈|jd&d |rtd'|jd&d |rtd(| jd&d | rtd)www)+a` Predicts the layout parsing result for the given input. Args: input (Union[str, list[str], np.ndarray, list[np.ndarray]]): Input image path, list of image paths, numpy array of an image, or list of numpy arrays. use_doc_orientation_classify (Optional[bool]): Whether to use document orientation classification. use_doc_unwarping (Optional[bool]): Whether to use document unwarping. use_layout_detection (Optional[bool]): Whether to use layout detection. Default is None. use_chart_recognition (Optional[bool]): Whether to use chart recognition. Default is None. use_seal_recognition (Optional[bool]): Whether to use seal recognition. Default is None. layout_threshold (Optional[float]): The threshold value to filter out low-confidence predictions. Default is None. layout_nms (Optional[bool], optional): Whether to use layout-aware NMS. Defaults to `False`. layout_unclip_ratio (Optional[Union[float, Tuple[float, float]]], optional): The ratio of unclipping the bounding box. Defaults to `None`. If it's a single number, then both width and height are used. If it's a tuple of two numbers, then they are used separately for width and height respectively. If it's None, then no unclipping will be performed. layout_merge_bboxes_mode (Optional[str], optional): The mode for merging bounding boxes. Defaults to `None`. layout_shape_mode (Optional[str], optional): The mode for layout shape. Defaults to "auto", [ "rect", "quad","poly", "auto"] are supported. use_queues (Optional[bool], optional): Whether to use queues. Defaults to `None`. prompt_label (Optional[Union[str, None]], optional): The label of the prompt in ['ocr', 'formula', 'table', 'chart']. Defaults to `None`. format_block_content (Optional[bool]): Whether to format the block content. Default is None. repetition_penalty (Optional[float]): The repetition penalty parameter used for VL model sampling. Default is None. temperature (Optional[float]): Temperature parameter used for VL model sampling. Default is None. top_p (Optional[float]): Top-p parameter used for VL model sampling. Default is None. min_pixels (Optional[int]): The minimum number of pixels allowed when the VL model preprocesses images. Default is None. max_pixels (Optional[int]): The maximum number of pixels allowed when the VL model preprocesses images. Default is None. max_new_tokens (Optional[int]): The maximum number of new tokens. Default is None. merge_layout_blocks (Optional[bool]): Whether to merge layout blocks. Default is None. markdown_ignore_labels (Optional[list[str]]): The list of ignored markdown labels. Default is None. **kwargs (Any): Additional settings to extend functionality. Returns: PaddleOCRVLResult: The predicted layout parsing result. rectFTZreturn_layout_polygon_pointsrhz0the input params for model settings are invalid!Nr=ocrrnrCrmrD)rrtrprnrurmzLayout detection is disabled (use_layout_detection=False). 'prompt_label' must be one of ['ocr', 'formula', 'table', 'chart'], but got 'z'.c 3sT|st|}tdt||D]}|j|||}|j|||}|j|||}|j|||}|}drItj| d}ndd|D}dd|D} drttj | dd } d dt | | D} n)g} | D]} | ddd d dd| j d | j dgd gd qxdd| D} |||| || | fVqdS)Nrr<)rarccSsg|]}d|iqSZ output_imgrO)r}ZarrrOrOr]r~zE_PaddleOCRVLPipeline.predict.._process_cv..cSg|]}|dqSrrO)r}itemrOrOr]r~rr=F)r?r@rArBrr"cSsg|] \}}t||dqS)ro)r)r}Z doc_pp_imgrrOrOr]r~s r)Zcls_idrrZscoreZ coordinate) input_path page_indexrocSsg|]}gqSrOrO)r}rrOrOr]r~s)rrange instances input_paths page_indexes page_countsrWrrSrTrrlowerr) batch_dataZnew_batch_sizeidxrrrrZ image_arraysdoc_preprocessor_resultsdoc_preprocessor_imagesrrdoc_preprocessor_image) rBr@rrrAmodel_settingsrrYrarcrOr] _process_cvrsx  z1_PaddleOCRVLPipeline.predict.._process_cvc3s|\}}}}}}}j|||dddd dd \}} } }t||||||| || | D]&\ } } } }}}}}}}| | | |jd|jd||||||d }t|Vq:dS) NrCrDrF)rrrrkrlr{rJ) rvrrrCrDrFrrJrrr) rr page_countwidthheightdoc_preprocessor_resrrrrrr)rrrr) results_cvrrrrrrrrrrrrrrrrrrrrZsingle_img_res) rr{rlrkrrrYrrrrOr] _process_vlms    z2_PaddleOCRVLPipeline.predict.._process_vlm@)maxsizec s|}s>zt|}Wn"tyYn&ty1}zdd|fWYd}~nd}~wwd|f~r dS)NFrT)rVis_setnext StopIteration Exceptionputr)Zinput_Zall_batch_datare)event_data_loading_doneevent_shutdown queue_inputrYrOr] _worker_inputs     z3_PaddleOCRVLPipeline.predict.._worker_inputc ssmzjdd}Wntjy!rYdSYqw|ds-|dSz|ddr;jjjndD] }d|f~q>~Wnt yf}zdd|fWYd}~dSd}~wwrdSdS) N?timeoutrrr=TFZcv) rrRqueueEmptyrrrTrVrGr)rrr)revent_cv_processing_donerrrqueue_cvrrYrOr] _worker_cv!s8  z0_PaddleOCRVLPipeline.predict.._worker_cvc sdd}jjj}sg}t}d}d} |t|}|dkr#n>zj|d}Wn tjy5Yn,w|dsB|d}n| |d~|ddD] }|t |d 7}qP||kr`nq|redS|srrq dSqd d t |D} ~z| D] } d| f~ q~ Wnt y} zdd | fWYd} ~ dSd} ~ wwr dSdS) NrFrTrrrocSsg|] }tt|qSrO)rr from_iterable)r}listsrOrOr]r~bs zE_PaddleOCRVLPipeline.predict.._worker_vlm..Zvlm)rUrVrGrtimerRrrrrrrrr) ZMAX_QUEUE_DELAY_SECSZ MAX_NUM_BOXESresults_cv_list start_timeZ should_breakZ num_boxesZremaining_timerresZmerged_results_cvZ result_vlmr)rrrevent_vlm_processing_doner queue_vlmrYrOr] _worker_vlm=sf    z1_PaddleOCRVLPipeline.predict.._worker_vlm)targetargsdaemon)rrrrrzException from the 'rz ' worker: rrz&Input worker did not terminate in timez#CV worker did not terminate in timez$VLM worker did not terminate in timeN)rfrirIrrQueuerVrG threadingEventThreadstartremptyrRr RuntimeErrorrrrjoinis_aliver warning)&rYrrarcr=rCrDrFrr@rArBrrIrrErrrrkrlr{rJrKrrZZmax_num_batches_in_processrrrZ thread_inputZ thread_cvZ thread_vlmrrrrrrO)rrrrrrrBr@rrrAr{rlrkrrrrrrrYrrrarcrr]r s@         IK   4                          z_PaddleOCRVLPipeline.predict markdown_listrtuplecCs"d}|D] }|d|d7}q|S)a Concatenate Markdown content from multiple pages into a single document. Args: markdown_list (list): A list containing Markdown data for each page. Returns: tuple: A tuple containing the processed Markdown text. rz markdown_textsrO)rYrr rrOrOr]concatenate_markdown_pagess z/_PaddleOCRVLPipeline.concatenate_markdown_pagesres_list merge_table title_level merge_pagescCstd|||||S)aConcatenate layout parsing results from multiple pages. Args: res_list: List of page parsing results merge_talble: Whether to merge tables across pages title_level: Whether to assign title levels merge_pages: Whether to concatenate pages using the new consolidate_pages() logic Returns: PaddleOCRVLResult: Combined OCR-VL result after merge_table or title_level policy zDeprecationWarning: `concatenate_pages()` is deprecated as of v3.3.14 and will be removed in v3.4.0. Please use `restructure_pages()` instead. It provides better support for table merging and title restructuring.)r rrestructure_pages)rYr r r rrOrOr]concatenate_pagessz&_PaddleOCRVLPipeline.concatenate_pages merge_tablesrelevel_titlesrc#st|dkr gSddfdd}d}g}|D]J}t|ts:|d}g|d<|dg} |d i} || | } n |d} |d i} g} | D]} || _|| _|d 7}| | qH| |d<||q|}d d |D} |rst| } |ryt| } g}|rg}|D] }| |dgq||dd<|d}g}t | D]\}}|D] }||_ ||qq||d<d |d<t||d<| drdd |D|d<| drdd |D|d<|t |nt |D]\}}| ||d<|t |q|Ed Hd S)aRestructure layout parsing results from multiple pages. Args: res_list: List of page parsing results merge_tables: Whether to merge tables across pages relevel_titles: Whether to relevel titles concatenate_pages: Whether to concatenate pages to a single document Returns: PaddleOCRVLResult: Combined OCR-VL result after merge_tables or relevel_titles policy rcSsX|ddr |dS|ddvs|ddkr*|dds*t|d|d}|ddSdS) Nr)r)r)rmrnrCFrr)rRr)rrrrOrOr] _get_img_objs     z<_PaddleOCRVLPipeline.restructure_pages.._get_img_objc shg}|D]-}t|d|d|ddtdd|d|ddd}||}r,||_||q|S) NrrZblock_polygon_pointsz^#+\srrr)rrrrrr)rrRresubr)r)rrrrobjrqrrOr]_conver_blocks_to_objs   zE_PaddleOCRVLPipeline.restructure_pages.._conver_blocks_to_objrrrrrcSr)rrOr}rrOrOr]r~ sz:_PaddleOCRVLPipeline.restructure_pages..Nrrr=cSr)rrOrrOrOr]r~#rrr<cSr)rrOrrOrOr]r~'rr) r isinstancerrRglobal_block_idZglobal_group_idrrrextendrrr)rYr rrrrrZ obj_res_listZ one_page_resrrrrZblocks_by_pageZconcatenate_resZall_imgs_in_docrZ all_page_resZ all_blocksZpage_idxZblksZblkrOrr]rsv               z&_PaddleOCRVLPipeline.restructure_pages)r4rr-r5r.r5r/r6r0r7r1r8r2r9r3r8r:r;r)rarbrcrbr=rbrCrbrDrbrFrbrErbrJrbrKrdr:re)rgrer:r8)FFFNTrj)FFNNNNNNNNrjNNNNNNNNNNNN)2rrrarbrcrbr=rbrCrbrDrbrFrbrrr@rrArrBr5rr5rIrrrrErbrrrrrrrkrrlrr{rrJrrKrdrrr:r)rrr:r)TTF)r rr r8r r8rr8)r rrr8rr8rr8)__name__ __module__ __qualname____doc__rQr_rfrirrr rr __classcell__rOrOr[r]r,6svv ?     r,c@s eZdZeddZddZdS)_BasePaddleOCRVLPipelinecCstSr)r,r`rOrOr] _pipeline_cls4sz&_BasePaddleOCRVLPipeline._pipeline_clscCs |ddS)NrGr)rR)rYr4rOrOr]_get_batch_size8s z(_BasePaddleOCRVLPipeline._get_batch_sizeN)rrrpropertyr#r$rOrOrOr]r"3s  r"rc@eZdZdZdS)PaddleOCRVLPipelinez PaddleOCR-VLNrrrentitiesrOrOrOr]r'<r'c@r&)PaddleOCRVL15PipelinezPaddleOCR-VL-1.5Nr(rOrOrOr]r+Ar*r+)A __future__rrrrr itertoolsrtypingrrrrrr numpynpZPILr utilsr Z utils.depsr Zcommon.batch_samplerrZ common.readerrmodelsrrZutils.benchmarkrZ _parallelrbaser componentsrZlayout_parsing.merge_tablerZlayout_parsing.title_levelrZlayout_parsing.utilsrrrrrrZuiltsr r!r"r#r$r%r&r'r(rZ time_methodsr,r"r'r+rOrOrOr]sN              ,