o 0jf@sddlmZmZmZmZmZmZddlZddl m Z ddl m Z ddl mZddlmZdd lmZmZdd lmZdd lmZd d lmZd dlmZd dlmZd dlmZddl m!Z!ddl m"Z"m#Z#ej$GdddeZ%e dGdddeZ&dS))AnyDictListOptionalTupleUnionN)logging)pipeline_requires_extra)ImageBatchSampler) ReadImage) HPIConfigPaddlePredictorOption) DetResult) benchmark)(AutoParallelImageSimpleInferencePipeline) BasePipeline) CropByBoxes) OCRResult)LayoutParsingResult)get_sub_regions_ocr_ressorted_layout_boxesc1s^eZdZdZddddddddedeedeedeeeefd eed e d ee eeefe fd dffd dZ ded dfddZ      d7dededededededeedeedeedeedeedeed efddZded e fd d!Zd"ee d#ee d$ee d%ee d&ee d ef d'd(Z                      d8d)e eeeejeejfd"ee d#ee d*ee d$ee d%ee d&ee d+ee eefd,ee d-ee eeeefefd.eedeedeedeedeedeedeed/eed0eed1eed2eed3eed4eed ef0d5d6ZZS)9_LayoutParsingPipelinezLayout Parsing PipelineNFdeviceengine engine_config pp_optionuse_hpip hpi_configconfigrrrr r!r"returnc sNtjd||||||d|||tdd|_tdd|_t|_dS)aInitializes the layout parsing pipeline. Args: config (Dict): Configuration dictionary containing various settings. device (Optional[str], optional): The device to use for prediction. Defaults to `None`. engine (Optional[str], optional): Inference engine. Defaults to `None`. engine_config (Optional[Dict[str, Any]], optional): Engine-specific config. Defaults to `None`. pp_option (Optional[PaddlePredictorOption], optional): Paddle predictor options. Defaults to `None`. use_hpip (bool, optional): Whether to use HPIP. Defaults to `False`. hpi_config (Optional[Union[Dict[str, Any], HPIConfig]], optional): HPIP configuration. Defaults to `None`. rr)Z batch_sizeZBGR)formatN) super__init__inintial_predictorr batch_samplerr img_readerrZ_crop_by_boxes) selfr#rrrr r!r"kwargs __class__r&t/var/www/html/Deteccion_Ine/venv/lib/python3.10/site-packages/paddlex/inference/pipelines/layout_parsing/pipeline.pyr(&s   z_LayoutParsingPipeline.__init__c Cs|dd|_|dd|_|dd|_|dd|_|jr1|diddd i}|||_|d id d d i}i}|dd}durM||d<|dd}dur[||d<|dd}duri||d<|dd}durw||d<|j|fi||_|didddi} || |_ |jr|didddi} || |_ |jr|didddi} || |_ |jr|didddi} || |_ dS)zInitializes the predictor based on the provided configuration. Args: config (Dict): A dictionary containing the configuration for the predictor. Returns: None use_doc_preprocessorTuse_table_recognitionuse_seal_recognitionuse_formula_recognitionZ SubPipelinesZDocPreprocessorZpipeline_config_errorz+config error for doc_preprocessor_pipeline!Z SubModulesZLayoutDetectionZmodel_config_errorz"config error for layout_det_model! thresholdN layout_nmslayout_unclip_ratiolayout_merge_bboxes_modeZ GeneralOCRz&config error for general_ocr_pipeline!ZSealRecognitionz+config error for seal_recognition_pipeline!ZTableRecognitionz,config error for table_recognition_pipeline!ZFormulaRecognitionz.config error for formula_recognition_pipeline!) getr1r2r3r4Zcreate_pipelinedoc_preprocessor_pipelineZ create_modellayout_det_modelgeneral_ocr_pipelineseal_recognition_pipelinetable_recognition_pipelineformula_recognition_pipeline) r,r#Zdoc_preprocessor_configZlayout_det_configZ layout_kwargsr5r6r7r8Zgeneral_ocr_configZseal_recognition_configZtable_recognition_configZformula_recognition_configr&r&r0r)Ps        z)_LayoutParsingPipeline.inintial_predictorimagelayout_det_resoverall_ocr_restable_res_list seal_res_listformula_res_listtext_det_limit_side_lentext_det_limit_typetext_det_threshtext_det_box_threshtext_det_unclip_ratiotext_rec_score_threshc & Csg} i}d}d}d}t|}g}t|dD]\}}i}|d}|d}||d<||d<d|d<|||d krYt|dkrXt||ksLJd ||d |d<|d 7}nr|d krzt|dkryt||ksmJd||d|d<|d 7}nQ|dkrt|dkrt||ksJdd||d|d<|d 7}n-t||gdd\}}|D]}||ddur|g||<q|||qd|d|d<| |q| D]]}t|d kr2|D]Q}tj |j |j dd}| |d}dd|D\}}} }!|||!|| ddf|||!|| ddf<t |j|||| | | | dd}"d|"d| |d<qqt||dd}#t|#d|#dD]\}$}%i}|$|d<d |d<|%|d<| |qCt| |j d d!} | S)"a Retrieves the layout parsing result based on the layout detection result, OCR result, and other recognition results. Args: image (list): The input image. layout_det_res (DetResult): The detection result containing the layout information of the document. overall_ocr_res (OCRResult): The overall OCR result containing text information. table_res_list (list): A list of table recognition results. seal_res_list (list): A list of seal recognition results. formula_res_list (list): A list of formula recognition results. text_det_limit_side_len (Optional[int], optional): The maximum side length of the text detection region. Defaults to `None`. text_det_limit_type (Optional[str], optional): The type of limit for the text detection region. Defaults to `None`. text_det_thresh (Optional[float], optional): The confidence threshold for text detection. Defaults to `None`. text_det_box_thresh (Optional[float], optional): The confidence threshold for text detection bounding boxes. Defaults to `None` text_det_unclip_ratio (Optional[float], optional): The unclip ratio for text detection. Defaults to `None`. text_rec_score_thresh (Optional[float], optional): The score threshold for text recognition. Defaults to `None`. Returns: list: A list of dictionaries representing the layout parsing result. rZboxesZ coordinatelabelZ block_bboxZ block_labelZ block_contentZformulazThe number of formula regions of layout parsing pipeline and formula recognition pipeline are different!Z rec_formulartablezThe number of table regions of layout parsing pipeline and table recognition pipeline are different!Z pred_htmlZsealzThe number of seal regions of layout parsing pipeline and seal recognition pipeline are different!z, Z rec_textsT)Zreturn_match_idxN )dtypecSsg|]}t|qSr&)int).0ir&r&r0 szA_LayoutParsingPipeline.get_layout_parsing_res..)rFrGrHrIrJrKF)Z flag_withinZ rec_boxesZ other_text)w)nparray enumeratelowerappendlenjoinrr9valuesZonesshaperPlistr<zipr)&r,r@rArBrCrDrErFrGrHrIrJrKZlayout_parsing_resZmatched_ocr_dictZ formula_indexZ table_indexZ seal_indexZ object_boxesZobject_box_idxZbox_infoZsingle_box_resboxrLZocr_res_in_boxZ matched_idxesZ matched_idxZlayout_box_idsidxZwht_imx1y1Zx2y2Z sub_ocr_resZocr_without_layout_boxesZ ocr_rec_boxZ ocr_rec_textr&r&r0get_layout_parsing_ress!            0    z-_LayoutParsingPipeline.get_layout_parsing_res input_paramscCsX|dr|jstddS|dr|jstddS|dr*|js*tddSdS) a4 Check if the input parameters are valid based on the initialized models. Args: input_params (Dict): A dictionary containing input parameters. Returns: bool: True if all required models are initialized according to input parameters, False otherwise. r1zRSet use_doc_preprocessor, but the models for doc preprocessor are not initialized.Fr3zRSet use_seal_recognition, but the models for seal recognition are not initialized.r2zTSet use_table_recognition, but the models for table recognition are not initialized.T)r1r errorr3r2)r,rhr&r&r0check_model_settings_valid,s  z1_LayoutParsingPipeline.check_model_settings_validuse_doc_orientation_classifyuse_doc_unwarpingr3r2r4cCsl|dur |dur |j}n |dus|durd}nd}|dur |j}|dur'|j}|dur.|j}t||||dS)a' Get the model settings based on the provided parameters or default values. Args: use_doc_orientation_classify (Optional[bool]): Whether to use document orientation classification. use_doc_unwarping (Optional[bool]): Whether to use document unwarping. use_seal_recognition (Optional[bool]): Whether to use seal recognition. use_table_recognition (Optional[bool]): Whether to use table recognition. Returns: dict: A dictionary containing the model settings. NTF)r1r3r2r4)r1r3r2r4dict)r,rkrlr3r2r4r1r&r&r0get_model_settingsKs"z)_LayoutParsingPipeline.get_model_settingsinputuse_textline_orientationlayout_thresholdr6r7r8seal_det_limit_side_lenseal_det_limit_typeseal_det_threshseal_det_box_threshseal_det_unclip_ratioseal_rec_score_threshc)ks||||||}||sddiVt||D]\}}||jd}|dr8t|j|||dd}nd|i}|d}t|j||| | | dd}t|j ||| | ||||dd} |d rxt|j |d d d d | |d d}!|!d }"ng}"|d rt|j |d d d |||||||d d}#|#d}$ng}$|drt|j |d d d |dd}%|%d}&ng}&|j ||| |"|$|&| | ||||d }'|jd|jd||| |"|$|&|'|d }(t|(VqdS)a This function predicts the layout parsing result for the given input. Args: input (Union[str, list[str], np.ndarray, list[np.ndarray]]): The input image(s) or pdf(s) to be processed. use_doc_orientation_classify (Optional[bool]): Whether to use document orientation classification. use_doc_unwarping (Optional[bool]): Whether to use document unwarping. use_textline_orientation (Optional[bool]): Whether to use textline orientation prediction. use_seal_recognition (Optional[bool]): Whether to use seal recognition. use_table_recognition (Optional[bool]): Whether to use table recognition. use_formula_recognition (Optional[bool]): Whether to use formula recognition. layout_threshold (Optional[float]): The threshold value to filter out low-confidence predictions. Default is None. layout_nms (Optional[bool], optional): Whether to use layout-aware NMS. Defaults to `False`. layout_unclip_ratio (Optional[Union[float, Tuple[float, float]]], optional): The ratio of unclipping the bounding box. Defaults to `None`. If it's a single number, then both width and height are used. If it's a tuple of two numbers, then they are used separately for width and height respectively. If it's None, then no unclipping will be performed. layout_merge_bboxes_mode (Optional[str], optional): The mode for merging bounding boxes. Defaults to `None`. text_det_limit_side_len (Optional[int]): Maximum side length for text detection. text_det_limit_type (Optional[str]): Type of limit to apply for text detection. text_det_thresh (Optional[float]): Threshold for text detection. text_det_box_thresh (Optional[float]): Threshold for text detection boxes. text_det_unclip_ratio (Optional[float]): Ratio for unclipping text detection boxes. text_rec_score_thresh (Optional[float]): Score threshold for text recognition. seal_det_limit_side_len (Optional[int]): Maximum side length for seal detection. seal_det_limit_type (Optional[str]): Type of limit to apply for seal detection. seal_det_thresh (Optional[float]): Threshold for seal detection. seal_det_box_thresh (Optional[float]): Threshold for seal detection boxes. seal_det_unclip_ratio (Optional[float]): Ratio for unclipping seal detection boxes. seal_rec_score_thresh (Optional[float]): Score threshold for seal recognition. **kwargs: Additional keyword arguments. Returns: LayoutParsingResult: The predicted layout parsing result. riz0the input params for model settings are invalid!rr1)rkrlZ output_img)r5r6r7r8)rprFrGrHrIrJrKr2F)rkrluse_layout_detectionZ use_ocr_modelrBrArCr3) rkrlrxrArrrsrtrurvrwrDr4)rxrkrlrArE) rArBrCrDrErFrGrHrIrJrK) Z input_pathZ page_indexdoc_preprocessor_resrArBrCrDrEparsing_res_listmodel_settingsN)rnrjrYr*r+Z instancesr`r:r;r<r>r=r?rgZ input_pathsZ page_indexesr))r,rorkrlrpr3r2r4rqr6r7r8rFrGrHrIrJrKrrrsrtrurvrwr-r{Zimg_idZ batch_dataZ image_arrayryZdoc_preprocessor_imagerArBZ table_res_allrCZ seal_res_allrDZformula_res_allrErzZsingle_img_resr&r&r0predictwsA          z_LayoutParsingPipeline.predict)NNNNNN)NNNNNNNNNNNNNNNNNNNNNN)__name__ __module__ __qualname____doc__rrstrrrboolrrr(r)r`rrrRfloatrgrjrmrnrrWZndarrayrrr| __classcell__r&r&r.r0r"s"  *`       /     rZocrc@s&eZdZdgZeddZddZdS)LayoutParsingPipelineZlayout_parsingcCstS)N)r)r,r&r&r0 _pipeline_cls>sz#LayoutParsingPipeline._pipeline_clscCsdS)Nrr&)r,r#r&r&r0_get_batch_sizeBsz%LayoutParsingPipeline._get_batch_sizeN)r}r~rentitiespropertyrrr&r&r&r0r:s   r)'typingrrrrrrnumpyrWutilsr Z utils.depsr Zcommon.batch_samplerr Z common.readerr modelsrrZmodels.object_detection.resultrZutils.benchmarkrZ _parallelrbaser componentsrZ ocr.resultrresultrrrZ time_methodsrrr&r&r&r0s.