o *Îj¥ã@s@dZddlmZmZmZddlmZe ¡ZGdd„deƒZ dS)zcTokenization classes for Space. mainly copied from :module:`~transformers.tokenization_xlm_roberta`é)ÚBasicTokenizerÚ BertTokenizerÚWordpieceTokenizer)Úloggerc@seZdZdZdS)ÚSpaceTokenizerzŽ This class overrides [`SpaceTokenizer`]. Please check the superclass for the appropriate documentation alongside usage examples. N)Ú__name__Ú __module__Ú __qualname__Ú__doc__©r r úu/var/www/html/Deteccion_Ine/venv/lib/python3.10/site-packages/modelscope/models/nlp/space/model/tokenization_space.pyrsrN) r Z transformersrrrZmodelscope.utilsrÚloggingZ get_loggerrr r r r Ús