tflite_support.metadata_writers.writer_utils.get_tokenizer_associated_files

Gets a list of associated files packed in the tokenzier_options.

tflite_support.metadata_writers.writer_utils.get_tokenizer_associated_files(
    tokenizer_options: Union[None, _metadata_fb.BertTokenizerOptionsT, _metadata_fb.
        SentencePieceTokenizerOptionsT, _metadata_fb.RegexTokenizerOptionsT]
) -> List[Optional[str]]

tokenizer_options a tokenizer metadata object. Support the following tokenizer types:

  1. BertTokenizerOptions: https://github.com/tensorflow/tflite-support/blob/b80289c4cd1224d0e1836c7654e82f070f9eefaa/tensorflow_lite_support/metadata/metadata_schema.fbs#L436
  2. SentencePieceTokenizerOptions: https://github.com/tensorflow/tflite-support/blob/b80289c4cd1224d0e1836c7654e82f070f9eefaa/tensorflow_lite_support/metadata/metadata_schema.fbs#L473
  3. RegexTokenizerOptions: https://github.com/tensorflow/tflite-support/blob/b80289c4cd1224d0e1836c7654e82f070f9eefaa/tensorflow_lite_support/metadata/metadata_schema.fbs#L475

A list of associated files included in tokenizer_options.