View source on GitHub
|
A container for the sentence piece tokenizer 1 metadata information.
tflite_support.metadata_writers.metadata_info.SentencePieceTokenizerMd(
sentence_piece_model_path: str, vocab_file_path: Optional[str] = None
)
Args | |
|---|---|
sentence_piece_model_path
|
path to the sentence piece model file. |
vocab_file_path
|
path to the vocabulary file. |
Methods
create_metadata
create_metadata() -> tflite_support.metadata_schema_py_generated.ProcessUnitT
Creates the sentence piece tokenizer metadata based on the information.
| Returns | |
|---|---|
| A Flatbuffers Python object of the sentence piece tokenizer metadata. |