LiteRT-LM File Builder

The litert-lm-builder package provides Python tools and Command Line Interfaces (CLIs) to build, inspect, and unpack LiteRT-LM (.litertlm) container files.

A .litertlm file is a single unified container that packages your TFLite models, tokenizer files, external weights, and associated model metadata so they can be distributed and loaded by the LiteRT-LM runtime.

Installation

Install the litert-lm-builder package from PyPI.

Method 1: uvx (Recommended for quick execution)

Run the CLIs immediately without a permanent installation. Requires uv.

uvx litert-lm-builder --help
uvx litert-lm-peek --help

Method 2: pip

Standard installation within a virtual environment:

python3 -m venv .venv
source .venv/bin/activate
pip install --upgrade litert-lm-builder

Build a .litertlm file: litert-lm-builder

Use the litert-lm-builder CLI to package your model components. You can do this either using a configuration file or by chaining command-line arguments.

Usage

The following methods can be used to prepare the .litertlm file using the builder.

Method 1: Build using TOML Configuration (Recommended)

Specify all components, paths, and metadata in a TOML configuration file, then run the builder:

litert-lm-builder toml --path config.toml output --path model.litertlm

Example config.toml:

[system_metadata]
entries = [
  { key = "author", value_type = "String", value = "Authors" }
]

[[section]]
section_type = "LlmMetadata"
data_path = "path/to/llm_metadata.pb"

[[section]]
section_type = "SP_Tokenizer"
data_path = "path/to/sp.model"

[[section]]
section_type = "TFLiteModel"
model_type = "PREFILL_DECODE"
data_path = "path/to/model.tflite"
additional_metadata = [
  { key = "model_version", value_type = "String", value = "1.0.1" }
]

Method 2: Build using Command Line Arguments

You can build the container dynamically by chaining subcommands in your terminal. The order of the subcommands determines the order of the sections in the generated file.

litert-lm-builder \
  system_metadata --str author "Authors" \
  llm_metadata --path path/to/llm_metadata.pb \
  sp_tokenizer --path path/to/sp.model \
  tflite_model --path path/to/model.tflite --model_type prefill_decode --str_metadata model_version "1.0.1" \
  output --path model.litertlm

CLI Options Reference

The litert-lm-builder CLI supports the following subcommands:

  • output (Required): Specifies the output path.
    • --path PATH: Path to save the built .litertlm file.
  • toml: Load configuration from a TOML file.
    • --path PATH: Path to the .toml file.
  • system_metadata: Add global system metadata.
    • --str KEY VALUE: Add a string key-value pair (can be specified multiple times).
    • --int KEY VALUE: Add an integer key-value pair (can be specified multiple times).
    • Note: The builder automatically generates and appends a unique uuid and creation_timestamp (in UTC ISO 8601 format) to the system metadata. Do not specify these keys manually. These fields are used by the runtime to identify the model build uniquely and manage compiled cache invalidation.
  • llm_metadata: Add LLM-specific configuration.
    • --path PATH: Path to the LLM metadata (text or binary proto).
  • tflite_model: Add a TFLite model.
    • --path PATH: Path to the .tflite file.
    • --model_type TYPE: One of: embedder, prefill_decode (representing both prefill and decode), prefill, decode.
    • --backend_constraint BACKEND: (Optional) Backend constraint (e.g., gpu, cpu, npu).
    • --prefer_activation_type TYPE: (Optional) Preferred activation type (fp16, fp32, fp32_fp16).
    • --str_metadata KEY VALUE: (Optional) String metadata for this model section.
  • sp_tokenizer: Add a SentencePiece tokenizer.
    • --path PATH: Path to the .model file.
    • --str_metadata KEY VALUE: (Optional) String metadata.
  • hf_tokenizer: Add a Hugging Face tokenizer.
    • --path PATH: Path to the tokenizer.json file.
    • --str_metadata KEY VALUE: (Optional) String metadata.

Inspect and unpack a .litertlm file: litert-lm-peek

Use the litert-lm-peek CLI to inspect a .litertlm container or unpack its embedded components. Running it displays the full container structure, all packaged sections, and the automatically generated system metadata (such as uuid and creation_timestamp).

Usage

litert-lm-peek --litertlm_file model.litertlm [options]

CLI Options Reference

  • --litertlm_file PATH (Required): The path to the .litertlm file to inspect.
  • --dump_files_dir PATH (Optional): The directory where all packaged files (models, tokenizers, weights, metadata) should be extracted/unpacked. If not provided, the tool will only print the metadata and section structure to the console without extracting files.