Google Tensor (EdgeTPU) with LiteRT

Google Tensor is a custom-designed System-on-Chip (SoC) made for running AI models on Google Pixel phones. Tensor is optimized for computational efficiency and minimal energy consumption. It uses a dedicated ML inference accelerator called TPU (Tensor Processing Unit), which is accessible through Google Tensor SDK.

Sign up for access to Google Tensor SDK Beta

Google Tensor SDK is a software development kit created to optimize on-device machine learning for Google Pixel phones by utilizing the custom Tensor System-on-Chip (SoC) and its dedicated TPU inference accelerator. This SDK provides a comprehensive suite of tools that help developers access curated open-source models in Model Garden. This kit also enables the compilation of models into TPU-compatible formats.

Sign-up

Key features

• Direct access to dedicated TPU hardware for efficient ML inference on Pixel devices.
• Curated open-source models optimized for the SDK in Model Garden.

Set up the development environment

Following are the required hardware and software specifications, and the prerequisites for utilizing the Google Tensor SDK:

Supported SoCs

Google Tensor SDK supports the following SoCs:

• Google Tensor G5 (Tensor_G5)

Next steps

1. Follow conversion and deployment steps in NPU acceleration with LiteRT, choosing Google Tensor as applicable.

2. For language models, see Execute LLMs on NPU using LiteRT-LM.