The AI Edge Torch Generative API is a mobile-optimized library for converting transformer-based PyTorch models to the TF Lite format, enabling you to convert Large Language Models (LLMs) for on-device image and text generation with TF Lite and MediaPipe. The AI Edge Torch Generative API is currently CPU-only, with planned GPU and NPU support.
Mobile app developers can use the Edge Generative API to integrate PyTorch LLMs with the MediaPipe LLM Inference API to create apps that run those converted PyTorch models completely on-device.
For more information, see the Generative Torch API GitHub repo.