The AI Edge Torch Generative API is a mobile-optimized library for converting transformer-based PyTorch models to the LiteRT format, enabling you to convert Large Language Models (LLMs) for on-device image and text generation with LiteRT and MediaPipe. The AI Edge Torch Generative API is currently CPU-only, with planned GPU and NPU support.
Mobile app developers can use the Edge Generative API to integrate PyTorch LLMs with the MediaPipe LLM Inference API to create apps that run those converted PyTorch models completely on-device.
For more information, see the Generative Torch API GitHub repo.