Recycling Advisor 3D

What it does

Recycling Advisor 3D is a web app where you can upload a photo and receive recycling advice from a talking 3D avatar. If applicable, the avatar also suggests the nearest recycling center that accepts the type of items shown in the photo. Local government entities responsible for waste management can embed the app as a component on their websites to promote proper disposal methods and help their residents. Each component can be localized to include information about local recycling centers and local guidelines. It is also possible to change the 3D character, language, and/or voice. By making recycling easy, fun, and engaging, the app helps protect the environment.

The app uploads the photo to Gemini using its File API and makes multimodal Gemini API calls utilizing multi-turn function calling. In the first turn, Gemini classifies the items in the photo and asks for more detailed information based on the category of the items. The final text response stream is directed to an animated 3D avatar that can speak and lip-sync in real-time using Google Cloud Text-to-Speech AI voices, accompanied by subtitles for people who are deaf or hard of hearing. Gemini not only generates the response for the avatar to speak but also controls its gestures and facial expressions using keywords and emojis. The nearest local recycling center is calculated based on the Gemini response and the user's location and is presented with a map generated by the Google Maps Static API.