SmartAvatar

What it does

Our app transforms live streaming into an interactive experience where viewers can ask questions via YouTube chat and receive real-time answers from an AI-driven avatar of Einstein. The app leverages the Gemini API to process and generate responses to audience questions. When a question is received, it is sent to the Gemini API for a response, while simultaneously being converted to audio via a text-to-speech service. This audio is then synced with a lipsync-enabled Einstein avatar in Unity3D, creating the illusion that Einstein is speaking directly to the viewers. The entire process is streamed live on YouTube, allowing viewers to see their question, name, and photo displayed on screen as Einstein responds. To ensure smooth operation, questions are queued and processed sequentially, with both the questions and responses managed in an orderly fashion. The integration of the Gemini API is key to providing accurate and timely responses, making the experience dynamic and engaging for users.