AvvA

What it does

AvvA was built to be accessible from anywhere in Android. As soon as it's activated, it captures a screenshot of the currently open app.
The user has instant access to this screenshot and can input text via voice or typing to complete any task they wish to request from Gemini.We configured the Gemini API to handle the screenshots and respond in chat mode, allowing the user to clarify or validate a simple doubt, or even engage in a deeper conversation if they wish.
We also use Gemini to interpret possible user actions, such as identifying if they want to play a song. Gemini's response can prompt AvvA to open other apps and perform actions; for example, it can open YouTube and click on the song identified by Gemini.
Other actions that use screen context include: Checking social media news; Translating content; Requesting suggestions for something to say in a conversation;
All Gemini responses can be easily copied to be used in other applications.