Ally

What it does

Ally is an everyday assistant designed to help people who are blind or have low vision live more independently. It provides quick and accessible information about their surroundings through a conversational interface, making it easy to access essential details in real-time. The app can handle various tasks, such as offering weather advice, making personalized recommendations from a menu, or filling out web forms, all by simply asking.
We harness the power of the Gemini API to enhance the app’s capabilities in two significant ways:
Personalization and Contextual Insights: With Gemini’s long context window and integration with Firebase, we store embeddings of user information, enabling the app to deliver personalized, contextual, and insightful answers. This feature ensures that the assistant remembers user preferences and history, providing responses that are tailored to their specific needs and situations.
Optimized Functionality: Gemini’s function calling is pivotal in routing user questions to the most effective algorithms. Whether it’s checking your schedule, weather, even filling out forms on the Web, Gemini routes the query to the right tool seamlessly.
Ally’s main benefit is its accessibility, allowing users to engage with it seamlessly across the web, smartphones, smartglasses, or even through a simple phone call.