SmartVision

What it does

SmartVision is a mobile assistant for visually
impaired/challenged users which provides them with enhanced accessibility,
independence, and assistance in navigating their daily lives. This mobile assistant leverages technology to offer a wide range of features and functionalities tailored to the needs of visually impaired individuals with the aim to make their everyday tasks more manageable and enable greater participation in society.
This android app has features like Detect Objects, Summarize docs, Reading Mode, Detect Faces, Scan Products. Detects Object and Summarize docs features are built using Gemini API.Detect Objects in particular uses Gemini-1.5-Flash model as it is faster compared to the Gemini-1.5-Pro. To use detect objects feature, the user will have to wear a Smart cap which will have a WiFi enabled camera module on it. The images taken from this camera will be displayed on the mobile app and sent to the remote Gemini API for describing image captured in real time by the Gemini API, the description will be read out or announced to the user using the text to speech feature for convenience of the visually impaired user. Moreover, the summarize docs(pdf only) is built using Gemini-1.5-pro model. To use this feature, the user needs to select a pdf document that resides in the phone's memory and set a prompt text (lets say to summarize the pdf document in 150 words.) The Gemini API will summarize the text content present in the pdf and announce it to the user.