Menu Buddy

What it does

Menu Buddy is a progressive web app designed to empower speech-impaired individuals, such as those who are deaf or have conditions like Aphasia or Dysarthria, in everyday situations like ordering at restaurants and coffee shops.
The app lets users take an image of a menu, which is then transformed into an accessible digital version with the help of Google's Gemini API. All items on the menu become buttons with large labels so that people with limited motor skills can select them. Once an order is selected, Menu Buddy generates a clear and concise "order card" that can be easily read by cashiers or waitstaff. If the staff have follow-up questions, the app records their voice and uses Gemini to transcribe any questions, and generate buttons for the user to quickly communicate relevant answers.
By bridging communication gaps, Menu Buddy not only simplifies the ordering process for those with speech impairments but also promotes independence and confidence in them.