Mouth Watering Chat

What it does

Web application to chat with menus of 5 vegan restaurants.
- Based on user's dietary restrictions, recommends dishes, drinks, and takes orders.
- Creates different personas for each restaurant to keep chat lively.
- Brings flavors, aromas and textures to life--- impossible not to come back for more. Gemini API (on Vertex AI) features used:
- Function Calling: - uses `AutomaticFunctionCallingResponder`.
- Code extracts `function_call`, `function_response` and `text` from `chat.history`
- Fail-safe functions returning always something
- Handling `chat.send_message()` raised errors: e.g., Gemini hallucinates functions
- Document Understanding:
- Tried:
- Grounding with Vertex AI Search data store gave unsatisfactory search results.
- Adding PDFs to Gemini requests was expensive (each PDF page is charged as an image)
- Caching was not an option due to min size requirement.
- Decided to use:
- PyPDF2 to extract menu files. Store them in GS (see utilities folder)
- System Instructions so that a restaurant's food and drinks menus are inserted, when user picks the restaurant.
- Grounding with Google Search: An initial model is prompted to use restaurants' URLs and to create:
- Personas
- Restaurant descriptions
- System Instructions:
- Generic portion: instructs when to call functions and how to respond
- Persona: inserted when user picks a restaurant
- Menu: inserted when user picks a restaurant