DeepLook

What it does

DeepLook is an innovative platform that employs Gemini models to add Generative AI functionalities to video surveillance and analytics. These functionalities range from controlling the UI interface via chat or voice to intelligent video analysis and event detection on the cameras. DeepLook can perform tasks using natural language, such as quickly opening cameras, accessing recordings, and exporting videos without using menus and hard-to-find widgets. Most importantly, through Vision analytics, it is possible to perform Q&A on live cameras, summarize past recordings, detect objects, and even let a family know how a person living alone is doing. Additionally, it can trigger sentence-based events, such as, "Did someone fall?", "Alert if a weapon appears" or situational analysis in the context of adult care, such as monitoring elderly people.

In addition, DeepLook can alert users if registered events occur and execute manual or automatic PTZ movements on cameras, like automatically centering the camera on a determined frame object.
 
The system works primarily with Gemini-flash due to its cost-benefit ratio. The parsing of commands relies heavily on vertex AI Function Calling. Image analysis works with video snippets and tiled image mosaics submitted via prompts to the model API. DeepLook will have Web and Android versions. The server can run on Firebase App Host and Cloud Run, using an agent that connects the cameras locally, serving as a bridge between them and the server.