Youtube AI study app

What it does

My app that was developed in react native allows you to quiz yourself on youtube videos, chat to the video transcript content and gives you an intelligent summary where you can click and play at different points.

Gemini was used for the intelligent summary by create a prompt that accepts a transcript with MM:SS timestamps. From here I defined a json schema I expect back from gemini. It turned out to be very reliable and was able to parse this expected output in the react native application I developed, using these MM:SS timestamps at points to seek to the correct time in the Youtube player on the app. Gemini long context window was crucial. It could handle 5 hours of video transcript no problem

For the Q&A screen similarly after experimenting with different prompts and json schemas I settled on a multiple choice question and answer with that Gemini returned with the correct answer too. From here I was able to use this json response to build an interactive multiple choice question based entirely on the transcript of the video. This wouldn't have been easy with Gemini. Also implemented a refresh for different number of Q&A.

Lastly implemented a chat screen tab where we can chat to the video transcript and ask whatever questions we might have while studying the video and trying to learn more. This leveraged the powerful knowledge already existing with gemini along with the context of the video transcript.

Firebase hosting, functions and database were used too.