The Gemini Interactions API is an experimental API that allows developers to build generative AI applications using Gemini models. Gemini is our most capable model, built from the ground up to be multimodal. It can generalize and seamlessly understand, operate across, and combine different types of information including language, images, audio, video, and code. You can use the Gemini API for use cases like reasoning across text and images, content generation, dialogue agents, summarization and classification systems, and more.
Creating an interaction
Creates a new interaction.
Request body
The request body contains data with the following structure:
model ModelOption (optional)
The name of the `Model` used for generating the interaction.
Required if `agent` is not provided.
Possible values:
-
gemini-2.5-flashOur first hybrid reasoning model which supports a 1M token context window and has thinking budgets.
-
gemini-2.5-flash-imageOur native image generation model, optimized for speed, flexibility, and contextual understanding. Text input and output is priced the same as 2.5 Flash.
-
gemini-2.5-flash-liteOur smallest and most cost effective model, built for at scale usage.
-
gemini-2.5-flash-lite-preview-09-2025The latest model based on Gemini 2.5 Flash lite optimized for cost-efficiency, high throughput and high quality.
-
gemini-2.5-flash-native-audio-preview-12-2025Our native audio models optimized for higher quality audio outputs with better pacing, voice naturalness, verbosity, and mood.
-
gemini-2.5-flash-preview-09-2025The latest model based on the 2.5 Flash model. 2.5 Flash Preview is best for large scale processing, low-latency, high volume tasks that require thinking, and agentic use cases.
-
gemini-2.5-flash-preview-ttsOur 2.5 Flash text-to-speech model optimized for powerful, low-latency controllable speech generation.
-
gemini-2.5-proOur state-of-the-art multipurpose model, which excels at coding and complex reasoning tasks.
-
gemini-2.5-pro-preview-ttsOur 2.5 Pro text-to-speech audio model optimized for powerful, low-latency speech generation for more natural outputs and easier to steer prompts.
-
gemini-3-flash-previewOur most intelligent model built for speed, combining frontier intelligence with superior search and grounding.
-
gemini-3-pro-image-previewState-of-the-art image generation and editing model.
-
gemini-3-pro-previewOur most intelligent model with SOTA reasoning and multimodal understanding, and powerful agentic and vibe coding capabilities.
-
gemini-3.1-pro-previewOur latest SOTA reasoning model with unprecedented depth and nuance, and powerful multimodal understanding and coding capabilities.
-
gemini-3.1-flash-image-previewPro-level visual intelligence with Flash-speed efficiency and reality-grounded generation capabilities.
agent AgentOption (optional)
The name of the `Agent` used for generating the interaction.
Required if `model` is not provided.
Possible values:
-
deep-research-pro-preview-12-2025Gemini Deep Research Agent
The inputs for the interaction (common to both Model and Agent).
System instruction for the interaction.
A list of tool declarations the model may call during interaction.
Enforces that the generated response is a JSON object that complies with the JSON schema specified in this field.
The mime type of the response. This is required if response_format is set.
Input only. Whether the interaction will be streamed.
Input only. Whether to store the response and request for later retrieval.
Input only. Whether to run the model interaction in the background.
generation_config GenerationConfig (optional)
Model Configuration
Configuration parameters for the model interaction.
Alternative to `agent_config`. Only applicable when `model` is set.
Fields
Controls the randomness of the output.
The maximum cumulative probability of tokens to consider when sampling.
Seed used in decoding for reproducibility.
A list of character sequences that will stop output interaction.
thinking_level ThinkingLevel (optional)
The level of thought tokens that the model should generate.
Possible values:
-
minimal -
low -
medium -
high
thinking_summaries ThinkingSummaries (optional)
Whether to include thought summaries in the response.
Possible values:
-
auto -
none
The maximum number of tokens to include in the response.
speech_config SpeechConfig (optional)
Configuration for speech interaction.
Fields
The voice of the speaker.
The language of the speech.
The speaker's name, it should match the speaker name given in the prompt.
image_config ImageConfig (optional)
Configuration for image interaction.
Fields
No description provided.
Possible values:
-
1:1 -
2:3 -
3:2 -
3:4 -
4:3 -
4:5 -
5:4 -
9:16 -
16:9 -
21:9 -
1:8 -
8:1 -
1:4 -
4:1
No description provided.
Possible values:
-
1K -
2K -
4K -
512
The tool choice configuration.
agent_config object (optional)
Agent Configuration
Configuration for the agent.
Alternative to `generation_config`. Only applicable when `agent` is set.
Possible Types
Polymorphic discriminator: type
DynamicAgentConfig
Configuration for dynamic agents.
No description provided.
Always set to "dynamic".
DeepResearchAgentConfig
Configuration for the Deep Research agent.
No description provided.
Always set to "deep-research".
thinking_summaries ThinkingSummaries (optional)
Whether to include thought summaries in the response.
Possible values:
-
auto -
none
The ID of the previous interaction, if any.
response_modalities ResponseModality (optional)
The requested modalities of the response (TEXT, IMAGE, AUDIO).
Possible values:
-
text -
image -
audio
Response
Returns an Interaction resource.
Simple Request
Example Response
{ "created": "2025-11-26T12:25:15Z", "id": "v1_ChdPU0F4YWFtNkFwS2kxZThQZ05lbXdROBIXT1NBeGFhbTZBcEtpMWU4UGdOZW13UTg", "model": "gemini-3-flash-preview", "object": "interaction", "outputs": [ { "text": "Hello! I'm functioning perfectly and ready to assist you.\n\nHow are you doing today?", "type": "text" } ], "role": "model", "status": "completed", "updated": "2025-11-26T12:25:15Z", "usage": { "input_tokens_by_modality": [ { "modality": "text", "tokens": 7 } ], "total_cached_tokens": 0, "total_input_tokens": 7, "total_output_tokens": 20, "total_thought_tokens": 22, "total_tokens": 49, "total_tool_use_tokens": 0 } }
Multi-turn
Example Response
{ "id": "v1_ChdPU0F4YWFtNkFwS2kxZThQZ05lbXdROBIXT1NBeGFhbTZBcEtpMWU4UGdOZW13UTg", "model": "gemini-3-flash-preview", "status": "completed", "object": "interaction", "created": "2025-11-26T12:22:47Z", "updated": "2025-11-26T12:22:47Z", "role": "model", "outputs": [ { "type": "text", "text": "The capital of France is Paris." } ], "usage": { "input_tokens_by_modality": [ { "modality": "text", "tokens": 50 } ], "total_cached_tokens": 0, "total_input_tokens": 50, "total_output_tokens": 10, "total_thought_tokens": 0, "total_tokens": 60, "total_tool_use_tokens": 0 } }
Image Input
Example Response
{ "id": "v1_ChdPU0F4YWFtNkFwS2kxZThQZ05lbXdROBIXT1NBeGFhbTZBcEtpMWU4UGdOZW13UTg", "model": "gemini-3-flash-preview", "status": "completed", "object": "interaction", "created": "2025-11-26T12:22:47Z", "updated": "2025-11-26T12:22:47Z", "role": "model", "outputs": [ { "type": "text", "text": "A white humanoid robot with glowing blue eyes stands holding a red skateboard." } ], "usage": { "input_tokens_by_modality": [ { "modality": "text", "tokens": 10 }, { "modality": "image", "tokens": 258 } ], "total_cached_tokens": 0, "total_input_tokens": 268, "total_output_tokens": 20, "total_thought_tokens": 0, "total_tokens": 288, "total_tool_use_tokens": 0 } }
Function Calling
Example Response
{ "id": "v1_ChdPU0F4YWFtNkFwS2kxZThQZ05lbXdROBIXT1NBeGFhbTZBcEtpMWU4UGdOZW13UTg", "model": "gemini-3-flash-preview", "status": "requires_action", "object": "interaction", "created": "2025-11-26T12:22:47Z", "updated": "2025-11-26T12:22:47Z", "role": "model", "outputs": [ { "type": "function_call", "id": "gth23981", "name": "get_weather", "arguments": { "location": "Boston, MA" } } ], "usage": { "input_tokens_by_modality": [ { "modality": "text", "tokens": 100 } ], "total_cached_tokens": 0, "total_input_tokens": 100, "total_output_tokens": 25, "total_thought_tokens": 0, "total_tokens": 125, "total_tool_use_tokens": 50 } }
Deep Research
Example Response
{ "id": "v1_ChdPU0F4YWFtNkFwS2kxZThQZ05lbXdROBIXT1NBeGFhbTZBcEtpMWU4UGdOZW13UTg", "agent": "deep-research-pro-preview-12-2025", "status": "completed", "object": "interaction", "created": "2025-11-26T12:22:47Z", "updated": "2025-11-26T12:22:47Z", "role": "agent", "outputs": [ { "type": "text", "text": "Here is a comprehensive research report on the current state of cancer research..." } ], "usage": { "input_tokens_by_modality": [ { "modality": "text", "tokens": 20 } ], "total_cached_tokens": 0, "total_input_tokens": 20, "total_output_tokens": 1000, "total_thought_tokens": 500, "total_tokens": 1520, "total_tool_use_tokens": 0 } }
Retrieving an interaction
Retrieves the full details of a single interaction based on its `Interaction.id`.
Path / Query Parameters
The unique identifier of the interaction to retrieve.
If set to true, the generated content will be streamed incrementally.
Defaults to: False
Optional. If set, resumes the interaction stream from the next chunk after the event marked by the event id. Can only be used if `stream` is true.
If set to true, includes the input in the response.
Defaults to: False
Which version of the API to use.
Response
Returns an Interaction resource.
Get Interaction
Example Response
{ "id": "v1_ChdPU0F4YWFtNkFwS2kxZThQZ05lbXdROBIXT1NBeGFhbTZBcEtpMWU4UGdOZW13UTg", "model": "gemini-3-flash-preview", "status": "completed", "object": "interaction", "created": "2025-11-26T12:25:15Z", "updated": "2025-11-26T12:25:15Z", "role": "model", "outputs": [ { "type": "text", "text": "I'm doing great, thank you for asking! How can I help you today?" } ] }
Deleting an interaction
Deletes the interaction by id.
Path / Query Parameters
The unique identifier of the interaction to delete.
Which version of the API to use.
Response
If successful, the response is empty.
Delete Interaction
Canceling an interaction
Cancels an interaction by id. This only applies to background interactions that are still running.
Path / Query Parameters
The unique identifier of the interaction to cancel.
Which version of the API to use.
Response
Returns an Interaction resource.
Cancel Interaction
Example Response
{ "id": "v1_ChdPU0F4YWFtNkFwS2kxZThQZ05lbXdROBIXT1NBeGFhbTZBcEtpMWU4UGdOZW13UTg", "agent": "deep-research-pro-preview-12-2025", "status": "cancelled", "object": "interaction", "created": "2025-11-26T12:25:15Z", "updated": "2025-11-26T12:25:15Z", "role": "agent" }
Resources
Interaction
The Interaction resource.
Fields
model ModelOption (optional)
The name of the `Model` used for generating the interaction.
Possible values:
-
gemini-2.5-flashOur first hybrid reasoning model which supports a 1M token context window and has thinking budgets.
-
gemini-2.5-flash-imageOur native image generation model, optimized for speed, flexibility, and contextual understanding. Text input and output is priced the same as 2.5 Flash.
-
gemini-2.5-flash-liteOur smallest and most cost effective model, built for at scale usage.
-
gemini-2.5-flash-lite-preview-09-2025The latest model based on Gemini 2.5 Flash lite optimized for cost-efficiency, high throughput and high quality.
-
gemini-2.5-flash-native-audio-preview-12-2025Our native audio models optimized for higher quality audio outputs with better pacing, voice naturalness, verbosity, and mood.
-
gemini-2.5-flash-preview-09-2025The latest model based on the 2.5 Flash model. 2.5 Flash Preview is best for large scale processing, low-latency, high volume tasks that require thinking, and agentic use cases.
-
gemini-2.5-flash-preview-ttsOur 2.5 Flash text-to-speech model optimized for powerful, low-latency controllable speech generation.
-
gemini-2.5-proOur state-of-the-art multipurpose model, which excels at coding and complex reasoning tasks.
-
gemini-2.5-pro-preview-ttsOur 2.5 Pro text-to-speech audio model optimized for powerful, low-latency speech generation for more natural outputs and easier to steer prompts.
-
gemini-3-flash-previewOur most intelligent model built for speed, combining frontier intelligence with superior search and grounding.
-
gemini-3-pro-image-previewState-of-the-art image generation and editing model.
-
gemini-3-pro-previewOur most intelligent model with SOTA reasoning and multimodal understanding, and powerful agentic and vibe coding capabilities.
-
gemini-3.1-pro-previewOur latest SOTA reasoning model with unprecedented depth and nuance, and powerful multimodal understanding and coding capabilities.
-
gemini-3.1-flash-image-previewPro-level visual intelligence with Flash-speed efficiency and reality-grounded generation capabilities.
agent AgentOption (optional)
The name of the `Agent` used for generating the interaction.
Possible values:
-
deep-research-pro-preview-12-2025Gemini Deep Research Agent
Required. Output only. A unique identifier for the interaction completion.
Required. Output only. The status of the interaction.
Possible values:
-
in_progress -
requires_action -
completed -
failed -
cancelled -
incomplete
Required. Output only. The time at which the response was created in ISO 8601 format (YYYY-MM-DDThh:mm:ssZ).
Required. Output only. The time at which the response was last updated in ISO 8601 format (YYYY-MM-DDThh:mm:ssZ).
Output only. The role of the interaction.
Output only. Responses from the model.
System instruction for the interaction.
A list of tool declarations the model may call during interaction.
usage Usage (optional)
Output only. Statistics on the interaction request's token usage.
Fields
Number of tokens in the prompt (context).
input_tokens_by_modality ModalityTokens (optional)
A breakdown of input token usage by modality.
Fields
modality ResponseModality (optional)
The modality associated with the token count.
Possible values:
-
text -
image -
audio
Number of tokens for the modality.
Number of tokens in the cached part of the prompt (the cached content).
cached_tokens_by_modality ModalityTokens (optional)
A breakdown of cached token usage by modality.
Fields
modality ResponseModality (optional)
The modality associated with the token count.
Possible values:
-
text -
image -
audio
Number of tokens for the modality.
Total number of tokens across all the generated responses.
output_tokens_by_modality ModalityTokens (optional)
A breakdown of output token usage by modality.
Fields
modality ResponseModality (optional)
The modality associated with the token count.
Possible values:
-
text -
image -
audio
Number of tokens for the modality.
Number of tokens present in tool-use prompt(s).
tool_use_tokens_by_modality ModalityTokens (optional)
A breakdown of tool-use token usage by modality.
Fields
modality ResponseModality (optional)
The modality associated with the token count.
Possible values:
-
text -
image -
audio
Number of tokens for the modality.
Number of tokens of thoughts for thinking models.
Total token count for the interaction request (prompt + responses + other internal tokens).
response_modalities ResponseModality (optional)
The requested modalities of the response (TEXT, IMAGE, AUDIO).
Possible values:
-
text -
image -
audio
Enforces that the generated response is a JSON object that complies with the JSON schema specified in this field.
The mime type of the response. This is required if response_format is set.
The ID of the previous interaction, if any.
The input for the interaction.
agent_config object (optional)
Configuration parameters for the agent interaction.
Possible Types
Polymorphic discriminator: type
DynamicAgentConfig
Configuration for dynamic agents.
No description provided.
Always set to "dynamic".
DeepResearchAgentConfig
Configuration for the Deep Research agent.
No description provided.
Always set to "deep-research".
thinking_summaries ThinkingSummaries (optional)
Whether to include thought summaries in the response.
Possible values:
-
auto -
none
Examples
Example
{ "created": "2025-12-04T15:01:45Z", "id": "v1_ChdXS0l4YWZXTk9xbk0xZThQczhEcmlROBIXV0tJeGFmV05PcW5NMWU4UHM4RHJpUTg", "model": "gemini-3-flash-preview", "object": "interaction", "outputs": [ { "text": "Hello! I'm doing well, functioning as expected. Thank you for asking! How are you doing today?", "type": "text" } ], "role": "model", "status": "completed", "updated": "2025-12-04T15:01:45Z", "usage": { "input_tokens_by_modality": [ { "modality": "text", "tokens": 7 } ], "total_cached_tokens": 0, "total_input_tokens": 7, "total_output_tokens": 23, "total_thought_tokens": 49, "total_tokens": 79, "total_tool_use_tokens": 0 } }
Data Models
Content
The content of the response.
Possible Types
Polymorphic discriminator: type
TextContent
A text content block.
No description provided.
Always set to "text".
Required. The text content.
annotations Annotation (optional)
Citation information for model-generated content.
Possible Types
Polymorphic discriminator: type
UrlCitation
A URL citation annotation.
No description provided.
Always set to "url_citation".
The URL.
The title of the URL.
Start of segment of the response that is attributed to this source. Index indicates the start of the segment, measured in bytes.
End of the attributed segment, exclusive.
FileCitation
A file citation annotation.
No description provided.
Always set to "file_citation".
The URI of the file.
The name of the file.
Source attributed for a portion of the text.
Start of segment of the response that is attributed to this source. Index indicates the start of the segment, measured in bytes.
End of the attributed segment, exclusive.
PlaceCitation
A place citation annotation.
No description provided.
Always set to "place_citation".
The ID of the place, in `places/{place_id}` format.
Title of the place.
URI reference of the place.
review_snippets ReviewSnippet (optional)
Snippets of reviews that are used to generate answers about the features of a given place in Google Maps.
Fields
Title of the review.
A link that corresponds to the user review on Google Maps.
The ID of the review snippet.
Start of segment of the response that is attributed to this source. Index indicates the start of the segment, measured in bytes.
End of the attributed segment, exclusive.
ImageContent
An image content block.
No description provided.
Always set to "image".
The image content.
The URI of the image.
The mime type of the image.
Possible values:
-
image/png -
image/jpeg -
image/webp -
image/heic -
image/heif -
image/jpg -
image/gif -
image/bmp -
image/tiff
resolution MediaResolution (optional)
The resolution of the media.
Possible values:
-
low -
medium -
high -
ultra_high
AudioContent
An audio content block.
No description provided.
Always set to "audio".
The audio content.
The URI of the audio.
The mime type of the audio.
Possible values:
-
audio/wav -
audio/mp3 -
audio/aiff -
audio/aac -
audio/ogg -
audio/flac -
audio/mpeg -
audio/m4a
DocumentContent
A document content block.
No description provided.
Always set to "document".
The document content.
The URI of the document.
The mime type of the document.
Possible values:
-
application/pdf
VideoContent
A video content block.
No description provided.
Always set to "video".
The video content.
The URI of the video.
The mime type of the video.
Possible values:
-
video/mp4 -
video/mpeg -
video/mpg -
video/mov -
video/avi -
video/x-flv -
video/webm -
video/wmv -
video/3gpp
resolution MediaResolution (optional)
The resolution of the media.
Possible values:
-
low -
medium -
high -
ultra_high
ThoughtContent
A thought content block.
No description provided.
Always set to "thought".
Signature to match the backend source to be part of the generation.
summary ThoughtSummaryContent (optional)
A summary of the thought.
Possible Types
Polymorphic discriminator: type
TextContent
A text content block.
No description provided.
Always set to "text".
Required. The text content.
annotations Annotation (optional)
Citation information for model-generated content.
Possible Types
Polymorphic discriminator: type
UrlCitation
A URL citation annotation.
No description provided.
Always set to "url_citation".
The URL.
The title of the URL.
Start of segment of the response that is attributed to this source. Index indicates the start of the segment, measured in bytes.
End of the attributed segment, exclusive.
FileCitation
A file citation annotation.
No description provided.
Always set to "file_citation".
The URI of the file.
The name of the file.
Source attributed for a portion of the text.
Start of segment of the response that is attributed to this source. Index indicates the start of the segment, measured in bytes.
End of the attributed segment, exclusive.
PlaceCitation
A place citation annotation.
No description provided.
Always set to "place_citation".
The ID of the place, in `places/{place_id}` format.
Title of the place.
URI reference of the place.
review_snippets ReviewSnippet (optional)
Snippets of reviews that are used to generate answers about the features of a given place in Google Maps.
Fields
Title of the review.
A link that corresponds to the user review on Google Maps.
The ID of the review snippet.
Start of segment of the response that is attributed to this source. Index indicates the start of the segment, measured in bytes.
End of the attributed segment, exclusive.
ImageContent
An image content block.
No description provided.
Always set to "image".
The image content.
The URI of the image.
The mime type of the image.
Possible values:
-
image/png -
image/jpeg -
image/webp -
image/heic -
image/heif -
image/jpg -
image/gif -
image/bmp -
image/tiff
resolution MediaResolution (optional)
The resolution of the media.
Possible values:
-
low -
medium -
high -
ultra_high
FunctionCallContent
A function tool call content block.
No description provided.
Always set to "function_call".
Required. The name of the tool to call.
Required. The arguments to pass to the function.
Required. A unique ID for this specific tool call.
A signature hash for backend validation.
CodeExecutionCallContent
Code execution content.
No description provided.
Always set to "code_execution_call".
arguments CodeExecutionCallArguments (required)
Required. The arguments to pass to the code execution.
Fields
Programming language of the `code`.
Possible values:
-
python
The code to be executed.
Required. A unique ID for this specific tool call.
A signature hash for backend validation.
UrlContextCallContent
URL context content.
No description provided.
Always set to "url_context_call".
arguments UrlContextCallArguments (required)
Required. The arguments to pass to the URL context.
Fields
The URLs to fetch.
Required. A unique ID for this specific tool call.
A signature hash for backend validation.
McpServerToolCallContent
MCPServer tool call content.
No description provided.
Always set to "mcp_server_tool_call".
Required. The name of the tool which was called.
Required. The name of the used MCP server.
Required. The JSON object of arguments for the function.
Required. A unique ID for this specific tool call.
A signature hash for backend validation.
GoogleSearchCallContent
Google Search content.
No description provided.
Always set to "google_search_call".
arguments GoogleSearchCallArguments (required)
Required. The arguments to pass to Google Search.
Fields
Web search queries for the following-up web search.
The type of search grounding enabled.
Possible values:
-
web_search -
image_search
Required. A unique ID for this specific tool call.
A signature hash for backend validation.
FileSearchCallContent
File Search content.
No description provided.
Always set to "file_search_call".
Required. A unique ID for this specific tool call.
A signature hash for backend validation.
GoogleMapsCallContent
Google Maps content.
No description provided.
Always set to "google_maps_call".
arguments GoogleMapsCallArguments (optional)
The arguments to pass to the Google Maps tool.
Fields
The queries to be executed.
Required. A unique ID for this specific tool call.
A signature hash for backend validation.
FunctionResultContent
A function tool result content block.
No description provided.
Always set to "function_result".
The name of the tool that was called.
Whether the tool call resulted in an error.
Required. ID to match the ID from the function call block.
A signature hash for backend validation.
The result of the tool call.
CodeExecutionResultContent
Code execution result content.
No description provided.
Always set to "code_execution_result".
Required. The output of the code execution.
Whether the code execution resulted in an error.
Required. ID to match the ID from the function call block.
A signature hash for backend validation.
UrlContextResultContent
URL context result content.
No description provided.
Always set to "url_context_result".
result UrlContextResult (required)
Required. The results of the URL context.
Fields
The URL that was fetched.
The status of the URL retrieval.
Possible values:
-
success -
error -
paywall -
unsafe
Whether the URL context resulted in an error.
Required. ID to match the ID from the function call block.
A signature hash for backend validation.
GoogleSearchResultContent
Google Search result content.
No description provided.
Always set to "google_search_result".
result GoogleSearchResult (required)
Required. The results of the Google Search.
Fields
Web content snippet that can be embedded in a web page or an app webview.
Whether the Google Search resulted in an error.
Required. ID to match the ID from the function call block.
A signature hash for backend validation.
McpServerToolResultContent
MCPServer tool result content.
No description provided.
Always set to "mcp_server_tool_result".
Name of the tool which is called for this specific tool call.
The name of the used MCP server.
Required. ID to match the ID from the function call block.
A signature hash for backend validation.
The output from the MCP server call. Can be simple text or rich content.
FileSearchResultContent
File Search result content.
No description provided.
Always set to "file_search_result".
result FileSearchResult (required)
Required. The results of the File Search.
Fields
User provided metadata about the FileSearchResult.
Required. ID to match the ID from the function call block.
A signature hash for backend validation.
GoogleMapsResultContent
Google Maps result content.
No description provided.
Always set to "google_maps_result".
result GoogleMapsResult (required)
Required. The results of the Google Maps.
Fields
places Places (optional)
The places that were found.
Fields
The ID of the place, in `places/{place_id}` format.
Title of the place.
URI reference of the place.
review_snippets ReviewSnippet (optional)
Snippets of reviews that are used to generate answers about the features of a given place in Google Maps.
Fields
Title of the review.
A link that corresponds to the user review on Google Maps.
The ID of the review snippet.
Resource name of the Google Maps widget context token.
Required. ID to match the ID from the function call block.
A signature hash for backend validation.
Examples
Text
{ "type": "text", "text": "Hello, how are you?" }
Image
{ "type": "image", "data": "BASE64_ENCODED_IMAGE", "mime_type": "image/png" }
Audio
{ "type": "audio", "data": "BASE64_ENCODED_AUDIO", "mime_type": "audio/wav" }
Document
{ "type": "document", "data": "BASE64_ENCODED_DOCUMENT", "mime_type": "application/pdf" }
Video
{ "type": "video", "uri": "https://www.youtube.com/watch?v=9hE5-98ZeCg" }
Thought
{ "type": "thought", "summary": [ { "type": "text", "text": "The user is asking about the weather. I should use the get_weather tool." } ], "signature": "CoMDAXLI2nynRYojJIy6B1Jh9os2crpWLfB0+19xcLsGG46bd8wjkF/6RNlRUdvHrXyjsHkG0BZFcuO/bPOyA6Xh5jANNgx82wPHjGExN8A4ZQn56FlMwyZoqFVQz0QyY1lfibFJ2zU3J87uw26OewzcuVX0KEcs+GIsZa3EA6WwqhbsOd3wtZB3Ua2Qf98VAWZTS5y/tWpql7jnU3/CU7pouxQr/Bwft3hwnJNesQ9/dDJTuaQ8Zprh9VRWf1aFFjpIueOjBRrlT3oW6/y/eRl/Gt9BQXCYTqg/38vHFUU4Wo/d9dUpvfCe/a3o97t2Jgxp34oFKcsVb4S5WJrykIkw+14DzVnTpCpbQNFckqvFLuqnJCkL0EQFtunBXI03FJpPu3T1XU6id8S7ojoJQZSauGUCgmaLqUGdMrd08oo81ecoJSLs51Re9N/lISGmjWFPGpqJLoGq6uo4FHz58hmeyXCgHG742BHz2P3MiH1CXHUT2J8mF6zLhf3SR9Qb3lkrobAh" }
Function Call
{ "type": "function_call", "name": "get_weather", "id": "gth23981", "arguments": { "location": "Boston, MA" } }
Code Execution Call
{ "type": "code_execution_call", "id": "call_123456", "arguments": { "language": "python", "code": "print('hello world')" } }
Url Context Call
{ "type": "url_context_call", "id": "call_123456", "arguments": { "urls": [ "https://www.example.com" ] } }
Mcp Server Tool Call
{ "type": "mcp_server_tool_call", "id": "call_123456", "name": "get_forecast", "server_name": "weather_server", "arguments": { "city": "London" } }
Google Search Call
{ "type": "google_search_call", "id": "call_123456", "arguments": { "queries": [ "weather in Boston" ] } }
File Search Call
{ "type": "file_search_call", "id": "call_123456" }
Google Maps Call
{ "type": "google_maps_call", "id": "call_123456", "arguments": { "query": "best food near me" } }
Function Result
{ "type": "function_result", "name": "get_weather", "call_id": "gth23981", "result": [ { "type": "text", "text": "{\"weather\":\"sunny\"}" } ] }
Code Execution Result
{ "type": "code_execution_result", "call_id": "call_123456", "result": "hello world" }
Url Context Result
{ "type": "url_context_result", "call_id": "call_123456", "result": [ { "url": "https://www.example.com", "status": "SUCCESS" } ] }
Google Search Result
{ "type": "google_search_result", "call_id": "call_123456", "result": [ { "url": "https://www.google.com/search?q=weather+in+Boston", "title": "Weather in Boston" } ] }
Mcp Server Tool Result
{ "type": "mcp_server_tool_result", "name": "get_forecast", "server_name": "weather_server", "call_id": "call_123456", "result": "sunny" }
File Search Result
{ "type": "file_search_result", "call_id": "call_123456", "result": [ { "text": "search result chunk", "file_search_store": "file_search_store" } ] }
Google Maps Result
{ "type": "google_maps_result", "call_id": "call_123456", "result": [ { "places": [ { "url": "https://www.google.com/maps/search/best+food+near+me", "name": "Tasty Restaurant" } ] } ] }
Tool
A tool that can be used by the model.
Possible Types
Polymorphic discriminator: type
Function
A tool that can be used by the model.
No description provided.
Always set to "function".
The name of the function.
A description of the function.
The JSON Schema for the function's parameters.
CodeExecution
A tool that can be used by the model to execute code.
No description provided.
Always set to "code_execution".
UrlContext
A tool that can be used by the model to fetch URL context.
No description provided.
Always set to "url_context".
ComputerUse
A tool that can be used by the model to interact with the computer.
No description provided.
Always set to "computer_use".
The environment being operated.
Possible values:
-
browser
The list of predefined functions that are excluded from the model call.
McpServer
A MCPServer is a server that can be called by the model to perform actions.
No description provided.
Always set to "mcp_server".
The name of the MCPServer.
The full URL for the MCPServer endpoint. Example: "https://api.example.com/mcp"
Optional: Fields for authentication headers, timeouts, etc., if needed.
allowed_tools AllowedTools (optional)
The allowed tools.
Fields
mode ToolChoiceType (optional)
The mode of the tool choice.
Possible values:
-
auto -
any -
none -
validated
The names of the allowed tools.
GoogleSearch
A tool that can be used by the model to search Google.
No description provided.
Always set to "google_search".
The types of search grounding to enable.
Possible values:
-
web_search -
image_search
FileSearch
A tool that can be used by the model to search files.
No description provided.
Always set to "file_search".
The file search store names to search.
The number of semantic retrieval chunks to retrieve.
Metadata filter to apply to the semantic retrieval documents and chunks.
GoogleMaps
A tool that can be used by the model to call Google Maps.
No description provided.
Always set to "google_maps".
Whether to return a widget context token in the tool call result of the response.
The latitude of the user's location.
The longitude of the user's location.
Examples
Function
CodeExecution
UrlContext
ComputerUse
McpServer
GoogleSearch
FileSearch
GoogleMaps
Turn
Fields
The originator of this turn. Must be user for input or model for model output.
No description provided.
Examples
User Turn
{ "role": "user", "content": [ { "type": "text", "text": "user turn" } ] }
Model Turn
{ "role": "model", "content": [ { "type": "text", "text": "model turn" } ] }
InteractionSseEvent
Possible Types
Polymorphic discriminator: event_type
InteractionStartEvent
No description provided.
Always set to "interaction.start".
No description provided.
The event_id token to be used to resume the interaction stream, from this event.
InteractionCompleteEvent
No description provided.
Always set to "interaction.complete".
Required. The completed interaction with empty outputs to reduce the payload size. Use the preceding ContentDelta events for the actual output.
The event_id token to be used to resume the interaction stream, from this event.
InteractionStatusUpdate
No description provided.
Always set to "interaction.status_update".
No description provided.
No description provided.
Possible values:
-
in_progress -
requires_action -
completed -
failed -
cancelled -
incomplete
The event_id token to be used to resume the interaction stream, from this event.
ContentStart
No description provided.
Always set to "content.start".
No description provided.
No description provided.
The event_id token to be used to resume the interaction stream, from this event.
ContentDelta
No description provided.
Always set to "content.delta".
No description provided.
delta ContentDeltaData (required)
No description provided.
Possible Types
Polymorphic discriminator: type
TextDelta
No description provided.
Always set to "text".
No description provided.
annotations Annotation (optional)
Citation information for model-generated content.
Possible Types
Polymorphic discriminator: type
UrlCitation
A URL citation annotation.
No description provided.
Always set to "url_citation".
The URL.
The title of the URL.
Start of segment of the response that is attributed to this source. Index indicates the start of the segment, measured in bytes.
End of the attributed segment, exclusive.
FileCitation
A file citation annotation.
No description provided.
Always set to "file_citation".
The URI of the file.
The name of the file.
Source attributed for a portion of the text.
Start of segment of the response that is attributed to this source. Index indicates the start of the segment, measured in bytes.
End of the attributed segment, exclusive.
PlaceCitation
A place citation annotation.
No description provided.
Always set to "place_citation".
The ID of the place, in `places/{place_id}` format.
Title of the place.
URI reference of the place.
review_snippets ReviewSnippet (optional)
Snippets of reviews that are used to generate answers about the features of a given place in Google Maps.
Fields
Title of the review.
A link that corresponds to the user review on Google Maps.
The ID of the review snippet.
Start of segment of the response that is attributed to this source. Index indicates the start of the segment, measured in bytes.
End of the attributed segment, exclusive.
ImageDelta
No description provided.
Always set to "image".
No description provided.
No description provided.
No description provided.
Possible values:
-
image/png -
image/jpeg -
image/webp -
image/heic -
image/heif -
image/jpg -
image/gif -
image/bmp -
image/tiff
resolution MediaResolution (optional)
The resolution of the media.
Possible values:
-
low -
medium -
high -
ultra_high
AudioDelta
No description provided.
Always set to "audio".
No description provided.
No description provided.
No description provided.
Possible values:
-
audio/wav -
audio/mp3 -
audio/aiff -
audio/aac -
audio/ogg -
audio/flac -
audio/mpeg -
audio/m4a
DocumentDelta
No description provided.
Always set to "document".
No description provided.
No description provided.
No description provided.
Possible values:
-
application/pdf
VideoDelta
No description provided.
Always set to "video".
No description provided.
No description provided.
No description provided.
Possible values:
-
video/mp4 -
video/mpeg -
video/mpg -
video/mov -
video/avi -
video/x-flv -
video/webm -
video/wmv -
video/3gpp
resolution MediaResolution (optional)
The resolution of the media.
Possible values:
-
low -
medium -
high -
ultra_high
ThoughtSummaryDelta
No description provided.
Always set to "thought_summary".
content ThoughtSummaryContent (optional)
A new summary item to be added to the thought.
Possible Types
Polymorphic discriminator: type
TextContent
A text content block.
No description provided.
Always set to "text".
Required. The text content.
annotations Annotation (optional)
Citation information for model-generated content.
Possible Types
Polymorphic discriminator: type
UrlCitation
A URL citation annotation.
No description provided.
Always set to "url_citation".
The URL.
The title of the URL.
Start of segment of the response that is attributed to this source. Index indicates the start of the segment, measured in bytes.
End of the attributed segment, exclusive.
FileCitation
A file citation annotation.
No description provided.
Always set to "file_citation".
The URI of the file.
The name of the file.
Source attributed for a portion of the text.
Start of segment of the response that is attributed to this source. Index indicates the start of the segment, measured in bytes.
End of the attributed segment, exclusive.
PlaceCitation
A place citation annotation.
No description provided.
Always set to "place_citation".
The ID of the place, in `places/{place_id}` format.
Title of the place.
URI reference of the place.
review_snippets ReviewSnippet (optional)
Snippets of reviews that are used to generate answers about the features of a given place in Google Maps.
Fields
Title of the review.
A link that corresponds to the user review on Google Maps.
The ID of the review snippet.
Start of segment of the response that is attributed to this source. Index indicates the start of the segment, measured in bytes.
End of the attributed segment, exclusive.
ImageContent
An image content block.
No description provided.
Always set to "image".
The image content.
The URI of the image.
The mime type of the image.
Possible values:
-
image/png -
image/jpeg -
image/webp -
image/heic -
image/heif -
image/jpg -
image/gif -
image/bmp -
image/tiff
resolution MediaResolution (optional)
The resolution of the media.
Possible values:
-
low -
medium -
high -
ultra_high
ThoughtSignatureDelta
No description provided.
Always set to "thought_signature".
Signature to match the backend source to be part of the generation.
FunctionCallDelta
No description provided.
Always set to "function_call".
No description provided.
No description provided.
Required. A unique ID for this specific tool call.
A signature hash for backend validation.
CodeExecutionCallDelta
No description provided.
Always set to "code_execution_call".
arguments CodeExecutionCallArguments (required)
No description provided.
Fields
Programming language of the `code`.
Possible values:
-
python
The code to be executed.
Required. A unique ID for this specific tool call.
A signature hash for backend validation.
UrlContextCallDelta
No description provided.
Always set to "url_context_call".
arguments UrlContextCallArguments (required)
No description provided.
Fields
The URLs to fetch.
Required. A unique ID for this specific tool call.
A signature hash for backend validation.
GoogleSearchCallDelta
No description provided.
Always set to "google_search_call".
arguments GoogleSearchCallArguments (required)
No description provided.
Fields
Web search queries for the following-up web search.
Required. A unique ID for this specific tool call.
A signature hash for backend validation.
McpServerToolCallDelta
No description provided.
Always set to "mcp_server_tool_call".
No description provided.
No description provided.
No description provided.
Required. A unique ID for this specific tool call.
A signature hash for backend validation.
FileSearchCallDelta
No description provided.
Always set to "file_search_call".
Required. A unique ID for this specific tool call.
A signature hash for backend validation.
GoogleMapsCallDelta
No description provided.
Always set to "google_maps_call".
arguments GoogleMapsCallArguments (optional)
The arguments to pass to the Google Maps tool.
Fields
The queries to be executed.
Required. A unique ID for this specific tool call.
A signature hash for backend validation.
FunctionResultDelta
No description provided.
Always set to "function_result".
No description provided.
No description provided.
Required. ID to match the ID from the function call block.
A signature hash for backend validation.
No description provided.
CodeExecutionResultDelta
No description provided.
Always set to "code_execution_result".
No description provided.
No description provided.
Required. ID to match the ID from the function call block.
A signature hash for backend validation.
UrlContextResultDelta
No description provided.
Always set to "url_context_result".
result UrlContextResult (required)
No description provided.
Fields
The URL that was fetched.
The status of the URL retrieval.
Possible values:
-
success -
error -
paywall -
unsafe
No description provided.
Required. ID to match the ID from the function call block.
A signature hash for backend validation.
GoogleSearchResultDelta
No description provided.
Always set to "google_search_result".
result GoogleSearchResult (required)
No description provided.
Fields
Web content snippet that can be embedded in a web page or an app webview.
No description provided.
Required. ID to match the ID from the function call block.
A signature hash for backend validation.
McpServerToolResultDelta
No description provided.
Always set to "mcp_server_tool_result".
No description provided.
No description provided.
Required. ID to match the ID from the function call block.
A signature hash for backend validation.
No description provided.
FileSearchResultDelta
No description provided.
Always set to "file_search_result".
result FileSearchResult (required)
No description provided.
Fields
User provided metadata about the FileSearchResult.
Required. ID to match the ID from the function call block.
A signature hash for backend validation.
GoogleMapsResultDelta
No description provided.
Always set to "google_maps_result".
result GoogleMapsResult (optional)
The results of the Google Maps.
Fields
places Places (optional)
The places that were found.
Fields
The ID of the place, in `places/{place_id}` format.
Title of the place.
URI reference of the place.
review_snippets ReviewSnippet (optional)
Snippets of reviews that are used to generate answers about the features of a given place in Google Maps.
Fields
Title of the review.
A link that corresponds to the user review on Google Maps.
The ID of the review snippet.
Resource name of the Google Maps widget context token.
Required. ID to match the ID from the function call block.
A signature hash for backend validation.
The event_id token to be used to resume the interaction stream, from this event.
ContentStop
No description provided.
Always set to "content.stop".
No description provided.
The event_id token to be used to resume the interaction stream, from this event.
ErrorEvent
No description provided.
Always set to "error".
error Error (optional)
No description provided.
Fields
A URI that identifies the error type.
A human-readable error message.
The event_id token to be used to resume the interaction stream, from this event.
Examples
Interaction Start
{ "event_type": "interaction.start", "interaction": { "id": "v1_ChdTMjQ0YWJ5TUF1TzcxZThQdjRpcnFRcxIXUzI0NGFieU1BdU83MWU4UHY0aXJxUXM", "model": "gemini-3-flash-preview", "object": "interaction", "status": "in_progress" } }
Interaction Complete
{ "event_type": "interaction.complete", "interaction": { "created": "2025-12-09T18:45:40Z", "id": "v1_ChdTMjQ0YWJ5TUF1TzcxZThQdjRpcnFRcxIXUzI0NGFieU1BdU83MWU4UHY0aXJxUXM", "model": "gemini-3-flash-preview", "object": "interaction", "role": "model", "status": "completed", "updated": "2025-12-09T18:45:40Z", "usage": { "input_tokens_by_modality": [ { "modality": "text", "tokens": 11 } ], "total_cached_tokens": 0, "total_input_tokens": 11, "total_output_tokens": 364, "total_thought_tokens": 1120, "total_tokens": 1495, "total_tool_use_tokens": 0 } } }
Interaction Status Update
{ "event_type": "interaction.status_update", "interaction_id": "v1_ChdTMjQ0YWJ5TUF1TzcxZThQdjRpcnFRcxIXUzI0NGFieU1BdU83MWU4UHY0aXJxUXM", "status": "in_progress" }
Content Start
{ "event_type": "content.start", "content": { "type": "text" }, "index": 1 }
Content Delta
{ "event_type": "content.delta", "delta": { "type": "text", "text": "Elara\u2019s life was a symphony of quiet moments. A librarian, she found solace in the hushed aisles, the scent of aged paper, and the predictable rhythm of her days. Her small apartment, meticulously ordered, reflected this internal calm, save" }, "index": 1 }
Content Stop
{ "event_type": "content.stop", "index": 1 }
Error Event
{ "event_type": "error", "error": { "message": "Failed to get completed interaction: Result not found.", "code": "not_found" } }