# Gemini Interactions API The Gemini Interactions API is an experimental API that allows developers to build generative AI applications using Gemini models. Gemini is our most capable model, built from the ground up to be multimodal. It can generalize and seamlessly understand, operate across, and combine different types of information including language, images, audio, video, and code. You can use the Gemini API for use cases like reasoning across text and images, content generation, dialogue agents, summarization and classification systems, and more. ## Interactions ### Creating an interaction `POST https://generativelanguage.googleapis.com/v1beta/interactions` Creates a new interaction. #### Parameters - **api_version** (`string`) Which version of the API to use. #### Request Body - **model** (`ModelOption`) The name of the `Model` used for generating the interaction.
Required if `agent` is not provided. Possible values: - `gemini-2.5-computer-use-preview-10-2025`: An agentic capability model designed for direct interface interaction, allowing Gemini to perceive and navigate digital environments. - `gemini-3.1-flash-tts-preview`: Gemini 3.1 Flash TTS: Powerful, low-latency speech generation. Enjoy natural outputs, steerable prompts, and new expressive audio tags for precise narration control. - `gemini-2.5-flash-preview-tts`: Our 2.5 Flash text-to-speech model optimized for powerful, low-latency controllable speech generation. - `gemini-2.5-pro-preview-tts`: Our 2.5 Pro text-to-speech audio model optimized for powerful, low-latency speech generation for more natural outputs and easier to steer prompts. - `lyria-3-pro-preview`: Our advanced, full-song generative model with deep compositional understanding, optimized for precise structural control and complex transitions across diverse musical styles. - `gemini-2.5-flash`: Our first hybrid reasoning model which supports a 1M token context window and has thinking budgets. - `gemini-3.1-pro-preview`: Our latest SOTA reasoning model with unprecedented depth and nuance, and powerful multimodal understanding and coding capabilities. - `lyria-3-clip-preview`: Our low-latency, music generation model optimized for high-fidelity audio clips and precise rhythmic control. - `gemini-3.1-flash-lite`: Our most cost-efficient model, optimized for high-volume agentic tasks, translation, and simple data processing. - `gemini-3.1-flash-lite-preview`: Our most cost-efficient model, optimized for high-volume agentic tasks, translation, and simple data processing. - `gemini-3-flash-preview`: Our most intelligent model built for speed, combining frontier intelligence with superior search and grounding. - `gemini-3.5-flash`: Our most intelligent model for sustained frontier performance in agentic and coding tasks. - `gemini-3-pro-preview`: Our most intelligent model with SOTA reasoning and multimodal understanding, and powerful agentic and vibe coding capabilities. - `gemini-2.5-flash-native-audio-preview-12-2025`: Our native audio models optimized for higher quality audio outputs with better pacing, voice naturalness, verbosity, and mood. - `gemini-2.5-flash-image`: Our native image generation model, optimized for speed, flexibility, and contextual understanding. Text input and output is priced the same as 2.5 Flash. - `gemini-2.5-flash-lite`: Our smallest and most cost effective model, built for at scale usage. - `gemini-2.5-pro`: Our state-of-the-art multipurpose model, which excels at coding and complex reasoning tasks. - `gemini-3.1-flash-image-preview`: Pro-level visual intelligence with Flash-speed efficiency and reality-grounded generation capabilities. - `gemini-3-pro-image-preview`: State-of-the-art image generation and editing model. - `gemini-2.5-flash-lite-preview-09-2025`: The latest model based on Gemini 2.5 Flash lite optimized for cost-efficiency, high throughput and high quality. - `gemini-2.5-flash-preview-09-2025`: The latest model based on the 2.5 Flash model. 2.5 Flash Preview is best for large scale processing, low-latency, high volume tasks that require thinking, and agentic use cases. - **agent** (`AgentOption`) The name of the `Agent` used for generating the interaction.
Required if `model` is not provided. Possible values: - `deep-research-preview-04-2026`: Gemini Deep Research Agent - `deep-research-pro-preview-12-2025`: Gemini Deep Research Agent - `deep-research-max-preview-04-2026`: Gemini Deep Research Max Agent - `antigravity-preview-05-2026`: Use the Antigravity managed agent to perform multi-step tasks that require reasoning, file operations, and tool use. - **input** (`Content or array (Content) or array (Step) or string`) *(Required)* The inputs for the interaction (common to both Model and Agent). - **system_instruction** (`string`) System instruction for the interaction. - **tools** (`array (Tool)`) A list of tool declarations the model may call during interaction. - **response_format** (`ResponseFormat or ResponseFormatList`) Enforces that the generated response is a JSON object that complies with the JSON schema specified in this field. - **stream** (`boolean`) Input only. Whether the interaction will be streamed. - **store** (`boolean`) Input only. Whether to store the response and request for later retrieval. - **background** (`boolean`) Input only. Whether to run the model interaction in the background. - **generation_config** (`GenerationConfig`) Model Configuration
Configuration parameters for the model interaction.
Alternative to `agent_config`. Only applicable when `model` is set. - **temperature** (`number`) Controls the randomness of the output. - **top_p** (`number`) The maximum cumulative probability of tokens to consider when sampling. - **seed** (`integer`) Seed used in decoding for reproducibility. - **stop_sequences** (`array (string)`) A list of character sequences that will stop output interaction. - **thinking_level** (`ThinkingLevel`) The level of thought tokens that the model should generate. Possible values: - `minimal`: Little to no thinking. - `low`: Low thinking level. - `medium`: Medium thinking level. - `high`: High thinking level. - **thinking_summaries** (`ThinkingSummaries`) Whether to include thought summaries in the response. Possible values: - `auto`: Auto thinking summaries. - `none`: No thinking summaries. - **max_output_tokens** (`integer`) The maximum number of tokens to include in the response. - **speech_config** (`array (SpeechConfig)`) Configuration for speech interaction. - **voice** (`string`) The voice of the speaker. - **language** (`string`) The language of the speech. - **speaker** (`string`) The speaker's name, it should match the speaker name given in the prompt. - **tool_choice** (`ToolChoiceConfig or ToolChoiceType`) The tool choice configuration. - **agent_config** (`DeepResearchAgentConfig or DynamicAgentConfig`) Agent Configuration
Configuration for the agent.
Alternative to `generation_config`. Only applicable when `agent` is set. **Possible Types:** (Discriminator: `type`) - **DeepResearchAgentConfig**: Configuration for the Deep Research agent. - **type** (`object`) *(Required)* Value: `deep-research` - **thinking_summaries** (`ThinkingSummaries`) Whether to include thought summaries in the response. Possible values: - `auto`: Auto thinking summaries. - `none`: No thinking summaries. - **visualization** (`enum (string)`) Whether to include visualizations in the response. Possible values: - `off`: Do not include visualizations. - `auto`: Automatically include visualizations. - **collaborative_planning** (`boolean`) Enables human-in-the-loop planning for the Deep Research agent. If set to true, the Deep Research agent will provide a research plan in its response. The agent will then proceed only if the user confirms the plan in the next turn. - **enable_bigquery_tool** (`boolean`) Enables bigquery tool for the Deep Research agent. - **DynamicAgentConfig**: Configuration for dynamic agents. - **type** (`object`) *(Required)* Value: `dynamic` - **environment** (`EnvironmentConfig or string`) The environment configuration for the interaction. Can be an object specifying remote environment sources or a string referencing an existing environment ID. - **previous_interaction_id** (`string`) The ID of the previous interaction, if any. - **response_modalities** (`array (ResponseModality)`) The requested modalities of the response (TEXT, IMAGE, AUDIO). Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **service_tier** (`ServiceTier`) The service tier for the interaction. Possible values: - `flex`: Flex service tier. - `standard`: Standard service tier. - `priority`: Priority service tier. - **webhook_config** (`WebhookConfig`) Optional. Webhook configuration for receiving notifications when the interaction completes. - **uris** (`array (string)`) Optional. If set, these webhook URIs will be used for webhook events instead of the registered webhooks. - **user_metadata** (`object`) Optional. The user metadata that will be returned on each event emission to the webhooks. #### Response Returns [Interaction](#interaction) resources. #### Examples **Simple Request** **REST** ```sh curl -X POST https://generativelanguage.googleapis.com/v1beta/interactions \ -H "x-goog-api-key: $GEMINI_API_KEY" \ -H "Content-Type: application/json" \ -H "Api-Revision: 2026-05-20" \ -d '{ "model": "gemini-3.5-flash", "input": "Hello, how are you?" }' ``` **Python** ```python from google import genai client = genai.Client() interaction = client.interactions.create( model="gemini-3.5-flash", input="Hello, how are you?", ) print(interaction.output_text) ``` **JavaScript** ```javascript import {GoogleGenAI} from '@google/genai'; const ai = new GoogleGenAI({}); const interaction = await ai.interactions.create({ model: 'gemini-3.5-flash', input: 'Hello, how are you?', }); console.log(interaction.output_text); ``` Response: ```json { "created": "2025-11-26T12:25:15Z", "id": "v1_ChdPU0F4YWFtNkFwS2kxZThQZ05lbXdROBIXT1NBeGFhbTZBcEtpMWU4UGdOZW13UTg", "model": "gemini-3.5-flash", "object": "interaction", "steps": [ { "type": "model_output", "content": [ { "type": "text", "text": "Hello! I'm functioning perfectly and ready to assist you.\n\nHow are you doing today?" } ] } ], "status": "completed", "updated": "2025-11-26T12:25:15Z", "usage": { "input_tokens_by_modality": [ { "modality": "text", "tokens": 7 } ], "total_cached_tokens": 0, "total_input_tokens": 7, "total_output_tokens": 20, "total_thought_tokens": 22, "total_tokens": 49, "total_tool_use_tokens": 0 } } ``` **Multi-turn** **REST** ```sh curl -X POST https://generativelanguage.googleapis.com/v1beta/interactions \ -H "x-goog-api-key: $GEMINI_API_KEY" \ -H "Content-Type: application/json" \ -H "Api-Revision: 2026-05-20" \ -d '{ "model": "gemini-3.5-flash", "input": [ { "type": "user_input", "content": [{ "type": "text", "text": "Hello!" }] }, { "type": "model_output", "content": [{ "type": "text", "text": "Hi there! How can I help you today?" }] }, { "type": "user_input", "content": [{ "type": "text", "text": "What is the capital of France?" }] } ] }' ``` **Python** ```python from google import genai client = genai.Client() response = client.interactions.create( model="gemini-3.5-flash", input=[ { "type": "user_input", "content": [{ "type": "text", "text": "Hello!" }] }, { "type": "model_output", "content": [{ "type": "text", "text": "Hi there! How can I help you today?" }] }, { "type": "user_input", "content": [{ "type": "text", "text": "What is the capital of France?" }] } ] ) print(response.output_text) ``` **JavaScript** ```javascript import {GoogleGenAI} from '@google/genai'; const ai = new GoogleGenAI({}); const interaction = await ai.interactions.create({ model: 'gemini-3.5-flash', input: [ { type: 'user_input', content: [{ type: 'text', text: 'Hello' }] }, { type: 'model_output', content: [{ type: 'text', text: 'Hi there! How can I help you today?' }] }, { type: 'user_input', content: [{ type: 'text', text: 'What is the capital of France?' }] } ] }); console.log(interaction.output_text); ``` Response: ```json { "id": "v1_ChdPU0F4YWFtNkFwS2kxZThQZ05lbXdROBIXT1NBeGFhbTZBcEtpMWU4UGdOZW13UTg", "model": "gemini-3.5-flash", "status": "completed", "object": "interaction", "created": "2025-11-26T12:22:47Z", "updated": "2025-11-26T12:22:47Z", "steps": [ { "type": "model_output", "content": [ { "type": "text", "text": "The capital of France is Paris." } ] } ], "usage": { "input_tokens_by_modality": [ { "modality": "text", "tokens": 50 } ], "total_cached_tokens": 0, "total_input_tokens": 50, "total_output_tokens": 10, "total_thought_tokens": 0, "total_tokens": 60, "total_tool_use_tokens": 0 } } ``` **Image Input** **REST** ```sh curl -X POST https://generativelanguage.googleapis.com/v1beta/interactions \ -H "x-goog-api-key: $GEMINI_API_KEY" \ -H "Content-Type: application/json" \ -H "Api-Revision: 2026-05-20" \ -d '{ "model": "gemini-3.5-flash", "input": [ { "type": "text", "text": "What is in this picture?" }, { "type": "image", "data": "BASE64_ENCODED_IMAGE", "mime_type": "image/png" } ] }' ``` **Python** ```python from google import genai client = genai.Client() response = client.interactions.create( model="gemini-3.5-flash", input=[ { "type": "text", "text": "What is in this picture?" }, { "type": "image", "data": "BASE64_ENCODED_IMAGE", "mime_type": "image/png" } ] ) print(response.output_text) ``` **JavaScript** ```javascript import {GoogleGenAI} from '@google/genai'; const ai = new GoogleGenAI({}); const interaction = await ai.interactions.create({ model: 'gemini-3.5-flash', input: [ { type: 'text', text: 'What is in this picture?' }, { type: 'image', data: 'BASE64_ENCODED_IMAGE', mime_type: 'image/png' } ] }); console.log(interaction.output_text); ``` Response: ```json { "id": "v1_ChdPU0F4YWFtNkFwS2kxZThQZ05lbXdROBIXT1NBeGFhbTZBcEtpMWU4UGdOZW13UTg", "model": "gemini-3.5-flash", "status": "completed", "object": "interaction", "created": "2025-11-26T12:22:47Z", "updated": "2025-11-26T12:22:47Z", "steps": [ { "type": "model_output", "content": [ { "type": "text", "text": "A white humanoid robot with glowing blue eyes stands holding a red skateboard." } ] } ], "usage": { "input_tokens_by_modality": [ { "modality": "text", "tokens": 10 }, { "modality": "image", "tokens": 258 } ], "total_cached_tokens": 0, "total_input_tokens": 268, "total_output_tokens": 20, "total_thought_tokens": 0, "total_tokens": 288, "total_tool_use_tokens": 0 } } ``` **Function Calling** **REST** ```sh curl -X POST https://generativelanguage.googleapis.com/v1beta/interactions \ -H "x-goog-api-key: $GEMINI_API_KEY" \ -H "Content-Type: application/json" \ -H "Api-Revision: 2026-05-20" \ -d '{ "model": "gemini-3.5-flash", "tools": [ { "type": "function", "name": "get_weather", "description": "Get the current weather in a given location", "parameters": { "type": "object", "properties": { "location": { "type": "string", "description": "The city and state, e.g. San Francisco, CA" } }, "required": [ "location" ] } } ], "input": "What is the weather like in Boston, MA?" }' ``` **Python** ```python from google import genai client = genai.Client() response = client.interactions.create( model="gemini-3.5-flash", tools=[{ "type": "function", "name": "get_weather", "description": "Get the current weather in a given location", "parameters": { "type": "object", "properties": { "location": { "type": "string", "description": "The city and state, e.g. San Francisco, CA" } }, "required": ["location"] } }], input="What is the weather like in Boston, MA?" ) print(response.steps[-1]) ``` **JavaScript** ```javascript import {GoogleGenAI} from '@google/genai'; const ai = new GoogleGenAI({}); const interaction = await ai.interactions.create({ model: 'gemini-3.5-flash', tools: [{ type: 'function', name: 'get_weather', description: 'Get the current weather in a given location', parameters: { type: 'object', properties: { location: { type: 'string', description: 'The city and state, e.g. San Francisco, CA' } }, required: ['location'] } }], input: 'What is the weather like in Boston, MA?' }); console.log(interaction.steps.at(-1)); ``` Response: ```json { "id": "v1_ChdPU0F4YWFtNkFwS2kxZThQZ05lbXdROBIXT1NBeGFhbTZBcEtpMWU4UGdOZW13UTg", "model": "gemini-3.5-flash", "status": "requires_action", "object": "interaction", "created": "2025-11-26T12:22:47Z", "updated": "2025-11-26T12:22:47Z", "steps": [ { "type": "function_call", "id": "gth23981", "name": "get_weather", "arguments": { "location": "Boston, MA" } } ], "usage": { "input_tokens_by_modality": [ { "modality": "text", "tokens": 100 } ], "total_cached_tokens": 0, "total_input_tokens": 100, "total_output_tokens": 25, "total_thought_tokens": 0, "total_tokens": 125, "total_tool_use_tokens": 50 } } ``` **Deep Research** **REST** ```sh curl -X POST https://generativelanguage.googleapis.com/v1beta/interactions \ -H "x-goog-api-key: $GEMINI_API_KEY" \ -H "Content-Type: application/json" \ -H "Api-Revision: 2026-05-20" \ -d '{ "agent": "deep-research-pro-preview-12-2025", "input": "Find a cure to cancer", "background": true }' ``` **Python** ```python from google import genai client = genai.Client() interaction = client.interactions.create( agent="deep-research-pro-preview-12-2025", input="find a cure to cancer", background=True, ) print(interaction.status) ``` **JavaScript** ```javascript import {GoogleGenAI} from '@google/genai'; const ai = new GoogleGenAI({}); const interaction = await ai.interactions.create({ agent: 'deep-research-pro-preview-12-2025', input: 'find a cure to cancer', background: true, }); console.log(interaction.status); ``` Response: ```json { "id": "v1_ChdPU0F4YWFtNkFwS2kxZThQZ05lbXdROBIXT1NBeGFhbTZBcEtpMWU4UGdOZW13UTg", "agent": "deep-research-pro-preview-12-2025", "status": "completed", "object": "interaction", "created": "2025-11-26T12:22:47Z", "updated": "2025-11-26T12:22:47Z", "steps": [ { "type": "model_output", "content": [ { "type": "text", "text": "Here is a comprehensive research report on the current state of cancer research..." } ] } ], "usage": { "input_tokens_by_modality": [ { "modality": "text", "tokens": 20 } ], "total_cached_tokens": 0, "total_input_tokens": 20, "total_output_tokens": 1000, "total_thought_tokens": 500, "total_tokens": 1520, "total_tool_use_tokens": 0 } } ``` --- ### Canceling an interaction `POST https://generativelanguage.googleapis.com/v1beta/interactions/{id}/cancel` Cancels an interaction by id. This only applies to background interactions that are still running. #### Parameters - **api_version** (`string`) Which version of the API to use. - **id** (`string`) *(Required)* The unique identifier of the interaction to cancel. #### Response Returns [Interaction](#interaction) resources. #### Examples **Cancel Interaction** **REST** ```sh curl -X POST "https://generativelanguage.googleapis.com/v1beta/interactions/$INTERACTION_ID/cancel" \ -H "x-goog-api-key: $GEMINI_API_KEY" \ -H "Api-Revision: 2026-05-20" ``` **Python** ```python from google import genai client = genai.Client() # Start a background interaction so it stays in-progress. created = client.interactions.create( model="gemini-3.5-flash", input="Write a long essay about the history of computing.", tools=[{"type": "computer_use"}], background=True, ) # Cancel the in-progress interaction. interaction = client.interactions.cancel(id=created.id) print(interaction.status) ``` **JavaScript** ```javascript import {GoogleGenAI} from '@google/genai'; const ai = new GoogleGenAI({}); // Start a background interaction so it stays in-progress. const created = await ai.interactions.create({ model: 'gemini-3.5-flash', input: 'Write a long essay about the history of computing.', tools: [{ type: 'computer_use' }], background: true, }); // Cancel the in-progress interaction. const interaction = await ai.interactions.cancel(created.id); console.log(interaction.status); ``` Response: ```json { "id": "v1_ChdPU0F4YWFtNkFwS2kxZThQZ05lbXdROBIXT1NBeGFhbTZBcEtpMWU4UGdOZW13UTg", "agent": "deep-research-pro-preview-12-2025", "status": "cancelled", "object": "interaction", "created": "2025-11-26T12:25:15Z", "updated": "2025-11-26T12:25:15Z" } ``` --- ### Retrieving an interaction `GET https://generativelanguage.googleapis.com/v1beta/interactions/{id}` Retrieves the full details of a single interaction based on its `Interaction.id`. #### Parameters - **api_version** (`string`) Which version of the API to use. - **id** (`string`) *(Required)* The unique identifier of the interaction to retrieve. - **last_event_id** (`string`) Optional. If set, resumes the interaction stream from the next chunk after the event marked by the event id. Can only be used if `stream` is true. - **stream** (`boolean`) If set to true, the generated content will be streamed incrementally. Default: `False` #### Response Returns [Interaction](#interaction) resources. #### Examples **Get Interaction** **REST** ```sh curl -X GET "https://generativelanguage.googleapis.com/v1beta/interactions/$INTERACTION_ID" \ -H "x-goog-api-key: $GEMINI_API_KEY" \ -H "Api-Revision: 2026-05-20" ``` **Python** ```python from google import genai client = genai.Client() interaction = client.interactions.get(id=created.id) print(interaction.status) ``` **JavaScript** ```javascript import {GoogleGenAI} from '@google/genai'; const ai = new GoogleGenAI({}); const interaction = await ai.interactions.get(created.id); console.log(interaction.status); ``` Response: ```json { "id": "v1_ChdPU0F4YWFtNkFwS2kxZThQZ05lbXdROBIXT1NBeGFhbTZBcEtpMWU4UGdOZW13UTg", "model": "gemini-3.5-flash", "status": "completed", "object": "interaction", "created": "2025-11-26T12:25:15Z", "updated": "2025-11-26T12:25:15Z", "steps": [ { "type": "model_output", "content": [ { "type": "text", "text": "I'm doing great, thank you for asking! How can I help you today?" } ] } ] } ``` --- ### Deleting an interaction `DELETE https://generativelanguage.googleapis.com/v1beta/interactions/{id}` Deletes the interaction by id. #### Parameters - **api_version** (`string`) Which version of the API to use. - **id** (`string`) *(Required)* The unique identifier of the interaction to delete. #### Response #### Examples **Delete Interaction** **REST** ```sh curl -X DELETE "https://generativelanguage.googleapis.com/v1beta/interactions/$INTERACTION_ID" \ -H "x-goog-api-key: $GEMINI_API_KEY" \ -H "Api-Revision: 2026-05-20" ``` **Python** ```python from google import genai client = genai.Client() client.interactions.delete(id=created.id) print("Interaction deleted successfully.") ``` **JavaScript** ```javascript import {GoogleGenAI} from '@google/genai'; const ai = new GoogleGenAI({}); await ai.interactions.delete(created.id); console.log('Interaction deleted successfully.'); ``` --- ## Resources ### Interaction { #Resource:Interaction } The Interaction resource. **Properties:** - **model** (`ModelOption`) The name of the `Model` used for generating the interaction. Possible values: - `gemini-2.5-computer-use-preview-10-2025`: An agentic capability model designed for direct interface interaction, allowing Gemini to perceive and navigate digital environments. - `gemini-3.1-flash-tts-preview`: Gemini 3.1 Flash TTS: Powerful, low-latency speech generation. Enjoy natural outputs, steerable prompts, and new expressive audio tags for precise narration control. - `gemini-2.5-flash-preview-tts`: Our 2.5 Flash text-to-speech model optimized for powerful, low-latency controllable speech generation. - `gemini-2.5-pro-preview-tts`: Our 2.5 Pro text-to-speech audio model optimized for powerful, low-latency speech generation for more natural outputs and easier to steer prompts. - `lyria-3-pro-preview`: Our advanced, full-song generative model with deep compositional understanding, optimized for precise structural control and complex transitions across diverse musical styles. - `gemini-2.5-flash`: Our first hybrid reasoning model which supports a 1M token context window and has thinking budgets. - `gemini-3.1-pro-preview`: Our latest SOTA reasoning model with unprecedented depth and nuance, and powerful multimodal understanding and coding capabilities. - `lyria-3-clip-preview`: Our low-latency, music generation model optimized for high-fidelity audio clips and precise rhythmic control. - `gemini-3.1-flash-lite`: Our most cost-efficient model, optimized for high-volume agentic tasks, translation, and simple data processing. - `gemini-3.1-flash-lite-preview`: Our most cost-efficient model, optimized for high-volume agentic tasks, translation, and simple data processing. - `gemini-3-flash-preview`: Our most intelligent model built for speed, combining frontier intelligence with superior search and grounding. - `gemini-3.5-flash`: Our most intelligent model for sustained frontier performance in agentic and coding tasks. - `gemini-3-pro-preview`: Our most intelligent model with SOTA reasoning and multimodal understanding, and powerful agentic and vibe coding capabilities. - `gemini-2.5-flash-native-audio-preview-12-2025`: Our native audio models optimized for higher quality audio outputs with better pacing, voice naturalness, verbosity, and mood. - `gemini-2.5-flash-image`: Our native image generation model, optimized for speed, flexibility, and contextual understanding. Text input and output is priced the same as 2.5 Flash. - `gemini-2.5-flash-lite`: Our smallest and most cost effective model, built for at scale usage. - `gemini-2.5-pro`: Our state-of-the-art multipurpose model, which excels at coding and complex reasoning tasks. - `gemini-3.1-flash-image-preview`: Pro-level visual intelligence with Flash-speed efficiency and reality-grounded generation capabilities. - `gemini-3-pro-image-preview`: State-of-the-art image generation and editing model. - `gemini-2.5-flash-lite-preview-09-2025`: The latest model based on Gemini 2.5 Flash lite optimized for cost-efficiency, high throughput and high quality. - `gemini-2.5-flash-preview-09-2025`: The latest model based on the 2.5 Flash model. 2.5 Flash Preview is best for large scale processing, low-latency, high volume tasks that require thinking, and agentic use cases. - **agent** (`AgentOption`) The name of the `Agent` used for generating the interaction. Possible values: - `deep-research-preview-04-2026`: Gemini Deep Research Agent - `deep-research-pro-preview-12-2025`: Gemini Deep Research Agent - `deep-research-max-preview-04-2026`: Gemini Deep Research Max Agent - `antigravity-preview-05-2026`: Use the Antigravity managed agent to perform multi-step tasks that require reasoning, file operations, and tool use. - **id** (`string`) *(Required)* Required. Output only. A unique identifier for the interaction completion. - **status** (`enum (string)`) *(Required)* Required. Output only. The status of the interaction. Possible values: - `in_progress`: The interaction is in progress. - `requires_action`: The interaction requires action/input from the user. - `completed`: The interaction is completed. - `failed`: The interaction failed. - `cancelled`: The interaction was cancelled. - `incomplete`: The interaction is completed, but contains incomplete results (e.g. hitting max_tokens). - `budget_exceeded`: The interaction was halted because the token budget was exceeded. - **created** (`string`) *(Required)* Required. Output only. The time at which the response was created in ISO 8601 format (YYYY-MM-DDThh:mm:ssZ). - **updated** (`string`) *(Required)* Required. Output only. The time at which the response was last updated in ISO 8601 format (YYYY-MM-DDThh:mm:ssZ). - **system_instruction** (`string`) System instruction for the interaction. - **tools** (`array (Tool)`) A list of tool declarations the model may call during interaction. - **usage** (`Usage`) Output only. Statistics on the interaction request's token usage. - **total_input_tokens** (`integer`) Number of tokens in the prompt (context). - **input_tokens_by_modality** (`array (ModalityTokens)`) A breakdown of input token usage by modality. - **modality** (`ResponseModality`) The modality associated with the token count. Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **tokens** (`integer`) Number of tokens for the modality. - **total_cached_tokens** (`integer`) Number of tokens in the cached part of the prompt (the cached content). - **cached_tokens_by_modality** (`array (ModalityTokens)`) A breakdown of cached token usage by modality. - **modality** (`ResponseModality`) The modality associated with the token count. Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **tokens** (`integer`) Number of tokens for the modality. - **total_output_tokens** (`integer`) Total number of tokens across all the generated responses. - **output_tokens_by_modality** (`array (ModalityTokens)`) A breakdown of output token usage by modality. - **modality** (`ResponseModality`) The modality associated with the token count. Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **tokens** (`integer`) Number of tokens for the modality. - **total_tool_use_tokens** (`integer`) Number of tokens present in tool-use prompt(s). - **tool_use_tokens_by_modality** (`array (ModalityTokens)`) A breakdown of tool-use token usage by modality. - **modality** (`ResponseModality`) The modality associated with the token count. Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **tokens** (`integer`) Number of tokens for the modality. - **total_thought_tokens** (`integer`) Number of tokens of thoughts for thinking models. - **total_tokens** (`integer`) Total token count for the interaction request (prompt + responses + other internal tokens). - **grounding_tool_count** (`array (GroundingToolCount)`) Grounding tool count. - **type** (`enum (string)`) The grounding tool type associated with the count. Possible values: - `google_search`: Grounding with Google Web Search and Image Search, & Web Grounding for Enterprise. - `google_maps`: Grounding with Google Maps. - `retrieval`: Grounding with customer's data, for example, VertexAISearch. - **count** (`integer`) The number of grounding tool counts. - **response_modalities** (`array (ResponseModality)`) The requested modalities of the response (TEXT, IMAGE, AUDIO). Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **previous_interaction_id** (`string`) The ID of the previous interaction, if any. - **environment_id** (`string`) Output only. The environment ID for the interaction. Only populated if environment config is set in the request. - **service_tier** (`ServiceTier`) The service tier for the interaction. Possible values: - `flex`: Flex service tier. - `standard`: Standard service tier. - `priority`: Priority service tier. - **webhook_config** (`WebhookConfig`) Optional. Webhook configuration for receiving notifications when the interaction completes. - **uris** (`array (string)`) Optional. If set, these webhook URIs will be used for webhook events instead of the registered webhooks. - **user_metadata** (`object`) Optional. The user metadata that will be returned on each event emission to the webhooks. - **steps** (`array (Step)`) *(Required)* Required. Output only. The steps that make up the interaction. - **input** (`Content or array (Content) or array (Step) or string`) The input for the interaction. - **response_format** (`ResponseFormat or ResponseFormatList`) Enforces that the generated response is a JSON object that complies with the JSON schema specified in this field. - **environment** (`EnvironmentConfig or string`) The environment configuration for the interaction. Can be an object specifying remote environment sources or a string referencing an existing environment ID. - **generation_config** (`GenerationConfig`) Input only. Configuration parameters for the model interaction. - **temperature** (`number`) Controls the randomness of the output. - **top_p** (`number`) The maximum cumulative probability of tokens to consider when sampling. - **seed** (`integer`) Seed used in decoding for reproducibility. - **stop_sequences** (`array (string)`) A list of character sequences that will stop output interaction. - **thinking_level** (`ThinkingLevel`) The level of thought tokens that the model should generate. Possible values: - `minimal`: Little to no thinking. - `low`: Low thinking level. - `medium`: Medium thinking level. - `high`: High thinking level. - **thinking_summaries** (`ThinkingSummaries`) Whether to include thought summaries in the response. Possible values: - `auto`: Auto thinking summaries. - `none`: No thinking summaries. - **max_output_tokens** (`integer`) The maximum number of tokens to include in the response. - **speech_config** (`array (SpeechConfig)`) Configuration for speech interaction. - **voice** (`string`) The voice of the speaker. - **language** (`string`) The language of the speech. - **speaker** (`string`) The speaker's name, it should match the speaker name given in the prompt. - **tool_choice** (`ToolChoiceConfig or ToolChoiceType`) The tool choice configuration. - **agent_config** (`DeepResearchAgentConfig or DynamicAgentConfig`) Configuration parameters for the agent interaction. **Possible Types:** (Discriminator: `type`) - **DeepResearchAgentConfig**: Configuration for the Deep Research agent. - **type** (`object`) *(Required)* Value: `deep-research` - **thinking_summaries** (`ThinkingSummaries`) Whether to include thought summaries in the response. Possible values: - `auto`: Auto thinking summaries. - `none`: No thinking summaries. - **visualization** (`enum (string)`) Whether to include visualizations in the response. Possible values: - `off`: Do not include visualizations. - `auto`: Automatically include visualizations. - **collaborative_planning** (`boolean`) Enables human-in-the-loop planning for the Deep Research agent. If set to true, the Deep Research agent will provide a research plan in its response. The agent will then proceed only if the user confirms the plan in the next turn. - **enable_bigquery_tool** (`boolean`) Enables bigquery tool for the Deep Research agent. - **DynamicAgentConfig**: Configuration for dynamic agents. - **type** (`object`) *(Required)* Value: `dynamic` **JSON Representation:** ```json { "created": "2025-12-04T15:01:45Z", "id": "v1_ChdXS0l4YWZXTk9xbk0xZThQczhEcmlROBIXV0tJeGFmV05PcW5NMWU4UHM4RHJpUTg", "model": "gemini-3.5-flash", "object": "interaction", "steps": [ { "type": "model_output", "content": [ { "type": "text", "text": "Hello! I'm doing well, functioning as expected. Thank you for asking! How are you doing today?" } ] } ], "status": "completed", "updated": "2025-12-04T15:01:45Z", "usage": { "input_tokens_by_modality": [ { "modality": "text", "tokens": 7 } ], "total_cached_tokens": 0, "total_input_tokens": 7, "total_output_tokens": 23, "total_thought_tokens": 49, "total_tokens": 79, "total_tool_use_tokens": 0 } } ``` **Examples** **Example** ```json { "created": "2025-12-04T15:01:45Z", "id": "v1_ChdXS0l4YWZXTk9xbk0xZThQczhEcmlROBIXV0tJeGFmV05PcW5NMWU4UHM4RHJpUTg", "model": "gemini-3.5-flash", "object": "interaction", "steps": [ { "type": "model_output", "content": [ { "type": "text", "text": "Hello! I'm doing well, functioning as expected. Thank you for asking! How are you doing today?" } ] } ], "status": "completed", "updated": "2025-12-04T15:01:45Z", "usage": { "input_tokens_by_modality": [ { "modality": "text", "tokens": 7 } ], "total_cached_tokens": 0, "total_input_tokens": 7, "total_output_tokens": 23, "total_thought_tokens": 49, "total_tokens": 79, "total_tool_use_tokens": 0 } } ``` ## Data Models ### Content { #Resource:Content } The content of the response. **Polymorphic Types:** (Discriminator: `type`)- **AudioContent** - An audio content block. - **type** (`object`) *(Required)* Value: `audio` - **data** (`string`) The audio content. - **uri** (`string`) The URI of the audio. - **mime_type** (`enum (string)`) The mime type of the audio. Possible values: - `audio/wav`: WAV audio format - `audio/mp3`: MP3 audio format - `audio/aiff`: AIFF audio format - `audio/aac`: AAC audio format - `audio/ogg`: OGG audio format - `audio/flac`: FLAC audio format - `audio/mpeg`: MPEG audio format - `audio/m4a`: M4A audio format - `audio/l16`: L16 audio format - `audio/opus`: OPUS audio format - `audio/alaw`: ALAW audio format - `audio/mulaw`: MULAW audio format - **channels** (`integer`) The number of audio channels. - **sample_rate** (`integer`) The sample rate of the audio. **Examples** **Audio** ```json { "type": "audio", "data": "BASE64_ENCODED_AUDIO", "mime_type": "audio/wav" } ``` - **DocumentContent** - A document content block. - **type** (`object`) *(Required)* Value: `document` - **data** (`string`) The document content. - **uri** (`string`) The URI of the document. - **mime_type** (`enum (string)`) The mime type of the document. Possible values: - `application/pdf`: PDF document format - `text/csv`: CSV document format **Examples** **Document** ```json { "type": "document", "data": "BASE64_ENCODED_DOCUMENT", "mime_type": "application/pdf" } ``` - **ImageContent** - An image content block. - **type** (`object`) *(Required)* Value: `image` - **data** (`string`) The image content. - **uri** (`string`) The URI of the image. - **mime_type** (`enum (string)`) The mime type of the image. Possible values: - `image/png`: PNG image format - `image/jpeg`: JPEG image format - `image/webp`: WebP image format - `image/heic`: HEIC image format - `image/heif`: HEIF image format - `image/gif`: GIF image format - `image/bmp`: BMP image format - `image/tiff`: TIFF image format - **resolution** (`MediaResolution`) The resolution of the media. Possible values: - `low`: Low resolution. - `medium`: Medium resolution. - `high`: High resolution. - `ultra_high`: Ultra high resolution. **Examples** **Image** ```json { "type": "image", "data": "BASE64_ENCODED_IMAGE", "mime_type": "image/png" } ``` - **TextContent** - A text content block. - **type** (`object`) *(Required)* Value: `text` - **text** (`string`) *(Required)* Required. The text content. - **annotations** (`array (Annotation)`) Citation information for model-generated content. **Possible Types:** (Discriminator: `type`) - **FileCitation**: A file citation annotation. - **type** (`object`) *(Required)* Value: `file_citation` - **document_uri** (`string`) The URI of the file. - **file_name** (`string`) The name of the file. - **source** (`string`) Source attributed for a portion of the text. - **custom_metadata** (`object`) User provided metadata about the retrieved context. - **page_number** (`integer`) Page number of the cited document, if applicable. - **media_id** (`string`) Media ID in-case of image citations, if applicable. - **start_index** (`integer`) Start of segment of the response that is attributed to this source. Index indicates the start of the segment, measured in bytes. - **end_index** (`integer`) End of the attributed segment, exclusive. - **PlaceCitation**: A place citation annotation. - **type** (`object`) *(Required)* Value: `place_citation` - **place_id** (`string`) The ID of the place, in `places/{place_id}` format. - **name** (`string`) Title of the place. - **url** (`string`) URI reference of the place. - **review_snippets** (`array (ReviewSnippet)`) Snippets of reviews that are used to generate answers about the features of a given place in Google Maps. - **title** (`string`) Title of the review. - **url** (`string`) A link that corresponds to the user review on Google Maps. - **review_id** (`string`) The ID of the review snippet. - **start_index** (`integer`) Start of segment of the response that is attributed to this source. Index indicates the start of the segment, measured in bytes. - **end_index** (`integer`) End of the attributed segment, exclusive. - **UrlCitation**: A URL citation annotation. - **type** (`object`) *(Required)* Value: `url_citation` - **url** (`string`) The URL. - **title** (`string`) The title of the URL. - **start_index** (`integer`) Start of segment of the response that is attributed to this source. Index indicates the start of the segment, measured in bytes. - **end_index** (`integer`) End of the attributed segment, exclusive. **Examples** **Text** ```json { "type": "text", "text": "Hello, how are you?" } ``` - **VideoContent** - A video content block. - **type** (`object`) *(Required)* Value: `video` - **data** (`string`) The video content. - **uri** (`string`) The URI of the video. - **mime_type** (`enum (string)`) The mime type of the video. Possible values: - `video/mp4`: MP4 video format - `video/mpeg`: MPEG video format - `video/mpg`: MPG video format - `video/mov`: MOV video format - `video/avi`: AVI video format - `video/x-flv`: FLV video format - `video/webm`: WebM video format - `video/wmv`: WMV video format - `video/3gpp`: 3GPP video format - **resolution** (`MediaResolution`) The resolution of the media. Possible values: - `low`: Low resolution. - `medium`: Medium resolution. - `high`: High resolution. - `ultra_high`: Ultra high resolution. **Examples** **Video** ```json { "type": "video", "uri": "https://www.youtube.com/watch?v=9hE5-98ZeCg" } ``` **JSON Representation:** ```json { "type": {}, "data": "string", "uri": "string", "mime_type": "audio/wav", "channels": 0, "sample_rate": 0 } ``` ### Tool { #Resource:Tool } A tool that can be used by the model. **Polymorphic Types:** (Discriminator: `type`)- **CodeExecution** - A tool that can be used by the model to execute code. - **type** (`object`) *(Required)* Value: `code_execution` **Examples** **code_execution** ```sh curl -X POST https://generativelanguage.googleapis.com/v1beta/interactions \ -H "x-goog-api-key: $GEMINI_API_KEY" \ -H "Content-Type: application/json" \ -H "Api-Revision: 2026-05-20" \ -d '{ "model": "gemini-3.5-flash", "tools": [{ "type": "code_execution" }], "input": "Calculate the first 10 Fibonacci numbers" }' ``` **code_execution** ```python from google import genai client = genai.Client() response = client.interactions.create( model="gemini-3.5-flash", tools=[{"type": "code_execution"}], input="Calculate the first 10 Fibonacci numbers" ) print(response.output_text) ``` **code_execution** ```javascript import {GoogleGenAI} from '@google/genai'; const ai = new GoogleGenAI({}); const interaction = await ai.interactions.create({ model: 'gemini-3.5-flash', tools: [{ type: 'code_execution' }], input: 'Calculate the first 10 Fibonacci numbers' }); console.log(interaction.output_text); ``` - **ComputerUse** - A tool that can be used by the model to interact with the computer. - **type** (`object`) *(Required)* Value: `computer_use` - **environment** (`enum (string)`) The environment being operated. Possible values: - `browser`: Operates in a web browser. - **excluded_predefined_functions** (`array (string)`) The list of predefined functions that are excluded from the model call. **Examples** **computer_use** ```sh curl -X POST https://generativelanguage.googleapis.com/v1beta/interactions \ -H "x-goog-api-key: $GEMINI_API_KEY" \ -H "Content-Type: application/json" \ -H "Api-Revision: 2026-05-20" \ -d '{ "model": "gemini-2.5-computer-use-preview-10-2025", "tools": [{ "type": "computer_use" }], "input": "Find a flight to Tokyo" }' ``` **computer_use** ```python from google import genai client = genai.Client() response = client.interactions.create( model="gemini-2.5-computer-use-preview-10-2025", tools=[{"type": "computer_use"}], input="Find a flight to Tokyo" ) print(response.output_text) ``` **computer_use** ```javascript import {GoogleGenAI} from '@google/genai'; const ai = new GoogleGenAI({}); const interaction = await ai.interactions.create({ model: 'gemini-2.5-computer-use-preview-10-2025', tools: [{ type: 'computer_use'}], input: 'Find a flight to Tokyo' }); console.log(interaction.output_text); ``` - **FileSearch** - A tool that can be used by the model to search files. - **type** (`object`) *(Required)* Value: `file_search` - **file_search_store_names** (`array (string)`) The file search store names to search. - **top_k** (`integer`) The number of semantic retrieval chunks to retrieve. - **metadata_filter** (`string`) Metadata filter to apply to the semantic retrieval documents and chunks. **Examples** **file_search** ```sh curl -X POST https://generativelanguage.googleapis.com/v1beta/interactions \ -H "x-goog-api-key: $GEMINI_API_KEY" \ -H "Content-Type: application/json" \ -H "Api-Revision: 2026-05-20" \ -d '{ "model": "gemini-3.5-flash", "tools": [{ "type": "file_search", "file_search_store_names": ["fileSearchStores/m64d1sevsr4y-xfyawui3fxqg"] }], "input": "Who is the author of the book?" }' ``` **file_search** ```python from google import genai client = genai.Client() # Create a file search store so we have a valid one to use. store = client.file_search_stores.create() response = client.interactions.create( model="gemini-3.5-flash", tools=[{ "type": "file_search", "file_search_store_names": [store.name] }], input="What documents are available?" ) print(response.output_text) # [cleanup] client.file_search_stores.delete(name=store.name) # [/cleanup] ``` **file_search** ```javascript import {GoogleGenAI} from '@google/genai'; const ai = new GoogleGenAI({}); // Create a file search store so we have a valid one to use. const store = await ai.fileSearchStores.create({}); if (!store.name) { throw new Error('Store creation failed: Name is undefined'); } const interaction = await ai.interactions.create({ model: 'gemini-3.5-flash', tools: [{ type: 'file_search', file_search_store_names: [store.name] }], input: 'What documents are available?' }); console.log(interaction.output_text); // [cleanup] await ai.fileSearchStores.delete({name: store.name}); // [/cleanup] ``` - **Function** - A tool that can be used by the model. - **type** (`object`) *(Required)* Value: `function` - **name** (`string`) The name of the function. - **description** (`string`) A description of the function. - **parameters** (`object`) The JSON Schema for the function's parameters. **Examples** **function_calling** ```sh curl -X POST https://generativelanguage.googleapis.com/v1beta/interactions \ -H "x-goog-api-key: $GEMINI_API_KEY" \ -H "Content-Type: application/json" \ -H "Api-Revision: 2026-05-20" \ -d '{ "model": "gemini-3.5-flash", "tools": [{ "type": "function", "name": "get_weather", "description": "Get the current weather in a given location", "parameters": { "type": "object", "properties": { "location": { "type": "string", "description": "The city and state, e.g. San Francisco, CA" } }, "required": ["location"] } }], "input": "What is the weather like in Boston, MA?" }' ``` **function_calling** ```python from google import genai client = genai.Client() response = client.interactions.create( model="gemini-3.5-flash", tools=[{ "type": "function", "name": "get_weather", "description": "Get the current weather in a given location", "parameters": { "type": "object", "properties": { "location": { "type": "string", "description": "The city and state, e.g. San Francisco, CA" } }, "required": ["location"] } }], input="What is the weather like in Boston?" ) print(response.steps[-1]) ``` **function_calling** ```javascript import {GoogleGenAI} from '@google/genai'; const ai = new GoogleGenAI({}); const interaction = await ai.interactions.create({ model: 'gemini-3.5-flash', tools: [{ type: 'function', name: 'get_weather', description: 'Get the current weather in a given location', parameters: { type: 'object', properties: { location: { type: 'string', description: 'The city and state, e.g. San Francisco, CA' } }, required: ['location'] } }], input: 'What is the weather like in Boston?' }); console.log(interaction.steps.at(-1)); ``` - **GoogleMaps** - A tool that can be used by the model to call Google Maps. - **type** (`object`) *(Required)* Value: `google_maps` - **enable_widget** (`boolean`) Whether to return a widget context token in the tool call result of the response. - **latitude** (`number`) The latitude of the user's location. - **longitude** (`number`) The longitude of the user's location. **Examples** **google_maps** ```sh curl -X POST https://generativelanguage.googleapis.com/v1beta/interactions \ -H "x-goog-api-key: $GEMINI_API_KEY" \ -H "Content-Type: application/json" \ -H "Api-Revision: 2026-05-20" \ -d '{ "model": "gemini-3.5-flash", "tools": [{ "type": "google_maps", "latitude": 37.7749, "longitude": -122.4194 }], "input": "What is the best food near me?" }' ``` **google_maps** ```python from google import genai client = genai.Client() response = client.interactions.create( model="gemini-3.5-flash", tools=[{ "type": "google_maps", "latitude": 37.7749, "longitude": -122.4194 }], input="What is the best food near me?" ) print(response.output_text) ``` **google_maps** ```javascript import {GoogleGenAI} from '@google/genai'; const ai = new GoogleGenAI({}); const interaction = await ai.interactions.create({ model: 'gemini-3.5-flash', tools: [{ type: 'google_maps', latitude: 37.7749, longitude: -122.4194 }], input: 'What is the best food near me?' }); console.log(interaction.output_text); ``` - **GoogleSearch** - A tool that can be used by the model to search Google. - **type** (`object`) *(Required)* Value: `google_search` - **search_types** (`array (enum (string))`) The types of search grounding to enable. Possible values: - `web_search`: Setting this field enables web search. Only text results are returned. - `image_search`: Setting this field enables image search. Image bytes are returned. - `enterprise_web_search`: Setting this field enables enterprise web search. **Examples** **google_search** ```sh curl -X POST https://generativelanguage.googleapis.com/v1beta/interactions \ -H "x-goog-api-key: $GEMINI_API_KEY" \ -H "Content-Type: application/json" \ -H "Api-Revision: 2026-05-20" \ -d '{ "model": "gemini-3.5-flash", "tools": [{ "type": "google_search" }], "input": "Who is the current president of France?" }' ``` **google_search** ```python from google import genai client = genai.Client() response = client.interactions.create( model="gemini-3.5-flash", tools=[{"type": "google_search"}], input="Who is the current president of France?" ) print(response.output_text) ``` **google_search** ```javascript import {GoogleGenAI} from '@google/genai'; const ai = new GoogleGenAI({}); const interaction = await ai.interactions.create({ model: 'gemini-3.5-flash', tools: [{ type: 'google_search' }], input: 'Who is the current president of France?' }); console.log(interaction.output_text); ``` - **McpServer** - A MCPServer is a server that can be called by the model to perform actions. - **type** (`object`) *(Required)* Value: `mcp_server` - **name** (`string`) The name of the MCPServer. - **url** (`string`) The full URL for the MCPServer endpoint. Example: "https://api.example.com/mcp" - **headers** (`object`) Optional: Fields for authentication headers, timeouts, etc., if needed. - **allowed_tools** (`array (AllowedTools)`) The allowed tools. - **mode** (`ToolChoiceType`) The mode of the tool choice. Possible values: - `auto`: Auto tool choice. - `any`: Any tool choice. - `none`: No tool choice. - `validated`: Validated tool choice. - **tools** (`array (string)`) The names of the allowed tools. **Examples** **mcp_server** ```sh curl -X POST https://generativelanguage.googleapis.com/v1beta/interactions \ -H "x-goog-api-key: $GEMINI_API_KEY" \ -H "Content-Type: application/json" \ -H "Api-Revision: 2026-05-20" \ -d '{ "model": "gemini-3.5-flash", "tools": [{ "type": "mcp_server", "name": "weather_service", "url": "https://gemini-api-demos.uc.r.appspot.com/mcp" }], "input": "Today is 12-05-2025, what is the temperature today in London?" }' ``` **mcp_server** ```python from google import genai client = genai.Client() response = client.interactions.create( model="gemini-3.5-flash", tools=[{ "type": "mcp_server", "name": "weather_service", "url": "https://gemini-api-demos.uc.r.appspot.com/mcp" }], input="Today is 12-05-2025, what is the temperature today in London?" ) print(response.output_text) ``` **mcp_server** ```javascript import {GoogleGenAI} from '@google/genai'; const ai = new GoogleGenAI({}); const interaction = await ai.interactions.create({ model: 'gemini-3.5-flash', tools: [{ type: 'mcp_server', name: 'weather_service', url: 'https://gemini-api-demos.uc.r.appspot.com/mcp' }], input: 'Today is 12-05-2025, what is the temperature today in London?' }); console.log(interaction.output_text); ``` - **Retrieval** - A tool that can be used by the model to retrieve files. - **type** (`object`) *(Required)* Value: `retrieval` - **retrieval_types** (`array (enum (string))`) The types of file retrieval to enable. Possible values: - `rag_store` - `exa_ai_search` - `parallel_ai_search` - **exa_ai_search_config** (`ExaAISearchConfig`) Used to specify configuration for ExaAISearch. - **api_key** (`string`) *(Required)* Required. The API key for ExaAiSearch. - **custom_config** (`object`) Optional. This field can be used to pass any parameter from the Exa.ai Search API. - **parallel_ai_search_config** (`ParallelAISearchConfig`) Used to specify configuration for ParallelAISearch. - **api_key** (`string`) Optional. The API key for ParallelAiSearch. - **custom_config** (`object`) Optional. Custom configs for ParallelAiSearch. - **rag_store_config** (`RagStoreConfig`) Used to specify configuration for RagStore. - **rag_resources** (`array (RagResource)`) Optional. The representation of the rag source. - **rag_corpus** (`string`) Optional. RagCorpora resource name. - **rag_file_ids** (`array (string)`) Optional. rag_file_id. The files should be in the same rag_corpus set in rag_corpus field. - **rag_retrieval_config** (`RagRetrievalConfig`) Optional. The retrieval config for the Rag query. - **top_k** (`integer`) Optional. The number of contexts to retrieve. - **hybrid_search** (`HybridSearch`) Optional. Config for Hybrid Search. - **alpha** (`number`) Optional. Alpha value controls the weight between dense and sparse vector search results. - **filter** (`Filter`) Optional. Config for filters. - **vector_distance_threshold** (`number`) Optional. Only returns contexts with vector distance smaller than the threshold. - **vector_similarity_threshold** (`number`) Optional. Only returns contexts with vector similarity larger than the threshold. - **metadata_filter** (`string`) Optional. String for metadata filtering. - **ranking** (`Ranking`) Optional. Config for ranking and reranking. - **UrlContext** - A tool that can be used by the model to fetch URL context. - **type** (`object`) *(Required)* Value: `url_context` **Examples** **url_context** ```sh curl -X POST https://generativelanguage.googleapis.com/v1beta/interactions \ -H "x-goog-api-key: $GEMINI_API_KEY" \ -H "Content-Type: application/json" \ -H "Api-Revision: 2026-05-20" \ -d '{ "model": "gemini-3.5-flash", "tools": [{ "type": "url_context" }], "input": "Summarize https://www.example.com" }' ``` **url_context** ```python from google import genai client = genai.Client() response = client.interactions.create( model="gemini-3.5-flash", tools=[{"type": "url_context"}], input="Summarize https://www.example.com" ) print(response.output_text) ``` **url_context** ```javascript import {GoogleGenAI} from '@google/genai'; const ai = new GoogleGenAI({}); const interaction = await ai.interactions.create({ model: 'gemini-3.5-flash', tools: [{ type: 'url_context' }], input: 'Summarize https://www.example.com' }); console.log(interaction.output_text); ``` **JSON Representation:** ```json { "type": {} } ``` ### InteractionSseEvent { #Resource:InteractionSseEvent } **Polymorphic Types:** (Discriminator: `event_type`)- **ErrorEvent** - - **event_type** (`object`) *(Required)* Value: `error` - **error** (`Error`) - **code** (`string`) A URI that identifies the error type. - **message** (`string`) A human-readable error message. - **event_id** (`string`) The event_id token to be used to resume the interaction stream, from this event. - **metadata** (`StreamMetadata`) Optional metadata accompanying ANY streamed event. - **usage** (`Usage`) - **total_input_tokens** (`integer`) Number of tokens in the prompt (context). - **input_tokens_by_modality** (`array (ModalityTokens)`) A breakdown of input token usage by modality. - **modality** (`ResponseModality`) The modality associated with the token count. Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **tokens** (`integer`) Number of tokens for the modality. - **total_cached_tokens** (`integer`) Number of tokens in the cached part of the prompt (the cached content). - **cached_tokens_by_modality** (`array (ModalityTokens)`) A breakdown of cached token usage by modality. - **modality** (`ResponseModality`) The modality associated with the token count. Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **tokens** (`integer`) Number of tokens for the modality. - **total_output_tokens** (`integer`) Total number of tokens across all the generated responses. - **output_tokens_by_modality** (`array (ModalityTokens)`) A breakdown of output token usage by modality. - **modality** (`ResponseModality`) The modality associated with the token count. Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **tokens** (`integer`) Number of tokens for the modality. - **total_tool_use_tokens** (`integer`) Number of tokens present in tool-use prompt(s). - **tool_use_tokens_by_modality** (`array (ModalityTokens)`) A breakdown of tool-use token usage by modality. - **modality** (`ResponseModality`) The modality associated with the token count. Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **tokens** (`integer`) Number of tokens for the modality. - **total_thought_tokens** (`integer`) Number of tokens of thoughts for thinking models. - **total_tokens** (`integer`) Total token count for the interaction request (prompt + responses + other internal tokens). - **grounding_tool_count** (`array (GroundingToolCount)`) Grounding tool count. - **type** (`enum (string)`) The grounding tool type associated with the count. Possible values: - `google_search`: Grounding with Google Web Search and Image Search, & Web Grounding for Enterprise. - `google_maps`: Grounding with Google Maps. - `retrieval`: Grounding with customer's data, for example, VertexAISearch. - **count** (`integer`) The number of grounding tool counts. **Examples** **Error Event** ```json { "event_type": "error", "error": { "message": "Failed to get completed interaction: Result not found.", "code": "not_found" } } ``` - **InteractionCompletedEvent** - - **event_type** (`object`) *(Required)* Value: `interaction.completed` - **interaction** (`Interaction`) *(Required)* Required. The completed interaction with empty outputs to reduce the payload size. Use the preceding ContentDelta events for the actual output. - **event_id** (`string`) The event_id token to be used to resume the interaction stream, from this event. - **metadata** (`StreamMetadata`) Optional metadata accompanying ANY streamed event. - **usage** (`Usage`) - **total_input_tokens** (`integer`) Number of tokens in the prompt (context). - **input_tokens_by_modality** (`array (ModalityTokens)`) A breakdown of input token usage by modality. - **modality** (`ResponseModality`) The modality associated with the token count. Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **tokens** (`integer`) Number of tokens for the modality. - **total_cached_tokens** (`integer`) Number of tokens in the cached part of the prompt (the cached content). - **cached_tokens_by_modality** (`array (ModalityTokens)`) A breakdown of cached token usage by modality. - **modality** (`ResponseModality`) The modality associated with the token count. Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **tokens** (`integer`) Number of tokens for the modality. - **total_output_tokens** (`integer`) Total number of tokens across all the generated responses. - **output_tokens_by_modality** (`array (ModalityTokens)`) A breakdown of output token usage by modality. - **modality** (`ResponseModality`) The modality associated with the token count. Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **tokens** (`integer`) Number of tokens for the modality. - **total_tool_use_tokens** (`integer`) Number of tokens present in tool-use prompt(s). - **tool_use_tokens_by_modality** (`array (ModalityTokens)`) A breakdown of tool-use token usage by modality. - **modality** (`ResponseModality`) The modality associated with the token count. Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **tokens** (`integer`) Number of tokens for the modality. - **total_thought_tokens** (`integer`) Number of tokens of thoughts for thinking models. - **total_tokens** (`integer`) Total token count for the interaction request (prompt + responses + other internal tokens). - **grounding_tool_count** (`array (GroundingToolCount)`) Grounding tool count. - **type** (`enum (string)`) The grounding tool type associated with the count. Possible values: - `google_search`: Grounding with Google Web Search and Image Search, & Web Grounding for Enterprise. - `google_maps`: Grounding with Google Maps. - `retrieval`: Grounding with customer's data, for example, VertexAISearch. - **count** (`integer`) The number of grounding tool counts. **Examples** **Interaction Completed** ```json { "event_type": "interaction.completed", "interaction": { "id": "v1_ChdXS0l4YWZXTk9xbk0xZThQczhEcmlROBIXV0tJeGFmV05PcW5NMWU4UHM4RHJpUTg", "model": "gemini-3.5-flash", "status": "completed", "created": "2025-12-04T15:01:45Z", "updated": "2025-12-04T15:01:45Z" }, "event_id": "evt_123" } ``` - **InteractionCreatedEvent** - - **event_type** (`object`) *(Required)* Value: `interaction.created` - **interaction** (`Interaction`) *(Required)* - **event_id** (`string`) The event_id token to be used to resume the interaction stream, from this event. - **metadata** (`StreamMetadata`) Optional metadata accompanying ANY streamed event. - **usage** (`Usage`) - **total_input_tokens** (`integer`) Number of tokens in the prompt (context). - **input_tokens_by_modality** (`array (ModalityTokens)`) A breakdown of input token usage by modality. - **modality** (`ResponseModality`) The modality associated with the token count. Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **tokens** (`integer`) Number of tokens for the modality. - **total_cached_tokens** (`integer`) Number of tokens in the cached part of the prompt (the cached content). - **cached_tokens_by_modality** (`array (ModalityTokens)`) A breakdown of cached token usage by modality. - **modality** (`ResponseModality`) The modality associated with the token count. Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **tokens** (`integer`) Number of tokens for the modality. - **total_output_tokens** (`integer`) Total number of tokens across all the generated responses. - **output_tokens_by_modality** (`array (ModalityTokens)`) A breakdown of output token usage by modality. - **modality** (`ResponseModality`) The modality associated with the token count. Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **tokens** (`integer`) Number of tokens for the modality. - **total_tool_use_tokens** (`integer`) Number of tokens present in tool-use prompt(s). - **tool_use_tokens_by_modality** (`array (ModalityTokens)`) A breakdown of tool-use token usage by modality. - **modality** (`ResponseModality`) The modality associated with the token count. Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **tokens** (`integer`) Number of tokens for the modality. - **total_thought_tokens** (`integer`) Number of tokens of thoughts for thinking models. - **total_tokens** (`integer`) Total token count for the interaction request (prompt + responses + other internal tokens). - **grounding_tool_count** (`array (GroundingToolCount)`) Grounding tool count. - **type** (`enum (string)`) The grounding tool type associated with the count. Possible values: - `google_search`: Grounding with Google Web Search and Image Search, & Web Grounding for Enterprise. - `google_maps`: Grounding with Google Maps. - `retrieval`: Grounding with customer's data, for example, VertexAISearch. - **count** (`integer`) The number of grounding tool counts. **Examples** **Interaction Created** ```json { "event_type": "interaction.created", "interaction": { "id": "v1_ChdXS0l4YWZXTk9xbk0xZThQczhEcmlROBIXV0tJeGFmV05PcW5NMWU4UHM4RHJpUTg", "model": "gemini-3.5-flash", "status": "in_progress", "created": "2025-12-04T15:01:45Z", "updated": "2025-12-04T15:01:45Z" }, "event_id": "evt_123" } ``` - **InteractionStatusUpdate** - - **event_type** (`object`) *(Required)* Value: `interaction.status_update` - **interaction_id** (`string`) *(Required)* - **status** (`enum (string)`) *(Required)* Possible values: - `in_progress`: The interaction is in progress. - `requires_action`: The interaction requires action/input from the user. - `completed`: The interaction is completed. - `failed`: The interaction failed. - `cancelled`: The interaction was cancelled. - `incomplete`: The interaction is completed, but contains incomplete results (e.g. hitting max_tokens). - `budget_exceeded`: The interaction was halted because the token budget was exceeded. - **event_id** (`string`) The event_id token to be used to resume the interaction stream, from this event. - **metadata** (`StreamMetadata`) Optional metadata accompanying ANY streamed event. - **usage** (`Usage`) - **total_input_tokens** (`integer`) Number of tokens in the prompt (context). - **input_tokens_by_modality** (`array (ModalityTokens)`) A breakdown of input token usage by modality. - **modality** (`ResponseModality`) The modality associated with the token count. Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **tokens** (`integer`) Number of tokens for the modality. - **total_cached_tokens** (`integer`) Number of tokens in the cached part of the prompt (the cached content). - **cached_tokens_by_modality** (`array (ModalityTokens)`) A breakdown of cached token usage by modality. - **modality** (`ResponseModality`) The modality associated with the token count. Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **tokens** (`integer`) Number of tokens for the modality. - **total_output_tokens** (`integer`) Total number of tokens across all the generated responses. - **output_tokens_by_modality** (`array (ModalityTokens)`) A breakdown of output token usage by modality. - **modality** (`ResponseModality`) The modality associated with the token count. Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **tokens** (`integer`) Number of tokens for the modality. - **total_tool_use_tokens** (`integer`) Number of tokens present in tool-use prompt(s). - **tool_use_tokens_by_modality** (`array (ModalityTokens)`) A breakdown of tool-use token usage by modality. - **modality** (`ResponseModality`) The modality associated with the token count. Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **tokens** (`integer`) Number of tokens for the modality. - **total_thought_tokens** (`integer`) Number of tokens of thoughts for thinking models. - **total_tokens** (`integer`) Total token count for the interaction request (prompt + responses + other internal tokens). - **grounding_tool_count** (`array (GroundingToolCount)`) Grounding tool count. - **type** (`enum (string)`) The grounding tool type associated with the count. Possible values: - `google_search`: Grounding with Google Web Search and Image Search, & Web Grounding for Enterprise. - `google_maps`: Grounding with Google Maps. - `retrieval`: Grounding with customer's data, for example, VertexAISearch. - **count** (`integer`) The number of grounding tool counts. **Examples** **Interaction Status Update** ```json { "event_type": "interaction.status_update", "interaction_id": "v1_ChdTMjQ0YWJ5TUF1TzcxZThQdjRpcnFRcxIXUzI0NGFieU1BdU83MWU4UHY0aXJxUXM", "status": "in_progress" } ``` - **StepDelta** - - **event_type** (`object`) *(Required)* Value: `step.delta` - **index** (`integer`) *(Required)* - **delta** (`StepDeltaData`) *(Required)* **Possible Types:** (Discriminator: `type`) - **ArgumentsDelta**: - **type** (`object`) *(Required)* Value: `arguments_delta` - **arguments** (`string`) - **AudioDelta**: - **type** (`object`) *(Required)* Value: `audio` - **data** (`string`) - **uri** (`string`) - **mime_type** (`enum (string)`) Possible values: - `audio/wav`: WAV audio format - `audio/mp3`: MP3 audio format - `audio/aiff`: AIFF audio format - `audio/aac`: AAC audio format - `audio/ogg`: OGG audio format - `audio/flac`: FLAC audio format - `audio/mpeg`: MPEG audio format - `audio/m4a`: M4A audio format - `audio/l16`: L16 audio format - `audio/opus`: OPUS audio format - `audio/alaw`: ALAW audio format - `audio/mulaw`: MULAW audio format - **sample_rate** (`integer`) The sample rate of the audio. - **channels** (`integer`) The number of audio channels. - **CodeExecutionCallDelta**: - **type** (`object`) *(Required)* Value: `code_execution_call` - **arguments** (`CodeExecutionCallArguments`) *(Required)* - **language** (`enum (string)`) Programming language of the `code`. Possible values: - `python`: Python >= 3.10, with numpy and simpy available. - **code** (`string`) The code to be executed. - **signature** (`string`) A signature hash for backend validation. - **CodeExecutionResultDelta**: - **type** (`object`) *(Required)* Value: `code_execution_result` - **result** (`string`) *(Required)* - **is_error** (`boolean`) - **signature** (`string`) A signature hash for backend validation. - **DocumentDelta**: - **type** (`object`) *(Required)* Value: `document` - **data** (`string`) - **uri** (`string`) - **mime_type** (`enum (string)`) Possible values: - `application/pdf`: PDF document format - `text/csv`: CSV document format - **FileSearchCallDelta**: - **type** (`object`) *(Required)* Value: `file_search_call` - **signature** (`string`) A signature hash for backend validation. - **FileSearchResultDelta**: - **type** (`object`) *(Required)* Value: `file_search_result` - **result** (`array (FileSearchResult)`) *(Required)* - **signature** (`string`) A signature hash for backend validation. - **FunctionResultDelta**: - **type** (`object`) *(Required)* Value: `function_result` - **name** (`string`) - **is_error** (`boolean`) - **call_id** (`string`) *(Required)* Required. ID to match the ID from the function call block. - **result** (`array (Content) or array (FunctionResultSubcontent) or string`) *(Required)* - **GoogleMapsCallDelta**: - **type** (`object`) *(Required)* Value: `google_maps_call` - **arguments** (`GoogleMapsCallArguments`) The arguments to pass to the Google Maps tool. - **queries** (`array (string)`) The queries to be executed. - **signature** (`string`) A signature hash for backend validation. - **GoogleMapsResultDelta**: - **type** (`object`) *(Required)* Value: `google_maps_result` - **result** (`array (GoogleMapsResult)`) The results of the Google Maps. - **places** (`array (Places)`) The places that were found. - **place_id** (`string`) The ID of the place, in `places/{place_id}` format. - **name** (`string`) Title of the place. - **url** (`string`) URI reference of the place. - **review_snippets** (`array (ReviewSnippet)`) Snippets of reviews that are used to generate answers about the features of a given place in Google Maps. - **title** (`string`) Title of the review. - **url** (`string`) A link that corresponds to the user review on Google Maps. - **review_id** (`string`) The ID of the review snippet. - **widget_context_token** (`string`) Resource name of the Google Maps widget context token. - **signature** (`string`) A signature hash for backend validation. - **GoogleSearchCallDelta**: - **type** (`object`) *(Required)* Value: `google_search_call` - **arguments** (`GoogleSearchCallArguments`) *(Required)* - **queries** (`array (string)`) Web search queries for the following-up web search. - **signature** (`string`) A signature hash for backend validation. - **GoogleSearchResultDelta**: - **type** (`object`) *(Required)* Value: `google_search_result` - **result** (`array (GoogleSearchResult)`) *(Required)* - **search_suggestions** (`string`) Web content snippet that can be embedded in a web page or an app webview. - **is_error** (`boolean`) - **signature** (`string`) A signature hash for backend validation. - **ImageDelta**: - **type** (`object`) *(Required)* Value: `image` - **data** (`string`) - **uri** (`string`) - **mime_type** (`enum (string)`) Possible values: - `image/png`: PNG image format - `image/jpeg`: JPEG image format - `image/webp`: WebP image format - `image/heic`: HEIC image format - `image/heif`: HEIF image format - `image/gif`: GIF image format - `image/bmp`: BMP image format - `image/tiff`: TIFF image format - **resolution** (`MediaResolution`) The resolution of the media. Possible values: - `low`: Low resolution. - `medium`: Medium resolution. - `high`: High resolution. - `ultra_high`: Ultra high resolution. - **McpServerToolCallDelta**: - **type** (`object`) *(Required)* Value: `mcp_server_tool_call` - **name** (`string`) *(Required)* - **server_name** (`string`) *(Required)* - **arguments** (`object`) *(Required)* - **McpServerToolResultDelta**: - **type** (`object`) *(Required)* Value: `mcp_server_tool_result` - **name** (`string`) - **server_name** (`string`) - **result** (`array (Content) or array (FunctionResultSubcontent) or string`) *(Required)* - **TextAnnotationDelta**: - **type** (`object`) *(Required)* Value: `text_annotation_delta` - **annotations** (`array (Annotation)`) Citation information for model-generated content. **Possible Types:** (Discriminator: `type`) - **FileCitation**: A file citation annotation. - **type** (`object`) *(Required)* Value: `file_citation` - **document_uri** (`string`) The URI of the file. - **file_name** (`string`) The name of the file. - **source** (`string`) Source attributed for a portion of the text. - **custom_metadata** (`object`) User provided metadata about the retrieved context. - **page_number** (`integer`) Page number of the cited document, if applicable. - **media_id** (`string`) Media ID in-case of image citations, if applicable. - **start_index** (`integer`) Start of segment of the response that is attributed to this source. Index indicates the start of the segment, measured in bytes. - **end_index** (`integer`) End of the attributed segment, exclusive. - **PlaceCitation**: A place citation annotation. - **type** (`object`) *(Required)* Value: `place_citation` - **place_id** (`string`) The ID of the place, in `places/{place_id}` format. - **name** (`string`) Title of the place. - **url** (`string`) URI reference of the place. - **review_snippets** (`array (ReviewSnippet)`) Snippets of reviews that are used to generate answers about the features of a given place in Google Maps. - **title** (`string`) Title of the review. - **url** (`string`) A link that corresponds to the user review on Google Maps. - **review_id** (`string`) The ID of the review snippet. - **start_index** (`integer`) Start of segment of the response that is attributed to this source. Index indicates the start of the segment, measured in bytes. - **end_index** (`integer`) End of the attributed segment, exclusive. - **UrlCitation**: A URL citation annotation. - **type** (`object`) *(Required)* Value: `url_citation` - **url** (`string`) The URL. - **title** (`string`) The title of the URL. - **start_index** (`integer`) Start of segment of the response that is attributed to this source. Index indicates the start of the segment, measured in bytes. - **end_index** (`integer`) End of the attributed segment, exclusive. - **TextDelta**: - **type** (`object`) *(Required)* Value: `text` - **text** (`string`) *(Required)* - **ThoughtSignatureDelta**: - **type** (`object`) *(Required)* Value: `thought_signature` - **signature** (`string`) Signature to match the backend source to be part of the generation. - **ThoughtSummaryDelta**: - **type** (`object`) *(Required)* Value: `thought_summary` - **content** (`Content`) A new summary item to be added to the thought. - **UrlContextCallDelta**: - **type** (`object`) *(Required)* Value: `url_context_call` - **arguments** (`UrlContextCallArguments`) *(Required)* - **urls** (`array (string)`) The URLs to fetch. - **signature** (`string`) A signature hash for backend validation. - **UrlContextResultDelta**: - **type** (`object`) *(Required)* Value: `url_context_result` - **result** (`array (UrlContextResult)`) *(Required)* - **url** (`string`) The URL that was fetched. - **status** (`enum (string)`) The status of the URL retrieval. Possible values: - `success`: Url retrieval is successful. - `error`: Url retrieval is failed due to error. - `paywall`: Url retrieval is failed because the content is behind paywall. - `unsafe`: Url retrieval is failed because the content is unsafe. - **is_error** (`boolean`) - **signature** (`string`) A signature hash for backend validation. - **VideoDelta**: - **type** (`object`) *(Required)* Value: `video` - **data** (`string`) - **uri** (`string`) - **mime_type** (`enum (string)`) Possible values: - `video/mp4`: MP4 video format - `video/mpeg`: MPEG video format - `video/mpg`: MPG video format - `video/mov`: MOV video format - `video/avi`: AVI video format - `video/x-flv`: FLV video format - `video/webm`: WebM video format - `video/wmv`: WMV video format - `video/3gpp`: 3GPP video format - **resolution** (`MediaResolution`) The resolution of the media. Possible values: - `low`: Low resolution. - `medium`: Medium resolution. - `high`: High resolution. - `ultra_high`: Ultra high resolution. - **event_id** (`string`) The event_id token to be used to resume the interaction stream, from this event. - **metadata** (`StreamMetadata`) Optional metadata accompanying ANY streamed event. - **usage** (`Usage`) - **total_input_tokens** (`integer`) Number of tokens in the prompt (context). - **input_tokens_by_modality** (`array (ModalityTokens)`) A breakdown of input token usage by modality. - **modality** (`ResponseModality`) The modality associated with the token count. Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **tokens** (`integer`) Number of tokens for the modality. - **total_cached_tokens** (`integer`) Number of tokens in the cached part of the prompt (the cached content). - **cached_tokens_by_modality** (`array (ModalityTokens)`) A breakdown of cached token usage by modality. - **modality** (`ResponseModality`) The modality associated with the token count. Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **tokens** (`integer`) Number of tokens for the modality. - **total_output_tokens** (`integer`) Total number of tokens across all the generated responses. - **output_tokens_by_modality** (`array (ModalityTokens)`) A breakdown of output token usage by modality. - **modality** (`ResponseModality`) The modality associated with the token count. Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **tokens** (`integer`) Number of tokens for the modality. - **total_tool_use_tokens** (`integer`) Number of tokens present in tool-use prompt(s). - **tool_use_tokens_by_modality** (`array (ModalityTokens)`) A breakdown of tool-use token usage by modality. - **modality** (`ResponseModality`) The modality associated with the token count. Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **tokens** (`integer`) Number of tokens for the modality. - **total_thought_tokens** (`integer`) Number of tokens of thoughts for thinking models. - **total_tokens** (`integer`) Total token count for the interaction request (prompt + responses + other internal tokens). - **grounding_tool_count** (`array (GroundingToolCount)`) Grounding tool count. - **type** (`enum (string)`) The grounding tool type associated with the count. Possible values: - `google_search`: Grounding with Google Web Search and Image Search, & Web Grounding for Enterprise. - `google_maps`: Grounding with Google Maps. - `retrieval`: Grounding with customer's data, for example, VertexAISearch. - **count** (`integer`) The number of grounding tool counts. **Examples** **Step Delta** ```json { "event_type": "step.delta", "index": 0, "delta": { "type": "text", "text": "Hello" } } ``` - **StepStart** - - **event_type** (`object`) *(Required)* Value: `step.start` - **index** (`integer`) *(Required)* - **step** (`Step`) *(Required)* - **event_id** (`string`) The event_id token to be used to resume the interaction stream, from this event. - **metadata** (`StreamMetadata`) Optional metadata accompanying ANY streamed event. - **usage** (`Usage`) - **total_input_tokens** (`integer`) Number of tokens in the prompt (context). - **input_tokens_by_modality** (`array (ModalityTokens)`) A breakdown of input token usage by modality. - **modality** (`ResponseModality`) The modality associated with the token count. Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **tokens** (`integer`) Number of tokens for the modality. - **total_cached_tokens** (`integer`) Number of tokens in the cached part of the prompt (the cached content). - **cached_tokens_by_modality** (`array (ModalityTokens)`) A breakdown of cached token usage by modality. - **modality** (`ResponseModality`) The modality associated with the token count. Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **tokens** (`integer`) Number of tokens for the modality. - **total_output_tokens** (`integer`) Total number of tokens across all the generated responses. - **output_tokens_by_modality** (`array (ModalityTokens)`) A breakdown of output token usage by modality. - **modality** (`ResponseModality`) The modality associated with the token count. Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **tokens** (`integer`) Number of tokens for the modality. - **total_tool_use_tokens** (`integer`) Number of tokens present in tool-use prompt(s). - **tool_use_tokens_by_modality** (`array (ModalityTokens)`) A breakdown of tool-use token usage by modality. - **modality** (`ResponseModality`) The modality associated with the token count. Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **tokens** (`integer`) Number of tokens for the modality. - **total_thought_tokens** (`integer`) Number of tokens of thoughts for thinking models. - **total_tokens** (`integer`) Total token count for the interaction request (prompt + responses + other internal tokens). - **grounding_tool_count** (`array (GroundingToolCount)`) Grounding tool count. - **type** (`enum (string)`) The grounding tool type associated with the count. Possible values: - `google_search`: Grounding with Google Web Search and Image Search, & Web Grounding for Enterprise. - `google_maps`: Grounding with Google Maps. - `retrieval`: Grounding with customer's data, for example, VertexAISearch. - **count** (`integer`) The number of grounding tool counts. **Examples** **Step Start** ```json { "event_type": "step.start", "index": 0, "step": { "type": "model_output" } } ``` - **StepStop** - - **event_type** (`object`) *(Required)* Value: `step.stop` - **index** (`integer`) *(Required)* - **event_id** (`string`) The event_id token to be used to resume the interaction stream, from this event. - **metadata** (`StreamMetadata`) Optional metadata accompanying ANY streamed event. - **usage** (`Usage`) - **total_input_tokens** (`integer`) Number of tokens in the prompt (context). - **input_tokens_by_modality** (`array (ModalityTokens)`) A breakdown of input token usage by modality. - **modality** (`ResponseModality`) The modality associated with the token count. Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **tokens** (`integer`) Number of tokens for the modality. - **total_cached_tokens** (`integer`) Number of tokens in the cached part of the prompt (the cached content). - **cached_tokens_by_modality** (`array (ModalityTokens)`) A breakdown of cached token usage by modality. - **modality** (`ResponseModality`) The modality associated with the token count. Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **tokens** (`integer`) Number of tokens for the modality. - **total_output_tokens** (`integer`) Total number of tokens across all the generated responses. - **output_tokens_by_modality** (`array (ModalityTokens)`) A breakdown of output token usage by modality. - **modality** (`ResponseModality`) The modality associated with the token count. Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **tokens** (`integer`) Number of tokens for the modality. - **total_tool_use_tokens** (`integer`) Number of tokens present in tool-use prompt(s). - **tool_use_tokens_by_modality** (`array (ModalityTokens)`) A breakdown of tool-use token usage by modality. - **modality** (`ResponseModality`) The modality associated with the token count. Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **tokens** (`integer`) Number of tokens for the modality. - **total_thought_tokens** (`integer`) Number of tokens of thoughts for thinking models. - **total_tokens** (`integer`) Total token count for the interaction request (prompt + responses + other internal tokens). - **grounding_tool_count** (`array (GroundingToolCount)`) Grounding tool count. - **type** (`enum (string)`) The grounding tool type associated with the count. Possible values: - `google_search`: Grounding with Google Web Search and Image Search, & Web Grounding for Enterprise. - `google_maps`: Grounding with Google Maps. - `retrieval`: Grounding with customer's data, for example, VertexAISearch. - **count** (`integer`) The number of grounding tool counts. **Examples** **Step Stop** ```json { "event_type": "step.stop", "index": 0 } ``` **JSON Representation:** ```json { "event_type": {}, "error": { "code": "string", "message": "string" }, "event_id": "string", "metadata": { "usage": { "total_input_tokens": 0, "input_tokens_by_modality": [ { "modality": "text", "tokens": 0 } ], "total_cached_tokens": 0, "cached_tokens_by_modality": [ { "modality": "text", "tokens": 0 } ], "total_output_tokens": 0, "output_tokens_by_modality": [ { "modality": "text", "tokens": 0 } ], "total_tool_use_tokens": 0, "tool_use_tokens_by_modality": [ { "modality": "text", "tokens": 0 } ], "total_thought_tokens": 0, "total_tokens": 0, "grounding_tool_count": [ { "type": "google_search", "count": 0 } ] } } } ``` ### ResponseFormat { #Resource:ResponseFormat } **Polymorphic Types:** - **AudioResponseFormat** - Configuration for audio output format. - **type** (`object`) *(Required)* Value: `audio` - **mime_type** (`enum (string)`) The MIME type of the audio output. Possible values: - `audio/mp3`: MP3 audio format. - `audio/ogg_opus`: OGG Opus audio format. - `audio/l16`: Raw PCM (L16) audio format. - `audio/wav`: WAV audio format. - `audio/alaw`: A-law audio format. - `audio/mulaw`: Mu-law audio format. - **delivery** (`enum (string)`) The delivery mode for the audio output. Possible values: - `inline`: Audio data is returned inline in the response. - `uri`: Audio data is returned as a URI. - **sample_rate** (`integer`) Sample rate in Hz. - **bit_rate** (`integer`) Bit rate in bits per second (bps). Only applicable for compressed formats (MP3, Opus). **Examples** **Audio Output** ```json { "type": "audio", "sample_rate": 24000 } ``` - **ImageResponseFormat** - Configuration for image output format. - **type** (`object`) *(Required)* Value: `image` - **mime_type** (`enum (string)`) The MIME type of the image output. Possible values: - `image/jpeg`: JPEG image format. - **delivery** (`enum (string)`) The delivery mode for the image output. Possible values: - `inline`: Image data is returned inline in the response. - `uri`: Image data is returned as a URI. - **aspect_ratio** (`enum (string)`) The aspect ratio for the image output. Possible values: - `1:1`: 1:1 aspect ratio. - `2:3`: 2:3 aspect ratio. - `3:2`: 3:2 aspect ratio. - `3:4`: 3:4 aspect ratio. - `4:3`: 4:3 aspect ratio. - `4:5`: 4:5 aspect ratio. - `5:4`: 5:4 aspect ratio. - `9:16`: 9:16 aspect ratio. - `16:9`: 16:9 aspect ratio. - `21:9`: 21:9 aspect ratio. - `1:8`: 1:8 aspect ratio. - `8:1`: 8:1 aspect ratio. - `1:4`: 1:4 aspect ratio. - `4:1`: 4:1 aspect ratio. - **image_size** (`enum (string)`) The size of the image output. Possible values: - `512`: 512px image size. - `1K`: 1K image size. - `2K`: 2K image size. - `4K`: 4K image size. **Examples** **Image Output** ```json { "type": "image", "mime_type": "image/jpeg", "aspect_ratio": "16:9", "image_size": "1K" } ``` - **TextResponseFormat** - Configuration for text output format. - **type** (`object`) *(Required)* Value: `text` - **mime_type** (`enum (string)`) The MIME type of the text output. Possible values: - `application/json`: JSON output format. - `text/plain`: Plain text output format. - **schema** (`object`) The JSON schema that the output should conform to. Only applicable when mime_type is application/json. **Examples** **Text Output (JSON Schema)** ```json { "type": "text", "mime_type": "application/json", "schema": { "type": "object", "properties": { "recipe_name": { "type": "string" }, "ingredients": { "type": "array", "items": { "type": "string" } } }, "required": [ "ingredients", "recipe_name" ] } } ``` **JSON Representation:** ```json { "type": {}, "mime_type": "audio/mp3", "delivery": "inline", "sample_rate": 0, "bit_rate": 0 } ``` ### ResponseFormatList { #Resource:ResponseFormatList } **Type:** Array of ResponseFormat **JSON Representation:** ```json [ { "type": "text", "mime_type": "application/json" } ] ``` **Examples** **Example** ```json [ { "type": "text", "mime_type": "application/json" } ] ``` ### Step { #Resource:Step } A step in the interaction. **Polymorphic Types:** (Discriminator: `type`)- **CodeExecutionCallStep** - Code execution call step. - **type** (`object`) *(Required)* Value: `code_execution_call` - **arguments** (`CodeExecutionCallStepArguments`) *(Required)* Required. The arguments to pass to the code execution. - **language** (`enum (string)`) Programming language of the `code`. Possible values: - `python`: Python >= 3.10, with numpy and simpy available. - **code** (`string`) The code to be executed. - **id** (`string`) *(Required)* Required. A unique ID for this specific tool call. - **signature** (`string`) A signature hash for backend validation. **Examples** **CodeExecutionCallStep** ```json { "type": "code_execution_call", "id": "code_call_71021", "arguments": { "code": "print(sum(range(1, 11)))" } } ``` - **CodeExecutionResultStep** - Code execution result step. - **type** (`object`) *(Required)* Value: `code_execution_result` - **result** (`string`) *(Required)* Required. The output of the code execution. - **is_error** (`boolean`) Whether the code execution resulted in an error. - **call_id** (`string`) *(Required)* Required. ID to match the ID from the function call block. - **signature** (`string`) A signature hash for backend validation. **Examples** **CodeExecutionResultStep** ```json { "type": "code_execution_result", "call_id": "code_call_71021", "result": "55\n" } ``` - **FileSearchCallStep** - File Search call step. - **type** (`object`) *(Required)* Value: `file_search_call` - **id** (`string`) *(Required)* Required. A unique ID for this specific tool call. - **signature** (`string`) A signature hash for backend validation. **Examples** **FileSearchCallStep** ```json { "type": "file_search_call", "id": "file_call_88192" } ``` - **FileSearchResultStep** - File Search result step. - **type** (`object`) *(Required)* Value: `file_search_result` - **call_id** (`string`) *(Required)* Required. ID to match the ID from the function call block. - **signature** (`string`) A signature hash for backend validation. **Examples** **FileSearchResultStep** ```json { "type": "file_search_result", "call_id": "file_call_88192" } ``` - **FunctionCallStep** - A function tool call step. - **type** (`object`) *(Required)* Value: `function_call` - **name** (`string`) *(Required)* Required. The name of the tool to call. - **arguments** (`object`) *(Required)* Required. The arguments to pass to the function. - **id** (`string`) *(Required)* Required. A unique ID for this specific tool call. **Examples** **FunctionCallStep** ```json { "type": "function_call", "id": "call_98231", "name": "get_weather", "arguments": { "location": "Boston, MA" } } ``` - **FunctionResultStep** - Result of a function tool call. - **type** (`object`) *(Required)* Value: `function_result` - **name** (`string`) The name of the tool that was called. - **is_error** (`boolean`) Whether the tool call resulted in an error. - **call_id** (`string`) *(Required)* Required. ID to match the ID from the function call block. - **result** (`array (Content) or array (FunctionResultSubcontent) or string`) *(Required)* The result of the tool call. **Examples** **FunctionResultStep** ```json { "type": "function_result", "call_id": "call_98231", "name": "get_weather", "result": [ { "type": "text", "text": "{\"weather\":\"sunny\"}" } ] } ``` - **GoogleMapsCallStep** - Google Maps call step. - **type** (`object`) *(Required)* Value: `google_maps_call` - **arguments** (`GoogleMapsCallStepArguments`) The arguments to pass to the Google Maps tool. - **queries** (`array (string)`) The queries to be executed. - **id** (`string`) *(Required)* Required. A unique ID for this specific tool call. - **signature** (`string`) A signature hash for backend validation. **Examples** **GoogleMapsCallStep** ```json { "type": "google_maps_call", "id": "maps_call_39201", "arguments": { "latitude": 37.7749, "longitude": -122.4194 } } ``` - **GoogleMapsResultStep** - Google Maps result step. - **type** (`object`) *(Required)* Value: `google_maps_result` - **result** (`array (GoogleMapsResultItem)`) *(Required)* - **places** (`array (GoogleMapsResultPlaces)`) - **place_id** (`string`) - **name** (`string`) - **url** (`string`) - **review_snippets** (`array (ReviewSnippet)`) - **title** (`string`) Title of the review. - **url** (`string`) A link that corresponds to the user review on Google Maps. - **review_id** (`string`) The ID of the review snippet. - **widget_context_token** (`string`) - **call_id** (`string`) *(Required)* Required. ID to match the ID from the function call block. - **signature** (`string`) A signature hash for backend validation. **Examples** **GoogleMapsResultStep** ```json { "type": "google_maps_result", "call_id": "maps_call_39201", "result": [ { "place_id": "ChIJIQBpAG2ahYAR9R7bNdTLg8M", "name": "Golden Gate Park", "rating": 4.8 } ] } ``` - **GoogleSearchCallStep** - Google Search call step. - **type** (`object`) *(Required)* Value: `google_search_call` - **arguments** (`GoogleSearchCallStepArguments`) *(Required)* Required. The arguments to pass to Google Search. - **queries** (`array (string)`) Web search queries for the following-up web search. - **search_type** (`enum (string)`) The type of search grounding enabled. Possible values: - `web_search`: Setting this field enables web search. Only text results are returned. - `image_search`: Setting this field enables image search. Image bytes are returned. - `enterprise_web_search`: Setting this field enables enterprise web search. - **id** (`string`) *(Required)* Required. A unique ID for this specific tool call. - **signature** (`string`) A signature hash for backend validation. **Examples** **GoogleSearchCallStep** ```json { "type": "google_search_call", "id": "search_call_19201", "arguments": { "query": "Who won the men's 100m in Paris 2024?" } } ``` - **GoogleSearchResultStep** - Google Search result step. - **type** (`object`) *(Required)* Value: `google_search_result` - **result** (`array (GoogleSearchResultItem)`) *(Required)* Required. The results of the Google Search. - **search_suggestions** (`string`) Web content snippet that can be embedded in a web page or an app webview. - **is_error** (`boolean`) Whether the Google Search resulted in an error. - **call_id** (`string`) *(Required)* Required. ID to match the ID from the function call block. - **signature** (`string`) A signature hash for backend validation. **Examples** **GoogleSearchResultStep** ```json { "type": "google_search_result", "call_id": "search_call_19201", "result": [ { "title": "Paris 2024 Olympics: Noah Lyles wins men's 100m gold", "url": "https://olympics.com/en/news/paris-2024-noah-lyles-wins-mens-100m-gold", "snippet": "American Noah Lyles won the Olympic men's 100m gold medal in a photo finish." } ] } ``` - **McpServerToolCallStep** - MCPServer tool call step. - **type** (`object`) *(Required)* Value: `mcp_server_tool_call` - **name** (`string`) *(Required)* Required. The name of the tool which was called. - **server_name** (`string`) *(Required)* Required. The name of the used MCP server. - **arguments** (`object`) *(Required)* Required. The JSON object of arguments for the function. - **id** (`string`) *(Required)* Required. A unique ID for this specific tool call. **Examples** **McpServerToolCallStep** ```json { "type": "mcp_server_tool_call", "id": "mcp_call_29012", "name": "calculate_tax", "server_name": "financial_mcp_server", "arguments": { "income": 120000, "state": "CA" } } ``` - **McpServerToolResultStep** - MCPServer tool result step. - **type** (`object`) *(Required)* Value: `mcp_server_tool_result` - **name** (`string`) Name of the tool which is called for this specific tool call. - **server_name** (`string`) The name of the used MCP server. - **call_id** (`string`) *(Required)* Required. ID to match the ID from the function call block. - **result** (`array (Content) or array (FunctionResultSubcontent) or object`) *(Required)* The output from the MCP server call. Can be simple text or rich content. **Examples** **McpServerToolResultStep** ```json { "type": "mcp_server_tool_result", "call_id": "mcp_call_29012", "result": { "tax_due": 32400 } } ``` - **ModelOutputStep** - Output generated by the model. - **type** (`object`) *(Required)* Value: `model_output` - **content** (`array (Content)`) **Examples** **ModelOutputStep** ```json { "type": "model_output", "content": [ { "type": "text", "text": "The capital of France is Paris." } ] } ``` - **ThoughtStep** - A thought step. - **type** (`object`) *(Required)* Value: `thought` - **signature** (`string`) A signature hash for backend validation. - **summary** (`array (ThoughtSummaryContent)`) A summary of the thought. **Possible Types:** (Discriminator: `type`) - **ImageContent**: An image content block. - **type** (`object`) *(Required)* Value: `image` - **data** (`string`) The image content. - **uri** (`string`) The URI of the image. - **mime_type** (`enum (string)`) The mime type of the image. Possible values: - `image/png`: PNG image format - `image/jpeg`: JPEG image format - `image/webp`: WebP image format - `image/heic`: HEIC image format - `image/heif`: HEIF image format - `image/gif`: GIF image format - `image/bmp`: BMP image format - `image/tiff`: TIFF image format - **resolution** (`MediaResolution`) The resolution of the media. Possible values: - `low`: Low resolution. - `medium`: Medium resolution. - `high`: High resolution. - `ultra_high`: Ultra high resolution. - **TextContent**: A text content block. - **type** (`object`) *(Required)* Value: `text` - **text** (`string`) *(Required)* Required. The text content. - **annotations** (`array (Annotation)`) Citation information for model-generated content. **Possible Types:** (Discriminator: `type`) - **FileCitation**: A file citation annotation. - **type** (`object`) *(Required)* Value: `file_citation` - **document_uri** (`string`) The URI of the file. - **file_name** (`string`) The name of the file. - **source** (`string`) Source attributed for a portion of the text. - **custom_metadata** (`object`) User provided metadata about the retrieved context. - **page_number** (`integer`) Page number of the cited document, if applicable. - **media_id** (`string`) Media ID in-case of image citations, if applicable. - **start_index** (`integer`) Start of segment of the response that is attributed to this source. Index indicates the start of the segment, measured in bytes. - **end_index** (`integer`) End of the attributed segment, exclusive. - **PlaceCitation**: A place citation annotation. - **type** (`object`) *(Required)* Value: `place_citation` - **place_id** (`string`) The ID of the place, in `places/{place_id}` format. - **name** (`string`) Title of the place. - **url** (`string`) URI reference of the place. - **review_snippets** (`array (ReviewSnippet)`) Snippets of reviews that are used to generate answers about the features of a given place in Google Maps. - **title** (`string`) Title of the review. - **url** (`string`) A link that corresponds to the user review on Google Maps. - **review_id** (`string`) The ID of the review snippet. - **start_index** (`integer`) Start of segment of the response that is attributed to this source. Index indicates the start of the segment, measured in bytes. - **end_index** (`integer`) End of the attributed segment, exclusive. - **UrlCitation**: A URL citation annotation. - **type** (`object`) *(Required)* Value: `url_citation` - **url** (`string`) The URL. - **title** (`string`) The title of the URL. - **start_index** (`integer`) Start of segment of the response that is attributed to this source. Index indicates the start of the segment, measured in bytes. - **end_index** (`integer`) End of the attributed segment, exclusive. **Examples** **ThoughtStep** ```json { "type": "thought", "signature": "thought_sig_abcd1234", "summary": [ { "type": "text", "text": "The model is searching Google for the capital of France." } ] } ``` - **UrlContextCallStep** - URL context call step. - **type** (`object`) *(Required)* Value: `url_context_call` - **arguments** (`UrlContextCallStepArguments`) *(Required)* Required. The arguments to pass to the URL context. - **urls** (`array (string)`) The URLs to fetch. - **id** (`string`) *(Required)* Required. A unique ID for this specific tool call. - **signature** (`string`) A signature hash for backend validation. **Examples** **UrlContextCallStep** ```json { "type": "url_context_call", "id": "url_call_10219", "arguments": { "urls": [ "https://www.example.com" ] } } ``` - **UrlContextResultStep** - URL context result step. - **type** (`object`) *(Required)* Value: `url_context_result` - **result** (`array (UrlContextResultItem)`) *(Required)* Required. The results of the URL context. - **url** (`string`) The URL that was fetched. - **status** (`enum (string)`) The status of the URL retrieval. Possible values: - `success` - `error` - `paywall` - `unsafe` - **is_error** (`boolean`) Whether the URL context resulted in an error. - **call_id** (`string`) *(Required)* Required. ID to match the ID from the function call block. - **signature** (`string`) A signature hash for backend validation. **Examples** **UrlContextResultStep** ```json { "type": "url_context_result", "call_id": "url_call_10219", "result": [ { "url": "https://www.example.com", "title": "Example Domain", "snippet": "This domain is for use in illustrative examples in documents." } ] } ``` - **UserInputStep** - Input provided by the user. - **content** (`array (Content)`) - **type** (`object`) *(Required)* Value: `user_input` **Examples** **UserInputStep** ```json { "type": "user_input", "content": [ { "type": "text", "text": "What is the capital of France?" } ] } ``` **JSON Representation:** ```json { "type": {}, "arguments": { "language": "python", "code": "string" }, "id": "string", "signature": "string" } ``` ### EnvironmentConfig { #Resource:EnvironmentConfig } Configuration for a custom environment. **Properties:** - **type** (`object`) *(Required)* Value: `remote` - **sources** (`array (Source)`) - **type** (`enum (string)`) Possible values: - `gcs`: A GCS bucket. - `inline`: Inline content. - `repository`: A generic repository. The protocol prefix in the source URL identifies the provider (e.g., github://, gcs://). - `skill_registry`: A skill resource from the Skill Registry Service. Skill: projects/{project}/locations/{location}/skills/{skill} SkillRevision: projects/{project}/locations/{location}/skills/{skill}/revisions/{revision} Support mounting all skills under a project: projects/{project}/locations/{location}/skills. - **source** (`string`) The source of the environment. For GCS, this is the GCS path. For GitHub, this is the GitHub path. - **target** (`string`) Where the source should appear in the environment. - **content** (`string`) The inline content if `type` is `INLINE`. - **encoding** (`string`) Optional encoding for inline content (e.g. `base64`). - **network** (`EnvironmentNetworkEgressAllowlist or enum (string)`) Network configuration for the environment. **JSON Representation:** ```json { "type": {}, "sources": [ { "type": "gcs", "source": "string", "target": "string", "content": "string", "encoding": "string" } ], "network": { "allowlist": [ { "domain": "pypi.org" }, { "domain": "*.github.com" } ] } } ``` **Examples** **Inline Sources** ```json { "type": "remote", "sources": [ { "type": "inline", "target": ".agents/AGENTS.md", "content": "You are a data analyst. Always include visualizations and export results as PDF." }, { "type": "inline", "target": ".agents/skills/slide-maker/SKILL.md", "content": "---\nname: slide-maker\ndescription: Create HTML slide decks\n---\n# Slide Maker\n\nWhen asked to create a presentation:\n1. Analyze the input data\n2. Create an HTML slide deck with reveal.js\n3. Save to /workspace/output/slides.html" } ] } ``` **External Sources** ```json { "type": "remote", "sources": [ { "type": "repository", "source": "https://github.com/my-org/my-skills.git", "target": ".agents/skills" }, { "type": "gcs", "source": "gs://my-bucket/my-folder", "target": "/workspace/data" } ] } ``` **Network Allowlist** ```json { "type": "remote", "network": { "allowlist": [ { "domain": "pypi.org" }, { "domain": "*.github.com" } ] } } ``` **Proxy Credentials** ```json { "type": "remote", "network": { "allowlist": [ { "domain": "api.github.com", "transform": { "Authorization": "Bearer YOUR_GITHUB_TOKEN" } } ] } } ``` ### EnvironmentNetworkEgressAllowlist { #Resource:EnvironmentNetworkEgressAllowlist } Outbound networking configuration for the sandbox. Accepts an object with an 'allowlist' array to restrict traffic, or the string 'disabled' to turn off all network access. Omit entirely to allow all outbound traffic with no header injection. **Polymorphic Types:** - **Allowlist** - Outbound networking configuration for the sandbox. When specified, restricts which external domains the sandbox can reach. Omit entirely to allow all outbound traffic with no header injection. - **allowlist** (`array (AllowlistEntry)`) List of allowed outbound domains. Only requests to listed domains are permitted. Use [{'domain': '*'}] to allow all domains while still injecting headers on specific ones. - **domain** (`string`) *(Required)* Domain to allow outbound requests to. Supports wildcards (e.g. '*.googleapis.com'). Use '*' to allow all domains. - **transform** (`array (object)`) Headers to inject on all outbound requests matching this domain. Each entry is a flat {header_name: header_value} object. The egress proxy injects these automatically. - **Disabled** - Turns all network off. **JSON Representation:** ```json { "allowlist": [ { "domain": "pypi.org" }, { "domain": "*.github.com" } ] } ``` ### FunctionResultSubcontent { #Resource:FunctionResultSubcontent } **Polymorphic Types:** (Discriminator: `type`)- **ImageContent** - An image content block. - **type** (`object`) *(Required)* Value: `image` - **data** (`string`) The image content. - **uri** (`string`) The URI of the image. - **mime_type** (`enum (string)`) The mime type of the image. Possible values: - `image/png`: PNG image format - `image/jpeg`: JPEG image format - `image/webp`: WebP image format - `image/heic`: HEIC image format - `image/heif`: HEIF image format - `image/gif`: GIF image format - `image/bmp`: BMP image format - `image/tiff`: TIFF image format - **resolution** (`MediaResolution`) The resolution of the media. Possible values: - `low`: Low resolution. - `medium`: Medium resolution. - `high`: High resolution. - `ultra_high`: Ultra high resolution. **Examples** **Image** ```json { "type": "image", "data": "BASE64_ENCODED_IMAGE", "mime_type": "image/png" } ``` - **TextContent** - A text content block. - **type** (`object`) *(Required)* Value: `text` - **text** (`string`) *(Required)* Required. The text content. - **annotations** (`array (Annotation)`) Citation information for model-generated content. **Possible Types:** (Discriminator: `type`) - **FileCitation**: A file citation annotation. - **type** (`object`) *(Required)* Value: `file_citation` - **document_uri** (`string`) The URI of the file. - **file_name** (`string`) The name of the file. - **source** (`string`) Source attributed for a portion of the text. - **custom_metadata** (`object`) User provided metadata about the retrieved context. - **page_number** (`integer`) Page number of the cited document, if applicable. - **media_id** (`string`) Media ID in-case of image citations, if applicable. - **start_index** (`integer`) Start of segment of the response that is attributed to this source. Index indicates the start of the segment, measured in bytes. - **end_index** (`integer`) End of the attributed segment, exclusive. - **PlaceCitation**: A place citation annotation. - **type** (`object`) *(Required)* Value: `place_citation` - **place_id** (`string`) The ID of the place, in `places/{place_id}` format. - **name** (`string`) Title of the place. - **url** (`string`) URI reference of the place. - **review_snippets** (`array (ReviewSnippet)`) Snippets of reviews that are used to generate answers about the features of a given place in Google Maps. - **title** (`string`) Title of the review. - **url** (`string`) A link that corresponds to the user review on Google Maps. - **review_id** (`string`) The ID of the review snippet. - **start_index** (`integer`) Start of segment of the response that is attributed to this source. Index indicates the start of the segment, measured in bytes. - **end_index** (`integer`) End of the attributed segment, exclusive. - **UrlCitation**: A URL citation annotation. - **type** (`object`) *(Required)* Value: `url_citation` - **url** (`string`) The URL. - **title** (`string`) The title of the URL. - **start_index** (`integer`) Start of segment of the response that is attributed to this source. Index indicates the start of the segment, measured in bytes. - **end_index** (`integer`) End of the attributed segment, exclusive. **Examples** **Text** ```json { "type": "text", "text": "Hello, how are you?" } ``` **JSON Representation:** ```json { "type": {}, "data": "string", "uri": "string", "mime_type": "image/png", "resolution": "low" } ``` ### ToolChoiceConfig { #Resource:ToolChoiceConfig } The tool choice configuration containing allowed tools. **Properties:** - **allowed_tools** (`AllowedTools`) The allowed tools. - **mode** (`ToolChoiceType`) The mode of the tool choice. Possible values: - `auto`: Auto tool choice. - `any`: Any tool choice. - `none`: No tool choice. - `validated`: Validated tool choice. - **tools** (`array (string)`) The names of the allowed tools. **JSON Representation:** ```json { "allowed_tools": { "mode": "auto", "tools": [ "string" ] } } ``` ### ToolChoiceType { #Resource:ToolChoiceType } **JSON Representation:** ```json "auto" ```