Gemini Robotics ER 模型可以撰寫及執行 Python 程式碼來處理圖片，並在回答問題前套用邏輯。本頁面涵蓋程式碼執行範例，包括：使用縮放和裁剪功能進行物件偵測、讀取儀表、測量液體、讀取電路板，以及圖像註解。
如要根據自己的用途調整這些範例，請將提示文字和上傳的圖片檔案換成自己的內容。您也可以在提示中調整要求的 JSON 結構定義，配合應用程式需要的輸出結構，或新增 system_instruction 強制輸出格式和精確度。
如需完整的可執行程式碼，請參閱「機器人食譜」。
思考程度
您可以控制模型的思考層級，以延遲換取準確度。物件偵測等空間工作在低思考層級下表現良好。對於計數或重量估算等複雜工作，較高的思考層次有助於提升準確度。
以下範例會將複雜的計數工作思考層級設為 high：
Python
from google import genai
client = genai.Client()
uploaded_file = client.files.upload(file="scene.jpeg")
interaction = client.interactions.create(
model="gemini-robotics-er-2-preview",
input=[
{
"type": "image",
"uri": uploaded_file.uri,
"mime_type": uploaded_file.mime_type
},
{"type": "text", "text": "Identify and count all objects on the table."}
],
generation_config={
"thinking_level": "high" # Use "minimal" or "low" for faster spatial tasks
}
)
print(interaction.output_text)
詳情請參閱「思考」一節。
物件偵測 (縮放及裁剪)
以下範例使用執行程式碼，在偵測到物件並傳回定界框時，縮放及裁剪圖片，以獲得更清楚的檢視畫面。
Python
from google import genai
client = genai.Client()
uploaded_file = client.files.upload(file="sorting.jpeg")
prompt = """
Return JSON in the format {label: val, y: val, x: val, y2: val, x2: val} for
the compostable objects in this scene. Please Zoom and crop the image for a
clearer view. Return an annotated image of the final result with the bounding
boxes drawn on it to the API caller as a part of your process.
"""
interaction = client.interactions.create(
model="gemini-robotics-er-2-preview",
input=[
{
"type": "image",
"uri": uploaded_file.uri,
"mime_type": uploaded_file.mime_type
},
{"type": "text", "text": prompt}
],
tools=[{"type": "code_execution"}]
)
print(interaction.output_text)
模型輸出內容會類似下列 JSON 回應：
[
{"label": "compostable", "y": 256, "x": 482, "y2": 295, "x2": 546},
{"label": "compostable", "y": 317, "x": 478, "y2": 350, "x2": 542},
{"label": "compostable", "y": 586, "x": 556, "y2": 668, "x2": 595},
{"label": "compostable", "y": 463, "x": 669, "y2": 511, "x2": 718},
{"label": "compostable", "y": 178, "x": 565, "y2": 250, "x2": 609}
]
下圖顯示模型傳回的方塊。
讀取類比儀表並套用邏輯
以下範例說明如何使用模型讀取類比儀表，並執行時間計算。並使用系統指令強制輸出 JSON 格式。
Python
from google import genai
client = genai.Client()
uploaded_file = client.files.upload(file="gauge.jpeg")
interaction = client.interactions.create(
model="gemini-robotics-er-2-preview",
system_instruction="Be precise. When JSON is requested, reply with ONLY that JSON (no preface, no code block).",
input=[
{
"type": "image",
"uri": uploaded_file.uri,
"mime_type": uploaded_file.mime_type
},
{"type": "text", "text": """Read the current value from this gauge. Then, calculate how long
it will take at the current rate for the value to reach maximum.
Reply in JSON: {"current_value": val, "max_value": val,
"time_to_max_minutes": val}"""}
],
tools=[{"type": "code_execution"}]
)
print(interaction.output_text)
測量容器中的液體
以下範例說明如何使用執行程式碼功能，測量容器中的液體量。
Python
from google import genai
client = genai.Client()
uploaded_file = client.files.upload(file="fluid.jpeg")
interaction = client.interactions.create(
model="gemini-robotics-er-2-preview",
system_instruction="Be precise. When JSON is requested, reply with ONLY that JSON (no preface, no code block).",
input=[
{
"type": "image",
"uri": uploaded_file.uri,
"mime_type": uploaded_file.mime_type
},
{"type": "text", "text": """Measure the amount of fluid in the container. Reply in JSON:
{"fluid_level_ml": val, "container_capacity_ml": val,
"percentage_full": val}"""}
],
tools=[{"type": "code_execution"}]
)
print(interaction.output_text)
解讀電路板上的標記
以下範例說明如何使用程式碼執行功能，讀取電路板上的標記。
Python
from google import genai
client = genai.Client()
uploaded_file = client.files.upload(file="circuit_board.jpeg")
interaction = client.interactions.create(
model="gemini-robotics-er-2-preview",
system_instruction="Be precise. When JSON is requested, reply with ONLY that JSON (no preface, no code block).",
input=[
{
"type": "image",
"uri": uploaded_file.uri,
"mime_type": uploaded_file.mime_type
},
{"type": "text", "text": """Read all visible component labels and markings on this circuit
board. Reply in JSON: {"components": [{"label": val,
"location": [y, x]}]}"""}
],
tools=[{"type": "code_execution"}]
)
print(interaction.output_text)
圖片註解
以下範例說明如何使用執行程式碼功能為圖片加上註解 (例如繪製箭頭表示處理說明)，並傳回修改後的圖片。
Python
from google import genai
client = genai.Client()
# Load your image
uploaded_file = client.files.upload(file="sorting.jpeg")
prompt = """
Look at this image and return it as an annotated version using arrows of
different colors to represent which items should go in which bins for
disposal. You must return the final image to the API caller.
"""
interaction = client.interactions.create(
model="gemini-robotics-er-2-preview",
input=[
{
"type": "image",
"uri": uploaded_file.uri,
"mime_type": uploaded_file.mime_type
},
{"type": "text", "text": prompt}
],
tools=[{"type": "code_execution"}]
)
print(interaction.output_text)
以下是圖片輸入內容範例。
模型輸出內容會與下列內容類似：
The annotated image shows the suggested disposal locations for the items on the table:
- **Green bin (Compost/Organic)**: Green chili, red chili, grapes, and cherries.
- **Blue bin (Recycling)**: Yellow crushed can and plastic container.
- **Black bin (Trash)**: Chocolate bar wrapper, Welch's packet, and white tissue.