The Gemini API gives you access to the latest generative models from Google. Once you're familiar with the general features available to you through the API, try a tutorial for your language of choice to start developing.
Models
Gemini is a series of multimodal generative AI models developed by Google. Gemini models can accept text and image in prompts, depending on what model variation you choose, and output text responses.
To get more detailed model information refer to the
Gemini models page. You can also use the
list_models
method to list all
the models available and then the
get_model
method to get the
metadata for a particular model.
Prompt data and design
Specific Gemini models accept both text data and media files as input. This capability creates many additional possibilities for generating content, analyzing data, and solving problems. There are some limitations and requirements to consider, including the general input token limit for the model you are using. For information on the token limits for specific models, see Gemini models.
Prompts using the Gemini API cannot exceed 20MB in size. The Gemini API provides a File API for temporarily storing media files for use in prompting, which lets you provide prompt data beyond the 20MB limit. For more information on using the Files API and file formats supported for prompting, see Prompting with media files.
Prompt design and text input
Creating effective prompts, or prompt engineering, is a combination of art and science. See the intro to prompting for guidance on how to approach prompting and the prompt 101 guide to learn about different approaches to prompting.
Generate content
The Gemini API lets you use both text and image data for prompting, depending on
what model variation you use. For example, you can generate text from either
text-only prompts or multimodal prompts using a Gemini 1.5 model. This section
gives basic code examples of each. Refer to the
generateContent
API reference for a
more detailed example that covers all of the parameters.
Text and image input
You can send a text prompt with an image to a Gemini 1.5 model to perform a vision-related task. For example, captioning an image or identifying that's in an image.
The following code examples demonstrate a basic implementation of a text and image prompt for each supported language:
Python
model = genai.GenerativeModel('gemini-1.5-flash')
cookie_picture = {
'mime_type': 'image/png',
'data': pathlib.Path('cookie.png').read_bytes()
}
prompt = "Do these look store-bought or homemade?"
response = model.generate_content(
model="gemini-1.5-flash",
content=[prompt, cookie_picture]
)
print(response.text)
See the Python tutorial to see the complete code snippet.
Go
vmodel := client.GenerativeModel("gemini-1.5-flash")
data, err := os.ReadFile(filepath.Join("path-to-image", imageFile))
if err != nil {
log.Fatal(err)
}
resp, err := vmodel.GenerateContent(ctx, genai.Text("Do these look store-bought or homemade?"), genai.ImageData("jpeg", data))
if err != nil {
log.Fatal(err)
}
See the Go tutorial for a full example.
Node.js
const model = genAI.getGenerativeModel({ model: "gemini-1.5-flash" });
const prompt = "Do these look store-bought or homemade?";
const image = {
inlineData: {
data: Buffer.from(fs.readFileSync("cookie.png")).toString("base64"),
mimeType: "image/png",
},
};
const result = await model.generateContent([prompt, image]);
console.log(result.response.text());
See the Node.js tutorial for a full example.
Web
const model = genAI.getGenerativeModel({ model: "gemini-1.5-flash" });
const prompt = "Do these look store-bought or homemade?";
const image = {
inlineData: {
data: base64EncodedImage /* see JavaScript quickstart for details */,
mimeType: "image/png",
},
};
const result = await model.generateContent([prompt, image]);
console.log(result.response.text());
See the web tutorial for a full example.
Dart (Flutter)
final model = GenerativeModel(model: 'gemini-1.5-flash', apiKey: apiKey);
final prompt = 'Do these look store-bought or homemade?';
final imageBytes = await File('cookie.png').readAsBytes();
final content = [
Content.multi([
TextPart(prompt),
DataPart('image/png', imageBytes),
])
];
final response = await model.generateContent(content);
print(response.text);
See the Dart (Flutter) tutorial for a full example.
Swift
let model = GenerativeModel(name: "gemini-1.5-flash", apiKey: "API_KEY")
let cookieImage = UIImage(...)
let prompt = "Do these look store-bought or homemade?"
let response = try await model.generateContent(prompt, cookieImage)
See the Swift tutorial for a full example.
Android
val generativeModel = GenerativeModel(
modelName = "gemini-1.5-flash",
apiKey = BuildConfig.apiKey
)
val cookieImage: Bitmap = // ...
val inputContent = content() {
image(cookieImage)
text("Do these look store-bought or homemade?")
}
val response = generativeModel.generateContent(inputContent)
print(response.text)
See the Android tutorial for a full example.
cURL
curl https://generativelanguage.googleapis.com/v1/models/gemini-1.5-flash:generateContent?key=${API_KEY} \
-H 'Content-Type: application/json' \
-X POST \
-d @<(echo'{
"contents":[
{ "parts":[
{"text": "Do these look store-bought or homemade?"},
{ "inlineData": {
"mimeType": "image/png",
"data": "'$(base64 -w0 cookie.png)'"
}
}
]
}
]
}')
See the REST API tutorial for more details.
Text only input
The Gemini API can also handle text-only input. This feature lets you perform natural language processing (NLP) tasks such as text completion and summarization.
The following code examples demonstrate a basic implementation of a text-only prompt for each supported language:
Python
model = genai.GenerativeModel('gemini-1.5-flash')
prompt = "Write a story about a magic backpack."
response = model.generate_content(prompt)
See the Python tutorial for the full example.
Go
ctx := context.Background()
client, err := genai.NewClient(ctx, option.WithAPIKey(os.Getenv("API_KEY")))
if err != nil {
log.Fatal(err)
}
defer client.Close()
model := client.GenerativeModel("gemini-1.5-flash")
resp, err := model.GenerateContent(ctx, genai.Text("Write a story about a magic backpack."))
if err != nil {
log.Fatal(err)
}
See the Go tutorial for a full example.
Node.js
const model = genAI.getGenerativeModel({ model: "gemini-1.5-flash" });
const prompt = "Write a story about a magic backpack.";
const result = await model.generateContent(prompt);
console.log(result.response.text());
See the Node.js tutorial for a full example.
Web
const model = genAI.getGenerativeModel({ model: "gemini-1.5-flash" });
const prompt = "Write a story about a magic backpack.";
const result = await model.generateContent(prompt);
console.log(result.response.text());
See the web tutorial for a full example.
Dart (Flutter)
final model = GenerativeModel(model: 'gemini-1.5-flash', apiKey: apiKey);
final prompt = 'Write a story about a magic backpack.';
final content = [Content.text(prompt)];
final response = await model.generateContent(content);
print(response.text);
See the Dart (Flutter) tutorial for a full example.
Swift
let model = GenerativeModel(name: "gemini-1.5-flash", apiKey: "API_KEY")
let prompt = "Write a story about a magic backpack."
let response = try await model.generateContent(prompt)
See the Swift tutorial for a full example.
Android
val generativeModel = GenerativeModel(
modelName = "gemini-1.5-flash",
apiKey = BuildConfig.apiKey
)
val prompt = "Write a story about a magic backpack."
val response = generativeModel.generateContent(prompt)
print(response.text)
See the Android tutorial for a full example.
cURL
curl https://generativelanguage.googleapis.com/v1/models/gemini-1.5-flash:generateContent?key=$API_KEY \
-H 'Content-Type: application/json' \
-X POST \
-d '{ "contents":[
{ "parts":[{"text": "Write a story about a magic backpack"}]}
]
}'
See the REST API tutorial for more details.
Multi-turn conversations (chat)
You can use the Gemini API to build interactive chat experiences for your users. Using the chat feature of the API lets you collect multiple rounds of questions and responses, allowing users to incrementally step toward answers or get help with multi-part problems. This feature is ideal for applications that require ongoing communication, such as chatbots, interactive tutors, or customer support assistants.
The following code examples demonstrate a basic implementation of chat interaction for each supported language:
Python
model = genai.GenerativeModel('gemini-1.5-flash')
chat = model.start_chat(history=[])
response = chat.send_message(
"Pretend you\'re a snowman and stay in character for each response.")
print(response.text)
response = chat.send_message(
"What\'s your favorite season of the year?")
print(response.text)
See the chat demo in the Python tutorial for a full example.
Go
model := client.GenerativeModel("gemini-1.5-flash")
cs := model.StartChat()
cs.History = []*genai.Content{
&genai.Content{
Parts: []genai.Part{
genai.Text("Pretend you're a snowman and stay in character for each response."),
},
Role: "user",
},
&genai.Content{
Parts: []genai.Part{
genai.Text("Hello! It's cold! Isn't that great?"),
},
Role: "model",
},
}
resp, err := cs.SendMessage(ctx, genai.Text("What's your favorite season of the year?"))
if err != nil {
log.Fatal(err)
}
See the chat demo in the Go tutorial for a full example.
Node.js
const model = genAI.getGenerativeModel({ model: "gemini-1.5-flash"});
const chat = model.startChat({
history: [
{
role: "user",
parts: "Pretend you're a snowman and stay in character for each response.",
},
{
role: "model",
parts: "Hello! It's cold! Isn't that great?",
},
],
generationConfig: {
maxOutputTokens: 100,
},
});
const msg = "What's your favorite season of the year?";
const result = await chat.sendMessage(msg);
console.log(result.response.text());
See the chat demo in the Node.js tutorial for a full example.
Web
const model = genAI.getGenerativeModel({ model: "gemini-1.5-flash"});
const chat = model.startChat({
history: [
{
role: "user",
parts: "Pretend you're a snowman and stay in character for each response.",
},
{
role: "model",
parts: "Hello! It's so cold! Isn't that great?",
},
],
generationConfig: {
maxOutputTokens: 100,
},
});
const msg = "What's your favorite season of the year?";
const result = await chat.sendMessage(msg);
console.log(result.response.text());
See the chat demo in the web tutorial for a full example.
Dart (Flutter)
final model = GenerativeModel(model: 'gemini-1.5-flash', apiKey: apiKey);
final chat = model.startChat(history: [
Content.text(
"Pretend you're a snowman and stay in character for each response."),
Content.model([TextPart("Hello! It's cold! Isn't that great?")]),
]);
final content = Content.text("What's your favorite season of the year?");
final response = await chat.sendMessage(content);
print(response.text);
See the chat demo in the Dart (Flutter) tutorial for a full example.
Swift
let model = GenerativeModel(name: "gemini-1.5-flash", apiKey: "API_KEY")
let chat = model.startChat()
var message = "Pretend you're a snowman and stay in character for each response."
var response = try await chat.sendMessage(message)
message = "What\'s your favorite season of the year?"
response = try await chat.sendMessage(message)
See the chat demo in the Swift tutorial for a full example.
Android
val generativeModel = GenerativeModel(
modelName = "gemini-1.5-flash",
apiKey = BuildConfig.apiKey
)
val chat = generativeModel.startChat()
val response = chat.sendMessage("Pretend you're a snowman and stay in
character for each response.")
print(response.text)
See the Android tutorial for a full example.
cURL
curl https://generativelanguage.googleapis.com/v1beta/models/gemini-1.5-flash:generateContent?key=$API_KEY \ -H 'Content-Type: application/json' \ -X POST \ -d '{ "contents": [ {"role":"user", "parts":[{ "text": "Pretend you're a snowman and stay in character for each {"role": "model", response."}]}, "parts":[{ "text": "Hello! It's so cold! Isn't that great?"}]}, {"role": "user", "parts":[{ "text": "What\'s your favorite season of the year?"}]}, ] }' 2> /dev/null | grep "text"
# response example: "text": "Winter, of course!"
See the REST API tutorial for more details.
Streamed responses
The Gemini API provides an additional way to receive responses from generative AI models: as a data stream. A streamed response sends incremental pieces of data back to your application as it is generated by the model. This feature lets you respond quickly to a user request to show progress and create a more interactive experience.
Streamed responses are an option for freeform prompting and chats with Gemini models. The following code examples show how to request a streamed response for a prompt for each supported language:
Python
prompt = "Write a story about a magic backpack."
response = genai.stream_generate_content(
model="models/gemini-1.5-flash",
prompt=prompt
)
See the Python tutorial to see the complete code snippet.
Go
ctx := context.Background()
client, err := genai.NewClient(ctx, option.WithAPIKey(os.Getenv("API_KEY")))
if err != nil {
log.Fatal(err)
}
defer client.Close()
model := client.GenerativeModel("gemini-1.5-flash")
iter := model.GenerateContentStream(ctx, genai.Text("Write a story about a magic backpack."))
for {
resp, err := iter.Next()
if err == iterator.Done {
break
}
if err != nil {
log.Fatal(err)
}
// print resp
}
See the Go tutorial for a full example.
Node.js
const model = genAI.getGenerativeModel({ model: "gemini-1.5-flash" });
const prompt = "Write a story about a magic backpack.";
const result = await model.generateContentStream([prompt]);
// print text as it comes in
for await (const chunk of result.stream) {
const chunkText = chunk.text();
console.log(chunkText);
}
See the Node.js tutorial for a full example.
Web
const model = genAI.getGenerativeModel({ model: "gemini-1.5-flash" });
const prompt = "Write a story about a magic backpack.";
const result = await model.generateContentStream([prompt]);
// print text as it comes in
for await (const chunk of result.stream) {
const chunkText = chunk.text();
console.log(chunkText);
}
See the web tutorial for a full example.
Dart (Flutter)
final model = GenerativeModel(model: 'gemini-1.5-flash', apiKey: apiKey);
final prompt = 'Write a story about a magic backpack.';
final content = [Content.text(prompt)];
final response = model.generateContentStream(content);
await for (final chunk in response) {
print(chunk.text);
}
See the Dart (Flutter) tutorial for a full example.
Swift
let model = GenerativeModel(name: "gemini-1.5-flash", apiKey: "API_KEY")
let prompt = "Write a story about a magic backpack."
let stream = model.generateContentStream(prompt)
for try await chunk in stream {
print(chunk.text ?? "No content")
}
See the Swift tutorial for a full example.
Android
val generativeModel = GenerativeModel(
modelName = "gemini-1.5-flash",
apiKey = BuildConfig.apiKey
)
val inputContent = content {
text("Write a story about a magic backpack.")
}
var fullResponse = ""
generativeModel.generateContentStream(inputContent).collect { chunk ->
print(chunk.text)
fullResponse += chunk.text
}
See the Android tutorial for a full example.
cURL
curl https://generativelanguage.googleapis.com/v1/models/gemini-1.5-flash:streamGenerateContent?key=${API_KEY} \
-H 'Content-Type: application/json' \
--no-buffer \
-d '{ "contents":[
{"role": "user",
"parts":[{"text": "Write a story about a magic backpack."}]
}
]
}' > response.json
See the REST API tutorial for more details.
JSON format responses
Depending on your application, you may want the response to a prompt to be returned in a structured data format, particularly if you are using the responses to populate programming interfaces. The Gemini API provides a configuration parameter to request a response in JSON format.
You can have the model output JSON by setting the response_mime_type
configuration option to application/json
and in the prompt, describe the
format of JSON you want in response:
Python
model = genai.GenerativeModel('gemini-1.5-flash',
generation_config={"response_mime_type": "application/json"})
prompt = """
List 5 popular cookie recipes.
Using this JSON schema:
Recipe = {"recipe_name": str}
Return a `list[Recipe]`
"""
response = model.generate_content(prompt)
print(response.text)
cURL
curl https://generativelanguage.googleapis.com/v1beta/models/gemini-1.5-flash:generateContent?key=$API_KEY \
-H 'Content-Type: application/json' \
-X POST \
-d '{
"contents": [
{
"parts": [
{
"text": "\nList 5 popular cookie recipes.\n\nUsing this JSON schema:\n\n Recipe = {\"recipe_name\": str}\n\nReturn a `list[Recipe]`\n "
}
]
}
]
"generationConfig": {
"response_mime_type": "application/json",
}
}'
While Gemini 1.5 Flash models only accept a text description of the JSON schema you want returned, the Gemini 1.5 Pro models let you pass a schema object (or a Python type equivalent), and the model output will strictly follow that schema. This is also known as controlled generation or constrained decoding.
For example, to get a list of Recipe
objects, pass list[Recipe]
to the
response_schema
field of the generation_config
argument:
Python
import typing_extensions as typing
class Recipe(typing.TypedDict):
recipe_name: str
model = genai.GenerativeModel(model_name="models/gemini-1.5-pro")
result = model.generate_content(
"List 5 popular cookie recipes",
generation_config=genai.GenerationConfig(response_mime_type="application/json",
response_schema = list[Recipe]))
print(result.text)
cURL
curl https://generativelanguage.googleapis.com/v1beta/models/models/gemini-1.5-pro:generateContent?
-H 'Content-Type: application/json'
-X POST \
-d '{
"contents": [
{
"parts": [
{
"text": "List 5 popular cookie recipes"
}
]
}
],
"generationConfig": {
"responseMimeType": "application/json",
"responseSchema": {
"type": "ARRAY",
"items": {
"type": "OBJECT",
"properties": {
"recipe_name": {
"type": "STRING"
}
}
}
}
}
}'
```
To learn more, refer to the JSON mode quickstart in the Gemini API cookbook.
Embeddings
The embedding service in the Gemini API generates state-of-the-art embeddings for words, phrases, and sentences. The resulting embeddings can then be used for NLP tasks, such as semantic search, text classification, and clustering, among many others. See the embeddings guide to learn what embeddings are and some key use cases for the embedding service to help you get started.
Next steps
- Get started with the Google AI Studio UI using the Google AI Studio quickstart.
- Try out server-side access to the Gemini API with the tutorials for Python, Go, or Node.js.
- Start building for the web with the web tutorial.
- Start building for mobile apps with the Swift tutorial or the Android tutorial.
- If you're an existing Google Cloud user (or you would like to use Gemini on Vertex to take advantage of the powerful Google Cloud ecosystem), check out Generative AI on Vertex AI to learn more.