Google Cloud logo

Build with Vertex AI on Google Cloud

Preview access

Gemini 1.5 Pro

Gemini 1.5 Pro achieves a breakthrough context window of 1 million tokens, the longest of any foundational model yet.

Three Gemini sizes
for unmatched versatility

Preview access

1.0 Ultra

Most capable model for large-scale, highly complex text and image reasoning tasks coming in early 2024.

Available now

1.0 Pro

The best performing model with features for a wide variety of text and image reasoning tasks.

Preview access

1.0 Nano

The most efficient model built for on-device experiences, enabling offline use cases. Leverages device processing power at no cost.

Integrate Gemini into your app with the API

It’s fast and free to start building with Gemini 1.0 Pro in Google AI Studio

1
2
3
4
5
6

Fully managed AI platform

Leverage Google Cloud's security, safety, privacy, data governance and compliance to customize Gemini with complete data control.

Built for scale

Tune and augment Gemini with your data, then manage and deploy in your AI-powered apps, search experiences, and conversational agents.

Performance benchmarks

Capability
Benchmark
Gemini 1.0 Ultra
GPT-4 (V)
General
MMLU

MMLU Representation of questions in 57 subjects (incl. STEM, humanities, and others)

90.0%
CoT@32*
86.4%
5-shot*** (reported)
Reasoning
Big-Bench Hard

Big-Bench Hard Diverse set of challenging tasks requiring multi-step reasoning

83.6%
3-shot
83.1%
3-shot (API)
Reasoning
DROP

DROP Reading comprehension (F1 Score)

82.4
Variable shots
80.9
3-shot (reported)
Reasoning
HellaSwag

HellaSwag Common sense reasoning for everyday tasks

87.8%
10-shot*
95.3%
10-shot* (reported)
Math
GSM8K

GSM8K Basic arithmetic manipulations (incl. Grade School math problems)

94.4%
maj1@32
92.0%
5-shot CoT (reported)
Math
MATH

MATH Challenging math problems (incl. algebra, geometry, pre-calculus, and others)

53.2%
4-shot
52.9%
4-shot (API)
Code
HumanEval

HumanEval Python code generation

74.4%
0-shot (IT)*
67.0%
0-shot* (reported)
Code
Natural2Code

Natural2Code Python code generation. New held out dataset HumanEval-like, not leaked on the web

74.9%
0-shot
73.9%
0-shot (API)
Image (Multimodal)
MMMU

MMMU Multi-discipline college-level reasoning problems

59.4%
0-shot pass@1 (pixel only**)
56.8%
0-shot pass@1 GPT-4V
Image (Multimodal)
VQAv2

VQAv2 Natural image understanding

77.8%
0-shot (pixel only**)
77.2%
0-shot GPT-4V
Image (Multimodal)
TextVQA

TextVQA OCR on natural images

82.3%
0-shot (pixel only**)
78.0%
0-shot GPT-4V
Image (Multimodal)
DocVQA

DocVQA Document understanding

90.9%
0-shot (pixel only**)
88.4%
0-shot GPT-4V (pixel only**)
Image (Multimodal)
Infographic VQA

Infographic VQA Infographic understanding

80.3%
0-shot (pixel only**)
75.1%
0-shot GPT-4V (pixel only**)
Image (Multimodal)
MathVista

MathVista Mathematical reasoning in visual contexts

53.0%
0-shot (pixel only**)
49.9%
0-shot GPT-4V
Capability
Benchmark
Gemini 1.0 Pro
GPT-3.5 (No reported multimodal results)
General
MMLU

MMLU Representation of questions in 57 subjects (incl. STEM, humanities, and others)

79.1%
CoT@8*
70.0%
5-shot* (reported)
Reasoning
Big-Bench Hard

Big-Bench Hard Diverse set of challenging tasks requiring multi-step reasoning

75.0%
3-shot
66.6%
3-shot (API)
Reasoning
DROP

DROP Reading comprehension (F1 Score)

74.1
F1 Score
64.1
3-shot (reported)
Reasoning
HellaSwag

HellaSwag Common sense reasoning for everyday tasks

84.7%
10-shot*
85.5%
10-shot* (reported)
Math
GSM8K

GSM8K Basic arithmetic manipulations (incl. Grade School math problems)

86.5%
maj1@32
57.1%
5-shot CoT (reported)
Math
MATH

MATH Challenging math problems (incl. algebra, geometry, pre-calculus, and others)

32.6%
4-shot
34.1%
4-shot (API)
Code
HumanEval

HumanEval Python code generation

67.7%
0-shot (IT)*
48.1%
0-shot* (reported)
Code
Natural2Code

Natural2Code Python code generation. New held out dataset HumanEval-like, not leaked on the web

69.6%
0-shot
62.3%
0-shot (API)
Image (Multimodal)
MMMU

MMMU Multi-discipline college-level reasoning problems

47.9%
0-shot pass@1 (pixel only**)
-
Image (Multimodal)
VQAv2

VQAv2 Natural image understanding

71.2%
0-shot (pixel only**)
-
Image (Multimodal)
TextVQA

TextVQA OCR on natural images

74.6%
0-shot (pixel only**)
-
Image (Multimodal)
DocVQA

DocVQA Document understanding

88.1%
0-shot (pixel only**)
-
Image (Multimodal)
Infographic VQA

Infographic VQA Infographic understanding

75.2%
0-shot (pixel only**)
-
Image (Multimodal)
MathVista

MathVista Mathematical reasoning in visual contexts

45.2%
0-shot (pixel only**)
-
  • * See the technical report for details on performance with other methodologies
  • ** Gemini image benchmarks are pixel only - no assistance from OCR systems
  • *** GPT-4 scores 87.29% with CoT@32 - see the technical report for full comparison

Gemini developer ecosystem

Google tools

Partner logo for Android Studio
Android Studio
Partner logo for Colab
Colab
Partner logo for Firebase
Firebase
Partner logo for Project IDX
Project IDX

Partners

Partner logo for Chroma
Chroma
Partner logo for Context AI
Context AI
Partner logo for Haystack
Haystack
Partner logo for LangChain
LangChain
Partner logo for LlamaIndex
LlamaIndex
Partner logo for Qdrant
Qdrant
Partner logo for vellum
Vellum
Partner logo for Weaviate
Weaviate

Gemini Nano

Android AICore enables our most powerful phones to run Gemini 1.0 Nano, making it easy for you to build AI experiences that work offline or use sensitive info better kept on-device.

Available on Pixel 8 Pro devices and more soon