Explore document processing capabilities with the Gemini API

Python Node.js Go REST

The Gemini API supports PDF input, including long documents (up to 3600 pages). Gemini models process PDFs with native vision, and are therefore able to understand both text and image contents inside documents. With native PDF vision support, Gemini models are able to:

• Analyze diagrams, charts, and tables inside documents.
• Extract information into structured output formats.
• Answer questions about visual and text contents in documents.
• Summarize documents.
• Transcribe document content (e.g. to HTML) preserving layouts and formatting, for use in downstream applications (such as in RAG pipelines).

This tutorial demonstrates some possible ways to use the Gemini API with PDF documents. All output is text-only.

Before you begin: Set up your project and API key

Before calling the Gemini API, you need to set up your project and configure your API key.

Prompting with PDFs

This guide demonstrates how to upload and process PDFs using the File API or by including them as inline data.

Technical details

Gemini 1.5 Pro and 1.5 Flash support a maximum of 3,600 document pages. Document pages must be in one of the following text data MIME types:

• PDF - application/pdf
• JavaScript - application/x-javascript, text/javascript
• Python - application/x-python, text/x-python
• TXT - text/plain
• HTML - text/html
• CSS - text/css
• Markdown - text/md
• CSV - text/csv
• XML - text/xml
• RTF - text/rtf

Each document page is equivalent to 258 tokens.

While there are no specific limits to the number of pixels in a document besides the model's context window, larger pages are scaled down to a maximum resolution of 3072x3072 while preserving their original aspect ratio, while smaller pages are scaled up to 768x768 pixels. There is no cost reduction for pages at lower sizes, other than bandwidth, or performance improvement for pages at higher resolution.

For best results:

• Rotate pages to the correct orientation before uploading.
• Avoid blurry pages.
• If using a single page, place the text prompt after the page.

PDF input

For PDF payloads under 20MB, you can choose between uploading base64 encoded documents or directly uploading locally stored files.

Base64 encoded documents

You can process PDF documents directly from URLs. Here's a code snippet showing how to do this:

import httpx
import base64

model = genai.GenerativeModel("gemini-1.5-flash")
doc_url = "https://discovery.ucl.ac.uk/id/eprint/10089234/1/343019_3_art_0_py4t4l_convrt.pdf"  # Replace with the actual URL of your PDF

# Retrieve and encode the PDF
doc_data = base64.standard_b64encode(httpx.get(doc_url).content).decode("utf-8")

prompt = "Summarize this document"

response = model.generate_content([{'mime_type':'application/pdf', 'data': doc_data}, prompt])
print(response.text)

Locally stored PDFs

For locally stored PDFs, you can use the following approach:

import base64

model = genai.GenerativeModel("gemini-1.5-flash")
doc_path = "/path/to/file.pdf" # Replace with the actual path to your local PDF

# Read and encode the local file
with open(doc_path, "rb") as doc_file:
    doc_data = base64.standard_b64encode(doc_file.read()).decode("utf-8")

prompt = "Summarize this document"

response = model.generate_content([{'mime_type': 'application/pdf', 'data': doc_data}, prompt])

print(response.text)

Large PDFs

You can use the File API to upload a document of any size. Always use the File API when the total request size (including the files, text prompt, system instructions, etc.) is larger than 20 MB.

Call media.upload to upload a file using the File API. The following code uploads a document file and then uses the file in a call to models.generateContent.

Large PDFs from URLs (:#large-pdfs-urls)

Use the File API for large PDF files available from URLs, simplifying the process of uploading and processing these documents directly through their URLs:

import io
import httpx

model = genai.GenerativeModel("gemini-1.5-flash")
long_context_pdf_path = "https://www.nasa.gov/wp-content/uploads/static/history/alsj/a17/A17_FlightPlan.pdf" # Replace with the actual URL of your large PDF

# Retrieve and upload the PDF using the File API
doc_data = io.BytesIO(httpx.get(long_context_pdf_path).content)
sample_doc = genai.upload_file(data=doc_data, mime_type='application/pdf')

prompt = "Summarize this document"

response = model.generate_content([sample_doc, prompt])
print(response.text)

Large PDFs stored locally (:#large-pdfs-local)

import google.generativeai as genai

model = genai.GenerativeModel("gemini-1.5-flash")
sample_pdf = genai.upload_file(media / "test.pdf")
response = model.generate_content(["Give me a summary of this pdf file.", sample_pdf])
print(response.text)
files.py

You can verify the API successfully stored the uploaded file and get its metadata by calling files.get. Only the name (and by extension, the uri) are unique.

import google.generativeai as genai

myfile = genai.upload_file(media / "poem.txt")
file_name = myfile.name
print(file_name)  # "files/*"

myfile = genai.get_file(file_name)
print(myfile)
files.py

Multiple PDFs

The Gemini API is capable of processing multiple PDF documents in a single request, as long as the combined size of the documents and the text prompt stays within the model's context window.

import io
import httpx

model = genai.GenerativeModel("gemini-1.5-flash")

doc_url_1 = "https://arxiv.org/pdf/2312.11805" # Replace with the URL to your first PDF
doc_url_2 = "https://arxiv.org/pdf/2403.05530" # Replace with the URL to your second PDF

# Retrieve and upload both PDFs using the File API
doc_data_1 = io.BytesIO(httpx.get(doc_url_1).content)
doc_data_2 = io.BytesIO(httpx.get(doc_url_2).content)

sample_pdf_1 = genai.upload_file(data=doc_data_1, mime_type='application/pdf')
sample_pdf_2 = genai.upload_file(data=doc_data_2, mime_type='application/pdf')

prompt = "What is the difference between each of the main benchmarks between these two papers? Output these in a table."

response = model.generate_content([sample_pdf_1, sample_pdf_2, prompt])
print(response.text)

List files

You can list all files uploaded using the File API and their URIs using files.list.

import google.generativeai as genai

print("My files:")
for f in genai.list_files():
    print("  ", f.name)
files.py

Delete files

Files uploaded using the File API are automatically deleted after 2 days. You can also manually delete them using files.delete.

import google.generativeai as genai

myfile = genai.upload_file(media / "poem.txt")

myfile.delete()

try:
    # Error.
    model = genai.GenerativeModel("gemini-1.5-flash")
    result = model.generate_content([myfile, "Describe this file."])
except google.api_core.exceptions.PermissionDenied:
    pass
files.py

Context caching with PDFs

import os
from google.generativeai import caching
import io
import httpx

# Define the path to the PDF document (or use a URL)
long_context_pdf_path = "https://www.nasa.gov/wp-content/uploads/static/history/alsj/a17/A17_FlightPlan.pdf" # Replace with the URL of your large PDF
doc_data = io.BytesIO(httpx.get(long_context_pdf_path).content)

# Upload the PDF document using the File API
document = genai.upload_file(data=doc_data, mime_type='application/pdf')

# Specify the model name and system instruction for caching
model_name = "gemini-1.5-flash-002" # Ensure this matches the model you intend to use
system_instruction = "You are an expert analyzing transcripts."

# Create a cached content object
cache = caching.CachedContent.create(
    model=model_name,
    system_instruction=system_instruction,
    contents=[document], # The document(s) and other content you wish to cache
)

# Display the cache details
print(cache)

# Initialize a generative model from the cached content
model = genai.GenerativeModel.from_cached_content(cache)

# Generate content using the cached prompt and document
response = model.generate_content("Please summarize this transcript")

# (Optional) Print usage metadata for insights into the API call
print(response.usage_metadata)

# Print the generated text
print(response.text)

List caches

It's not possible to retrieve or view cached content, but you can retrieve cache metadata (name, model, display_name, usage_metadata, create_time, update_time, and expire_time).

To list metadata for all uploaded caches, use CachedContent.list():

for c in caching.CachedContent.list():
  print(c)

Update a cache

You can set a new ttl or expire_time for a cache. Changing anything else about the cache isn't supported.

The following example shows how to update the ttl of a cache using CachedContent.update().

import datetime

cache.update(ttl=datetime.timedelta(hours=2))

Delete a cache

The caching service provides a delete operation for manually removing content from the cache. The following example shows how to delete a cache using CachedContent.delete().

cache.delete()

What's next

This guide shows how to use generateContent and to generate text outputs from processed documents. To learn more, see the following resources:

• File prompting strategies: The Gemini API supports prompting with text, image, audio, and video data, also known as multimodal prompting.
• System instructions: System instructions let you steer the behavior of the model based on your specific needs and use cases.
• Safety guidance: Sometimes generative AI models produce unexpected outputs, such as outputs that are inaccurate, biased, or offensive. Post-processing and human evaluation are essential to limit the risk of harm from such outputs.