Using files

Method: media.upload

Creates a File.

Endpoint

upload uri, for media upload requests:
`post https://generativelanguage.googleapis.com/upload/v1beta/files
  • Metadata URI, for metadata-only requests:
    POST https://generativelanguage.googleapis.com/v1beta/filesThe URL uses gRPC Transcoding syntax.

Request body

The request body contains data with the following structure:

Fields
file object (File)

Optional. Metadata for the file to create.

Example request

Image

Python

myfile = genai.upload_file(media / "Cajun_instruments.jpg")
print(f"{myfile=}")

model = genai.GenerativeModel("gemini-1.5-flash")
result = model.generate_content(
    [myfile, "\n\n", "Can you tell me about the instruments in this photo?"]
)
print(f"{result.text=}")

Node.js

// Make sure to include these imports:
// import { GoogleAIFileManager } from "@google/generative-ai/server";
// import { GoogleGenerativeAI } from "@google/generative-ai";
const fileManager = new GoogleAIFileManager(process.env.API_KEY);

const uploadResult = await fileManager.uploadFile(
  `${mediaPath}/jetpack.jpg`,
  {
    mimeType: "image/jpeg",
    displayName: "Jetpack drawing",
  },
);
// View the response.
console.log(
  `Uploaded file ${uploadResult.file.displayName} as: ${uploadResult.file.uri}`,
);

const genAI = new GoogleGenerativeAI(process.env.API_KEY);
const model = genAI.getGenerativeModel({ model: "gemini-1.5-flash" });
const result = await model.generateContent([
  "Tell me about this image.",
  {
    fileData: {
      fileUri: uploadResult.file.uri,
      mimeType: uploadResult.file.mimeType,
    },
  },
]);
console.log(result.response.text());

Audio

Python

myfile = genai.upload_file(media / "sample.mp3")
print(f"{myfile=}")

model = genai.GenerativeModel("gemini-1.5-flash")
result = model.generate_content([myfile, "Describe this audio clip"])
print(f"{result.text=}")

Node.js

// Make sure to include these imports:
// import { GoogleAIFileManager, FileState } from "@google/generative-ai/server";
// import { GoogleGenerativeAI } from "@google/generative-ai";
const fileManager = new GoogleAIFileManager(process.env.API_KEY);

const uploadResult = await fileManager.uploadFile(
  `${mediaPath}/samplesmall.mp3`,
  {
    mimeType: "audio/mp3",
    displayName: "Audio sample",
  },
);

let file = await fileManager.getFile(uploadResult.file.name);
while (file.state === FileState.PROCESSING) {
  process.stdout.write(".");
  // Sleep for 10 seconds
  await new Promise((resolve) => setTimeout(resolve, 10_000));
  // Fetch the file from the API again
  file = await fileManager.getFile(uploadResult.file.name);
}

if (file.state === FileState.FAILED) {
  throw new Error("Audio processing failed.");
}

// View the response.
console.log(
  `Uploaded file ${uploadResult.file.displayName} as: ${uploadResult.file.uri}`,
);

const genAI = new GoogleGenerativeAI(process.env.API_KEY);
const model = genAI.getGenerativeModel({ model: "gemini-1.5-flash" });
const result = await model.generateContent([
  "Tell me about this audio clip.",
  {
    fileData: {
      fileUri: uploadResult.file.uri,
      mimeType: uploadResult.file.mimeType,
    },
  },
]);
console.log(result.response.text());

Text

Python

myfile = genai.upload_file(media / "poem.txt")
print(f"{myfile=}")

model = genai.GenerativeModel("gemini-1.5-flash")
result = model.generate_content(
    [myfile, "\n\n", "Can you add a few more lines to this poem?"]
)
print(f"{result.text=}")

Node.js

// Make sure to include these imports:
// import { GoogleAIFileManager } from "@google/generative-ai/server";
// import { GoogleGenerativeAI } from "@google/generative-ai";
const fileManager = new GoogleAIFileManager(process.env.API_KEY);

const uploadResult = await fileManager.uploadFile(`${mediaPath}/a11.txt`, {
  mimeType: "text/plain",
  displayName: "Apollo 11",
});
// View the response.
console.log(
  `Uploaded file ${uploadResult.file.displayName} as: ${uploadResult.file.uri}`,
);

const genAI = new GoogleGenerativeAI(process.env.API_KEY);
const model = genAI.getGenerativeModel({ model: "gemini-1.5-flash" });
const result = await model.generateContent([
  "Transcribe the first few sentences of this document.",
  {
    fileData: {
      fileUri: uploadResult.file.uri,
      mimeType: uploadResult.file.mimeType,
    },
  },
]);
console.log(result.response.text());

Video

Python

import time

# Video clip (CC BY 3.0) from https://peach.blender.org/download/
myfile = genai.upload_file(media / "Big_Buck_Bunny.mp4")
print(f"{myfile=}")

# Videos need to be processed before you can use them.
while myfile.state.name == "PROCESSING":
    print("processing video...")
    time.sleep(5)
    myfile = genai.get_file(myfile.name)

model = genai.GenerativeModel("gemini-1.5-flash")
result = model.generate_content([myfile, "Describe this video clip"])
print(f"{result.text=}")

Node.js

// Make sure to include these imports:
// import { GoogleAIFileManager, FileState } from "@google/generative-ai/server";
// import { GoogleGenerativeAI } from "@google/generative-ai";
const fileManager = new GoogleAIFileManager(process.env.API_KEY);

const uploadResult = await fileManager.uploadFile(
  `${mediaPath}/Big_Buck_Bunny.mp4`,
  {
    mimeType: "video/mp4",
    displayName: "Big Buck Bunny",
  },
);

let file = await fileManager.getFile(uploadResult.file.name);
while (file.state === FileState.PROCESSING) {
  process.stdout.write(".");
  // Sleep for 10 seconds
  await new Promise((resolve) => setTimeout(resolve, 10_000));
  // Fetch the file from the API again
  file = await fileManager.getFile(uploadResult.file.name);
}

if (file.state === FileState.FAILED) {
  throw new Error("Video processing failed.");
}

// View the response.
console.log(
  `Uploaded file ${uploadResult.file.displayName} as: ${uploadResult.file.uri}`,
);

const genAI = new GoogleGenerativeAI(process.env.API_KEY);
const model = genAI.getGenerativeModel({ model: "gemini-1.5-flash" });
const result = await model.generateContent([
  "Tell me about this video.",
  {
    fileData: {
      fileUri: uploadResult.file.uri,
      mimeType: uploadResult.file.mimeType,
    },
  },
]);
console.log(result.response.text());

Response body

Response for media.upload.

If successful, the response body contains data with the following structure:

Fields
file object (File)

Metadata for the created file.

JSON representation 
{
  "file": {
    object (File)
  }
}

Method: files.get

Gets the metadata for the given File.

Endpoint

get https://generativelanguage.googleapis.com/v1beta/{name=files/*}

Path parameters

name string

Required. The name of the File to get. Example: files/abc-123 It takes the form files/{file}.

Request body

The request body must be empty.

Example request

Python

myfile = genai.upload_file(media / "poem.txt")
file_name = myfile.name
print(file_name)  # "files/*"

myfile = genai.get_file(file_name)
print(myfile)

Node.js

// Make sure to include these imports:
// import { GoogleAIFileManager } from "@google/generative-ai/server";
const fileManager = new GoogleAIFileManager(process.env.API_KEY);

const uploadResponse = await fileManager.uploadFile(
  `${mediaPath}/jetpack.jpg`,
  {
    mimeType: "image/jpeg",
    displayName: "Jetpack drawing",
  },
);

// Get the previously uploaded file's metadata.
const getResponse = await fileManager.getFile(uploadResponse.file.name);

// View the response.
console.log(
  `Retrieved file ${getResponse.displayName} as ${getResponse.uri}`,
);

Response body

If successful, the response body contains an instance of File.

Method: files.list

Lists the metadata for Files owned by the requesting project.

Endpoint

get https://generativelanguage.googleapis.com/v1beta/files

Query parameters

pageSize integer

Optional. Maximum number of Files to return per page. If unspecified, defaults to 10. Maximum pageSize is 100.

pageToken string

Optional. A page token from a previous files.list call.

Request body

The request body must be empty.

Example request

Python

print("My files:")
for f in genai.list_files():
    print("  ", f.name)

Node.js

// Make sure to include these imports:
// import { GoogleAIFileManager } from "@google/generative-ai/server";
const fileManager = new GoogleAIFileManager(process.env.API_KEY);

const listFilesResponse = await fileManager.listFiles();

// View the response.
for (const file of listFilesResponse.files) {
  console.log(`name: ${file.name} | display name: ${file.displayName}`);
}

Response body

Response for files.list.

If successful, the response body contains data with the following structure:

Fields
files[] object (File)

The list of Files.

nextPageToken string

A token that can be sent as a pageToken into a subsequent files.list call.

JSON representation 
{
  "files": [
    {
      object (File)
    }
  ],
  "nextPageToken": string
}

Method: files.delete

Deletes the File.

Endpoint

delete https://generativelanguage.googleapis.com/v1beta/{name=files/*}

Path parameters

name string

Required. The name of the File to delete. Example: files/abc-123 It takes the form files/{file}.

Request body

The request body must be empty.

Example request

Python

myfile = genai.upload_file(media / "poem.txt")

myfile.delete()

try:
    # Error.
    model = genai.GenerativeModel("gemini-1.5-flash")
    result = model.generate_content([myfile, "Describe this file."])
except google.api_core.exceptions.PermissionDenied:
    pass

Node.js

// Make sure to include these imports:
// import { GoogleAIFileManager } from "@google/generative-ai/server";
const fileManager = new GoogleAIFileManager(process.env.API_KEY);

const uploadResult = await fileManager.uploadFile(
  `${mediaPath}/jetpack.jpg`,
  {
    mimeType: "image/jpeg",
    displayName: "Jetpack drawing",
  },
);

// Delete the file.
await fileManager.deleteFile(uploadResult.file.name);

console.log(`Deleted ${uploadResult.file.displayName}`);

Response body

If successful, the response body is empty.

REST Resource: files

Resource: File

A file uploaded to the API.

JSON representation 
{
  "name": string,
  "displayName": string,
  "mimeType": string,
  "sizeBytes": string,
  "createTime": string,
  "updateTime": string,
  "expirationTime": string,
  "sha256Hash": string,
  "uri": string,
  "state": enum (State),
  "error": {
    object (Status)
  },

  // Union field metadata can be only one of the following:
  "videoMetadata": {
    object (VideoMetadata)
  }
  // End of list of possible types for union field metadata.
}
Fields
name string

Immutable. Identifier. The File resource name. The ID (name excluding the "files/" prefix) can contain up to 40 characters that are lowercase alphanumeric or dashes (-). The ID cannot start or end with a dash. If the name is empty on create, a unique name will be generated. Example: files/123-456

displayName string

Optional. The human-readable display name for the File. The display name must be no more than 512 characters in length, including spaces. Example: "Welcome Image"

mimeType string

Output only. MIME type of the file.

sizeBytes string (int64 format)

Output only. Size of the file in bytes.

createTime string (Timestamp format)

Output only. The timestamp of when the File was created.

A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: "2014-10-02T15:01:23Z" and "2014-10-02T15:01:23.045123456Z".

updateTime string (Timestamp format)

Output only. The timestamp of when the File was last updated.

A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: "2014-10-02T15:01:23Z" and "2014-10-02T15:01:23.045123456Z".

expirationTime string (Timestamp format)

Output only. The timestamp of when the File will be deleted. Only set if the File is scheduled to expire.

A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: "2014-10-02T15:01:23Z" and "2014-10-02T15:01:23.045123456Z".

sha256Hash string (bytes format)

Output only. SHA-256 hash of the uploaded bytes.

A base64-encoded string.

uri string

Output only. The uri of the File.

state enum (State)

Output only. Processing state of the File.

error object (Status)

Output only. Error status if File processing failed.

Union field metadata. Metadata for the File. metadata can be only one of the following:
videoMetadata object (VideoMetadata)

Output only. Metadata for a video.

VideoMetadata

Metadata for a video File.

JSON representation 
{
  "videoDuration": string
}
Fields
videoDuration string (Duration format)

Duration of the video.

A duration in seconds with up to nine fractional digits, ending with 's'. Example: "3.5s".

State

States for the lifecycle of a File.

Enums
STATE_UNSPECIFIED The default value. This value is used if the state is omitted.
PROCESSING File is being processed and cannot be used for inference yet.
ACTIVE File is processed and available for inference.
FAILED File failed processing.

Status

The Status type defines a logical error model that is suitable for different programming environments, including REST APIs and RPC APIs. It is used by gRPC. Each Status message contains three pieces of data: error code, error message, and error details.

You can find out more about this error model and how to work with it in the API Design Guide.

JSON representation 
{
  "code": integer,
  "message": string,
  "details": [
    {
      "@type": string,
      field1: ...,
      ...
    }
  ]
}
Fields
code integer

The status code, which should be an enum value of google.rpc.Code.

message string

A developer-facing error message, which should be in English. Any user-facing error message should be localized and sent in the google.rpc.Status.details field, or localized by the client.

details[] object

A list of messages that carry the error details. There is a common set of message types for APIs to use.

An object containing fields of an arbitrary type. An additional field "@type" contains a URI identifying the type. Example: { "id": 1234, "@type": "types.example.com/standard/id" }.