Gemini API: Tuning Quickstart with Python

View on ai.google.dev Run in Google Colab View source on GitHub Download notebook

In this notebook, you'll learn how to get started with the tuning service using the Python client library for the Gemini API. Here, you'll learn how to tune the text model behind the Gemini API's text generation service.

Setup

Authenticate

The Gemini API lets you tune models on your own data. Since it's your data and your tuned models this needs stricter access controls than API-Keys can provide.

Before you can run this tutorial, you'll need to setup OAuth for your project.

In Colab the easiest wat to get setup is to copy the contents of your client_secret.json file into Colab's "Secrets manager" (under the key icon in the left panel) with the secret name CLIENT_SECRET.

This gcloud command turns the client_secret.json file into credentials that can be used to authenticate with the service.

import os
if 'COLAB_RELEASE_TAG' in os.environ:
  from google.colab import userdata
  import pathlib
  pathlib.Path('client_secret.json').write_text(userdata.get('CLIENT_SECRET'))

  # Use `--no-browser` in colab
  !gcloud auth application-default login --no-browser --client-id-file client_secret.json --scopes='https://www.googleapis.com/auth/cloud-platform,https://www.googleapis.com/auth/generative-language.tuning'
else:
  !gcloud auth application-default login --client-id-file client_secret.json --scopes='https://www.googleapis.com/auth/cloud-platform,https://www.googleapis.com/auth/generative-language.tuning'

Install the client library

pip install -q google-generativeai

Import libraries

import google.generativeai as genai

genai.configure(transport='grpc')

You can check you existing tuned models with the genai.list_tuned_model method.

for i, m in zip(range(5), genai.list_tuned_models()):
  print(m.name)
tunedModels/my-model-8527
tunedModels/my-model-7092
tunedModels/my-model-2778
tunedModels/my-model-1298
tunedModels/my-model-3883

Create tuned model

To create a tuned model, you need to pass your dataset to the model in the genai.create_tuned_model method. You can do this be directly defining the input and output values in the call or importing from a file into a dataframe to pass to the method.

For this example, you will tune a model to generate the next number in the sequence. For example, if the input is 1, the model should output 2. If the input is one hundred, the output should be one hundred one.

base_model = [
    m for m in genai.list_models()
    if "createTunedModel" in m.supported_generation_methods][0]
base_model
Model(name='models/gemini-1.0-pro-001',
      base_model_id='',
      version='001',
      display_name='Gemini 1.0 Pro',
      description=('The best model for scaling across a wide range of tasks. This is a stable '
                   'model that supports tuning.'),
      input_token_limit=30720,
      output_token_limit=2048,
      supported_generation_methods=['generateContent', 'countTokens', 'createTunedModel'],
      temperature=0.9,
      top_p=1.0,
      top_k=1)
import random

name = f'generate-num-{random.randint(0,10000)}'
operation = genai.create_tuned_model(
    # You can use a tuned model here too. Set `source_model="tunedModels/..."`
    source_model=base_model.name,
    training_data=[
        {
             'text_input': '1',
             'output': '2',
        },{
             'text_input': '3',
             'output': '4',
        },{
             'text_input': '-3',
             'output': '-2',
        },{
             'text_input': 'twenty two',
             'output': 'twenty three',
        },{
             'text_input': 'two hundred',
             'output': 'two hundred one',
        },{
             'text_input': 'ninety nine',
             'output': 'one hundred',
        },{
             'text_input': '8',
             'output': '9',
        },{
             'text_input': '-98',
             'output': '-97',
        },{
             'text_input': '1,000',
             'output': '1,001',
        },{
             'text_input': '10,100,000',
             'output': '10,100,001',
        },{
             'text_input': 'thirteen',
             'output': 'fourteen',
        },{
             'text_input': 'eighty',
             'output': 'eighty one',
        },{
             'text_input': 'one',
             'output': 'two',
        },{
             'text_input': 'three',
             'output': 'four',
        },{
             'text_input': 'seven',
             'output': 'eight',
        }
    ],
    id = name,
    epoch_count = 100,
    batch_size=4,
    learning_rate=0.001,
)

Your tuned model is immediately added to the list of tuned models, but its status is set to "creating" while the model is tuned.

model = genai.get_tuned_model(f'tunedModels/{name}')

model
TunedModel(name='tunedModels/generate-num-2946',
           source_model='models/gemini-1.0-pro-001',
           base_model='models/gemini-1.0-pro-001',
           display_name='',
           description='',
           temperature=0.9,
           top_p=1.0,
           top_k=1,
           state=<State.CREATING: 1>,
           create_time=datetime.datetime(2024, 2, 21, 20, 4, 16, 448050, tzinfo=datetime.timezone.utc),
           update_time=datetime.datetime(2024, 2, 21, 20, 4, 16, 448050, tzinfo=datetime.timezone.utc),
           tuning_task=TuningTask(start_time=datetime.datetime(2024, 2, 21, 20, 4, 16, 890698, tzinfo=datetime.timezone.utc),
                                  complete_time=None,
                                  snapshots=[],
                                  hyperparameters=Hyperparameters(epoch_count=100,
                                                                  batch_size=4,
                                                                  learning_rate=0.001)))
model.state
<State.CREATING: 1>

Check tuning progress

Use metadata to check the state:

operation.metadata
total_steps: 375
tuned_model: "tunedModels/generate-num-2946"

Wait for the training to finish using operation.result(), or operation.wait_bar()

import time

for status in operation.wait_bar():
  time.sleep(30)
0%|          | 0/375 [00:00<?, ?it/s]

You can cancel your tuning job any time using the cancel() method. Uncomment the line below and run the code cell to cancel your job before it finishes.

# operation.cancel()

Once the tuning is complete, you can view the loss curve from the tuning results. The loss curve shows how much the model's predictions deviate from the ideal outputs.

import pandas as pd
import seaborn as sns

model = operation.result()

snapshots = pd.DataFrame(model.tuning_task.snapshots)

sns.lineplot(data=snapshots, x = 'epoch', y='mean_loss')
<Axes: xlabel='epoch', ylabel='mean_loss'>

png

Evaluate your model

You can use the genai.generate_text method and specify the name of your model to test your model performance.

model = genai.GenerativeModel(model_name=f'tunedModels/{name}')
result = model.generate_content('55')
result.text
'56'
result = model.generate_content('123455')
result.text
'123456'
result = model.generate_content('four')
result.text
'five'
result = model.generate_content('quatre') # French 4
result.text                               # French 5 is "cinq"
'cinq'
result = model.generate_content('III')    # Roman numeral 3
result.text                               # Roman numeral 4 is IV
'IV'
result = model.generate_content('七')  # Japanese 7
result.text                            # Japanese 8 is 八!
'八'

It really seems to have picked up the task despite the limited examples, but "next" is a simple concept, see the tuning guide for more guidance on improving performance.

Update the description

You can update the description of your tuned model any time using the genai.update_tuned_model method.

genai.update_tuned_model(f'tunedModels/{name}', {"description":"This is my model."});
model = genai.get_tuned_model(f'tunedModels/{name}')

model.description
'This is my model.'

Delete the model

You can clean up your tuned model list by deleting models you no longer need. Use the genai.delete_tuned_model method to delete a model. If you canceled any tuning jobs, you may want to delete those as their performance may be unpredictable.

genai.delete_tuned_model(f'tunedModels/{name}')

The model no longer exists:

try:
  m = genai.get_tuned_model(f'tunedModels/{name}')
  print(m)
except Exception as e:
  print(f"{type(e)}: {e}")
<class 'google.api_core.exceptions.NotFound'>: 404 Tuned model tunedModels/generate-num-2946 does not exist.