![]() |
![]() |
![]() |
In this notebook, you'll learn how to get started with the tuning service using the Python client library for the Gemini API. Here, you'll learn how to tune the text model behind the Gemini API's text generation service.
Before you begin: Set up your project and API key
Before calling the Gemini API, you need to set up your project and configure your API key.
Get and secure your API key
You need an API key to call the Gemini API. If you don't already have one, create a key in Google AI Studio.
It's strongly recommended that you do not check an API key into your version control system. Instead, you should store your API key in a secrets store (for example, Secret Manager).
Install the SDK package and configure your API key
The Python SDK for the Gemini API is contained in the
google-generativeai
package.
Install the dependency using pip:
pip install -U google-generativeai
Import the package and configure the service with your API key:
import os import google.generativeai as genai genai.configure(api_key=os.environ['API_KEY'])
Setup
Set up authentication
The Gemini API lets you tune models on your own data. Since it's your data and your tuned models this needs stricter access controls than API-Keys can provide.
Before you can run this tutorial, you'll need to setup OAuth for your project.
The easiest way to get setup in Colab is to copy the contents of your
client_secret.json
file into Colab's "Secrets manager" (under the key icon in
the left panel) with the secret name CLIENT_SECRET
.
This gcloud command turns the client_secret.json
file into credentials that
can be used to authenticate with the service.
import os
if 'COLAB_RELEASE_TAG' in os.environ:
from google.colab import userdata
import pathlib
pathlib.Path('client_secret.json').write_text(userdata.get('CLIENT_SECRET'))
# Use `--no-browser` in colab
!gcloud auth application-default login --no-browser --client-id-file client_secret.json --scopes='https://www.googleapis.com/auth/cloud-platform,https://www.googleapis.com/auth/generative-language.tuning'
else:
!gcloud auth application-default login --client-id-file client_secret.json --scopes='https://www.googleapis.com/auth/cloud-platform,https://www.googleapis.com/auth/generative-language.tuning'
Install the client library
pip install -q google-generativeai
Import libraries
import google.generativeai as genai
You can check your existing tuned models with the genai.list_tuned_model
method.
for i, m in zip(range(5), genai.list_tuned_models()):
print(m.name)
tunedModels/my-model-8527 tunedModels/my-model-7092 tunedModels/my-model-2778 tunedModels/my-model-1298 tunedModels/my-model-3883
Create tuned model
To create a tuned model, you need to pass your dataset to the model in the
genai.create_tuned_model
method. You can do this by directly defining the
input and output values in the call or importing from a file into a dataframe to
pass to the method.
For this example, you will tune a model to generate the next number in the
sequence. For example, if the input is 1
, the model should output 2
. If the
input is one hundred
, the output should be one hundred one
.
base_model = [
m for m in genai.list_models()
if "createTunedModel" in m.supported_generation_methods][0]
base_model
Model(name='models/gemini-1.0-pro-001', base_model_id='', version='001', display_name='Gemini 1.0 Pro', description=('The best model for scaling across a wide range of tasks. This is a stable ' 'model that supports tuning.'), input_token_limit=30720, output_token_limit=2048, supported_generation_methods=['generateContent', 'countTokens', 'createTunedModel'], temperature=0.9, top_p=1.0, top_k=1)
import random
name = f'generate-num-{random.randint(0,10000)}'
operation = genai.create_tuned_model(
# You can use a tuned model here too. Set `source_model="tunedModels/..."`
source_model=base_model.name,
training_data=[
{
'text_input': '1',
'output': '2',
},{
'text_input': '3',
'output': '4',
},{
'text_input': '-3',
'output': '-2',
},{
'text_input': 'twenty two',
'output': 'twenty three',
},{
'text_input': 'two hundred',
'output': 'two hundred one',
},{
'text_input': 'ninety nine',
'output': 'one hundred',
},{
'text_input': '8',
'output': '9',
},{
'text_input': '-98',
'output': '-97',
},{
'text_input': '1,000',
'output': '1,001',
},{
'text_input': '10,100,000',
'output': '10,100,001',
},{
'text_input': 'thirteen',
'output': 'fourteen',
},{
'text_input': 'eighty',
'output': 'eighty one',
},{
'text_input': 'one',
'output': 'two',
},{
'text_input': 'three',
'output': 'four',
},{
'text_input': 'seven',
'output': 'eight',
}
],
id = name,
epoch_count = 100,
batch_size=4,
learning_rate=0.001,
)
Your tuned model is immediately added to the list of tuned models, but its status is set to "creating" while the model is tuned.
model = genai.get_tuned_model(f'tunedModels/{name}')
model
TunedModel(name='tunedModels/generate-num-2946', source_model='models/gemini-1.0-pro-001', base_model='models/gemini-1.0-pro-001', display_name='', description='', temperature=0.9, top_p=1.0, top_k=1, state=<State.CREATING: 1>, create_time=datetime.datetime(2024, 2, 21, 20, 4, 16, 448050, tzinfo=datetime.timezone.utc), update_time=datetime.datetime(2024, 2, 21, 20, 4, 16, 448050, tzinfo=datetime.timezone.utc), tuning_task=TuningTask(start_time=datetime.datetime(2024, 2, 21, 20, 4, 16, 890698, tzinfo=datetime.timezone.utc), complete_time=None, snapshots=[], hyperparameters=Hyperparameters(epoch_count=100, batch_size=4, learning_rate=0.001)))
model.state
<State.CREATING: 1>
Check tuning progress
Use metadata
to check the state:
operation.metadata
total_steps: 375 tuned_model: "tunedModels/generate-num-2946"
Wait for the training to finish using operation.result()
, or
operation.wait_bar()
import time
for status in operation.wait_bar():
time.sleep(30)
0%| | 0/375 [00:00<?, ?it/s]
You can cancel your tuning job any time using the cancel()
method. Uncomment
the line below and run the code cell to cancel your job before it finishes.
# operation.cancel()
Once the tuning is complete, you can view the loss curve from the tuning results. The loss curve shows how much the model's predictions deviate from the ideal outputs.
import pandas as pd
import seaborn as sns
model = operation.result()
snapshots = pd.DataFrame(model.tuning_task.snapshots)
sns.lineplot(data=snapshots, x = 'epoch', y='mean_loss')
<Axes: xlabel='epoch', ylabel='mean_loss'>
Evaluate your model
You can use the genai.generate_text
method and specify the name of your model
to test your model performance.
model = genai.GenerativeModel(model_name=f'tunedModels/{name}')
result = model.generate_content('55')
result.text
'56'
result = model.generate_content('123455')
result.text
'123456'
result = model.generate_content('four')
result.text
'five'
result = model.generate_content('quatre') # French 4
result.text # French 5 is "cinq"
'cinq'
result = model.generate_content('III') # Roman numeral 3
result.text # Roman numeral 4 is IV
'IV'
result = model.generate_content('七') # Japanese 7
result.text # Japanese 8 is 八!
'八'
It really seems to have picked up the task despite the limited examples, but "next" is a relatively simple concept, see the tuning guide for more guidance on improving performance.
Update the description
You can update the description of your tuned model any time using the
genai.update_tuned_model
method.
genai.update_tuned_model(f'tunedModels/{name}', {"description":"This is my model."});
model = genai.get_tuned_model(f'tunedModels/{name}')
model.description
'This is my model.'
Delete the model
You can clean up your tuned model list by deleting models you no longer need.
Use the genai.delete_tuned_model
method to delete a model. If you canceled any
tuning jobs, you may want to delete those as their performance may be
unpredictable.
genai.delete_tuned_model(f'tunedModels/{name}')
The model no longer exists:
try:
m = genai.get_tuned_model(f'tunedModels/{name}')
print(m)
except Exception as e:
print(f"{type(e)}: {e}")
<class 'google.api_core.exceptions.NotFound'>: 404 Tuned model tunedModels/generate-num-2946 does not exist.