 View on ai.google.dev
|
 Run in Google Colab
|
 Run in Kaggle
|
 Open in Vertex AI
|
 View source on GitHub
|
FunctionGemma is a specialized version of the Gemma 3 270M model, trained specifically for function calling improvements. It has the same architecture as Gemma, but uses a different chat format and tokenizer.
This guide shows the process of using FunctionGemma within the Hugging Face ecosystem. It covers essential setup steps, such as installing the torch and transformers libraries and loading the model using AutoProcessor and AutoModelForCausalLM. Additionally, the guide explains how to pass tools to the model using either manual JSON schemas or raw Python functions and advises on when to use manual schemas to handle complex custom objects effectively.
Setup
Before starting this tutorial, complete the following steps:
- Get access to FunctionGemma by logging into Hugging Face and selecting Acknowledge license for a FunctionGemma model.
- Generate a Hugging Face Access Token and add it to your Colab environment.
This notebook will run on either CPU or GPU.
Install Python packages
Install the Hugging Face libraries required for running the FunctionGemma model and making requests.
# Install PyTorch & other librariespip install torch# Install the transformers librarypip install transformers
After you have accepted the license, you need a valid Hugging Face Token to access the model.
# Login into Hugging Face Hub
from huggingface_hub import login
login()
Load Model
Use the torch and transformers libraries to create an instance of a processor and model using the AutoProcessor and AutoModelForCausalLM classes as shown in the following code example:
from transformers import AutoProcessor, AutoModelForCausalLM
GEMMA_MODEL_ID = "google/functiongemma-270m-it"
processor = AutoProcessor.from_pretrained(GEMMA_MODEL_ID, device_map="auto")
model = AutoModelForCausalLM.from_pretrained(GEMMA_MODEL_ID, dtype="auto", device_map="auto")
Passing tools
You can pass tools to the model using the apply_chat_template() function via the tools argument. There are two methods for defining these tools:
- JSON schema: You can manually construct a JSON dictionary defining the function name, description, and parameters (including types and required fields).
- Raw Python Functions: You can pass actual Python functions. The system automatically generates the required JSON schema by parsing the function's type hints, arguments, and docstrings. For best results, docstrings should adhere to the Google Python Style Guide.
Below is the example with the JSON schema.
weather_function_schema = {
"type": "function",
"function": {
"name": "get_current_temperature",
"description": "Gets the current temperature for a given location.",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The city name, e.g. San Francisco",
},
},
"required": ["location"],
},
}
}
message = [
# ESSENTIAL SYSTEM PROMPT:
# This line activates the model's function calling logic.
{
"role": "developer", "content": "You are a model that can do function calling with the following functions"
},
{
"role": "user", "content": "What's the temperature in London?"
}
]
inputs = processor.apply_chat_template(message, tools=[weather_function_schema], add_generation_prompt=True, return_dict=True, return_tensors="pt")
out = model.generate(**inputs.to(model.device), pad_token_id=processor.eos_token_id, max_new_tokens=128)
output = processor.decode(out[0][len(inputs["input_ids"][0]):], skip_special_tokens=True)
print(f"Output: {output}")
Output: <start_function_call>call:get_current_temperature{location:<escape>London<escape>}<end_function_call>
message = [
# ESSENTIAL SYSTEM PROMPT:
# This line activates the model's function calling logic.
{"role": "developer", "content": "You are a model that can do function calling with the following functions"},
{"role": "user", "content": prompt},
]
And the same example with the raw Python function.
def get_current_temperature(location: str):
"""
Gets the current temperature for a given location.
Args:
location: The city name, e.g. San Francisco
"""
return "15°C"
message = [
{
"role": "user", "content": "What's the temperature in London?"
}
]
inputs = processor.apply_chat_template(message, tools=[weather_function_schema], add_generation_prompt=True, return_dict=True, return_tensors="pt")
out = model.generate(**inputs.to(model.device), pad_token_id=processor.eos_token_id, max_new_tokens=128)
output = processor.decode(out[0][len(inputs["input_ids"][0]):], skip_special_tokens=True)
print(f"Output: {output}")
Output: <start_function_call>call:get_current_temperature{location:<escape>London<escape>}<end_function_call>
Important Caveat: Automatic vs. Manual Schemas
When relying on automatic conversion from Python functions to JSON schema, the generated output may not always meet specific expectations regarding complex parameters.
If a function uses a custom object (like a Config class) as an argument, the automatic converter may describe it simply as a generic "object" without detailing its internal properties.
In these cases, manually defining the JSON schema is preferred to ensure nested properties (such as theme or font_size within a config object) are explicitly defined for the model.
import json
from transformers.utils import get_json_schema
class Config:
def __init__(self):
self.theme = "light"
self.font_size = 14
def update_config(config: Config):
"""
Updates the configuration of the system.
Args:
config: A Config object
Returns:
True if the configuration was successfully updated, False otherwise.
"""
update_config_schema = {
"type": "function",
"function": {
"name": "update_config",
"description": "Updates the configuration of the system.",
"parameters": {
"type": "object",
"properties": {
"config": {
"type": "object",
"description": "A Config object",
"properties": {"theme": {"type": "string"}, "font_size": {"type": "number"} },
},
},
"required": ["config"],
},
},
}
print(f"--- [Automatic] ---")
print(json.dumps(get_json_schema(update_config), indent=2))
print(f"\n--- [Manual Schemas] ---")
print(json.dumps(update_config_schema, indent=2))
--- [Automatic] ---
{
"type": "function",
"function": {
"name": "update_config",
"description": "Updates the configuration of the system.",
"parameters": {
"type": "object",
"properties": {
"config": {
"type": "object",
"description": "A Config object"
}
},
"required": [
"config"
]
}
}
}
--- [Manual Schemas] ---
{
"type": "function",
"function": {
"name": "update_config",
"description": "Updates the configuration of the system.",
"parameters": {
"type": "object",
"properties": {
"config": {
"type": "object",
"description": "A Config object",
"properties": {
"theme": {
"type": "string"
},
"font_size": {
"type": "number"
}
}
}
},
"required": [
"config"
]
}
}
}
Summary and next steps
Now you understand function calling with FunctionGemma. Key takeaways from this include:
- Defining Tools: You can define tools using two methods: creating a manual JSON schema or passing raw Python functions, where the system parses type hints and docstrings.
- Schema Caveats: While automatic conversion works for simple types, it struggles with complex custom objects. In these cases, manual JSON schema definition is required to ensure nested properties are visible to the model.
Check out the following docs next: