 Afficher dans l'IA de Google
|
 Exécuter dans Google Colab
|
 Consulter le code source sur GitHub
|
Dans ce tutoriel, vous allez utiliser un exemple d'extraction de données structurées en utilisant l'API Gemini pour extraire des listes de personnages, de relations, de choses et de lieux d'une histoire.
Préparation
pip install -U -q google-generativeai
import pathlib
import textwrap
import google.generativeai as genai
from IPython.display import display
from IPython.display import Markdown
from google.api_core import retry
def to_markdown(text):
text = text.replace('•', ' *')
return Markdown(textwrap.indent(text, '> ', predicate=lambda _: True))
Une fois que vous disposez de la clé API, transmettez-la au SDK. Pour cela, vous avez le choix entre deux méthodes :
- Placez la clé dans la variable d'environnement
GOOGLE_API_KEY(le SDK la récupère automatiquement à partir de là). - Transmettre la clé à
genai.configure(api_key=...)
try:
# Used to securely store your API key
from google.colab import userdata
# Or use `os.getenv('API_KEY')` to fetch an environment variable.
GOOGLE_API_KEY=userdata.get('GOOGLE_API_KEY')
except ImportError:
import os
GOOGLE_API_KEY = os.environ['GOOGLE_API_KEY']
genai.configure(api_key=GOOGLE_API_KEY)
L'exemple de tâche
Pour ce tutoriel, vous allez extraire des entités à partir d'histoires en langage naturel. À titre d'exemple, vous trouverez ci-dessous une histoire écrite par Gemini.
new_story = False
if new_story:
model = genai.GenerativeModel(model_name='models/gemini-1.5-pro-latest')
response = model.generate_content("""
Write a long story about a girl with magic backpack, her family, and at
least one other charater. Make sure everyone has names. Don't forget to
describe the contents of the backpack, and where everyone and everything
starts and ends up.""", request_options={'retry': retry.Retry()})
story = response.text
print(response.candidates[0].citation_metadata)
else:
story = """In the quaint town of Willow Creek, nestled amidst rolling hills and whispering willows, resided a young girl named Anya. As she stepped out of the creaky wooden door of her modest cottage, her heart skipped a beat with excitement and anticipation. Today was her first day of school, and she couldn't wait to show off her prized possession - a magical backpack.\n\nHanded down to her from her grandmother, the backpack was no ordinary satchel. Its soft, emerald-green fabric shimmered with an ethereal glow, and its leather straps held secrets that only Anya knew. Within its capacious interior lay an enchanted world, filled with wonders that would ignite her imagination and change her life forever.\n\nAnya's parents, kind-hearted Elise and wise-bearded Edward, bid her farewell with warm embraces. "Remember, my dear," whispered her mother, "use your magic wisely and for good." Her father added, "Always seek knowledge, and let the backpack be your trusted companion."\n\nWith a skip in her step, Anya set off towards the town's only schoolhouse. On her way, she passed her best friend, Samuel, a curious and adventurous boy with a mischievous grin. "Hey, Anya," he called out. "Can I see your backpack?"\n\nAnya hesitated for a moment before unzipping the flap and revealing its contents. Samuel's eyes widened in amazement as he peered inside. There, nestled amidst pencils and notebooks, were a shimmering sword, a book of ancient spells, a tiny compass that always pointed north, and a magical key that could open any lock.\n\nTogether, they marveled at the backpack's wonders, promising to keep its secrets safe. As they approached the schoolhouse, Anya noticed a group of older children huddled together, their faces etched with fear. Curiosity getting the better of her, she cautiously approached.\n\n"What's wrong?" she asked.\n\nA tall, lanky boy stepped forward. "There's a monster in the forest," he stammered. "It's been terrorizing the town, attacking animals and even people."\n\nAnya's heart sank. The town of Willow Creek was small and peaceful, and the thought of a monster brought a shiver down her spine. She knew she had to do something to protect her family and friends.\n\nWithout a moment's hesitation, Anya opened her backpack and retrieved the shimmering sword. With a determined gleam in her eye, she turned to her terrified peers. "Don't worry," she said, her voice steady. "I'll take care of it."\n\nWith Samuel close behind her, Anya ventured into the shadowy depths of the forest. The trees seemed to whisper secrets as she passed, and the undergrowth rustled with unseen creatures. As they walked deeper into the forest, the air grew heavy and the ground beneath their feet trembled.\n\nSuddenly, they came to a clearing, and there before their eyes was the monster - a massive beast with sharp teeth, glowing red eyes, and claws that could crush a human with ease. The creature roared, a thunderous sound that shook the forest to its core.\n\nFear surged through Anya, but she refused to let it consume her. She drew the sword from its sheath and charged towards the monster. The blade shimmered in the sunlight, and as it struck the beast's hide, a blinding light erupted, enveloping everything in its radiance.\n\nWhen the light faded, the monster was gone, and in its place was a pile of shattered crystals. Anya had defeated the creature with the magic of her backpack, proving that even the smallest of objects could hold the greatest of powers.\n\nAs she and Samuel returned to the town, they were greeted as heroes. The people of Willow Creek rejoiced, and the legend of Anya, the girl with the magic backpack, was passed down through generations. And so, Anya continued her adventures, using the backpack's wonders to make the world a better place, one magical step at a time."""
to_markdown(story)
Une jeune fille prénommée Anya vivait dans la ville pittoresque de Willow Creek, nichée au milieu de collines et de saules chuchotant. Alors qu'elle sortait de la porte en bois grinçante de son petit cottage, son cœur s'emballait avec enthousiasme et impatience. C'était aujourd'hui son premier jour d'école et elle était impatiente de montrer son bien précieux : un sac à dos magique.
Ce sac à dos lui a été transmis par sa grand-mère, mais ce n'était pas une sacoche ordinaire. Son tissu doux vert émeraude scintillait d'une lueur éthérée, et ses bracelets en cuir retenaient des secrets que seule Anya connaissait. Son intérieur vaste renferme un monde enchanté, rempli de merveilles qui allaient stimuler son imagination et changer sa vie à jamais.
Les parents d'Anya, Elise bienveillante et le sage barbe d'Edward, lui disent au revoir en l'embrassant chaleureusement. "Souvenez-vous, ma chérie," murmura sa mère, "utilise ta magie à bon escient et pour le bien." Son père ajoute : "Cherchez toujours la connaissance et laissez le sac à dos être votre compagnon de confiance".
Alors qu'elle sautait le pas, Anya s'est dirigée vers la seule école de la ville. Sur son chemin, elle est passée devant son meilleur ami, Samuel, un garçon curieux et aventureux au sourire malicieux. "Salut Anya", a-t-il crié. "Puis-je voir votre sac à dos ?"
Anya hésite un instant avant de décompresser le rabat et d'en révéler le contenu. Les yeux de Samuel s'élargissent, stupéfaits, alors qu'il regarde à l'intérieur. Là, nichée au milieu de crayons et de carnets, il y avait une épée scintillante, un livre de sortilèges anciens, une petite boussole toujours orientée vers le nord et une clé magique pouvant ouvrir n'importe quelle serrure.
Ensemble, ils se sont émerveillés des merveilles de ce sac à dos, promettant de protéger ses secrets. Alors qu'ils s'approchaient de l'école, Anya a remarqué qu'un groupe d'enfants plus âgés s'unissaient les uns contre les autres, le visage gravé de peur. La curiosité s'avançant, elle s'en empara prudemment.
"Qu'est-ce qui ne va pas ?" demande-t-elle.
Un grand garçon déformé s'est présenté en avant. "Il y a un monstre dans la forêt", balbutie-t-il. "Cela a terrorisé la ville, attaqué des animaux et même des gens."
Le cœur d'Anya s'est chamboulé. La ville de Willow Creek était petite et paisible, et l'idée d'un monstre lui faisait frissonner la tête. Elle savait qu'elle devait faire quelque chose pour protéger sa famille et ses amis.
Sans hésitation, Anya ouvrit son sac à dos et récupère l'épée scintillante. Avec un éclat déterminé dans les yeux, elle s'est tournée vers ses pairs terrifiés. "Ne t'inquiète pas", dit-elle d'une voix stabilisée. « Je m'en occuperai. »
Avec Samuel à proximité, Anya s'est aventurée dans les profondeurs sombres de la forêt. Les arbres semblaient chuchoter des secrets en passant, et les broussailles brouillaient de créatures inconnues. Alors qu'ils marchaient plus profondément dans la forêt, l'air s'est envahi et le sol sous leurs pieds trembla.
Soudain, ils sont arrivés dans une clairière, et devant leurs yeux se trouvait le monstre : une bête gigantesque aux dents acérées, aux yeux rouges brillants et aux griffes qui pouvaient facilement écraser un humain. La créature rugira, un bruit de tonnerre qui secoua la forêt au cœur de l'action.
La peur régnait à travers Anya, mais elle refusait de la laisser la consommer. Elle a retiré l'épée de son fourreau et se dirige vers le monstre. La lame scintillait à la lumière du soleil. Alors qu'elle frappait la peau de la bête, une lumière éblouissante s'est enchaînée et enveloppait tout dans son éclat.
Lorsque la lumière s'est estompée, le monstre a disparu, et un amas de cristaux brisé s'est retrouvé à sa place. Anya avait vaincu la créature grâce à la magie de son sac à dos, prouvant que même les plus petits objets pouvaient contenir les plus grandes forces.
À leur retour en ville, Samuel et elle sont accueillis comme des héros. Les habitants de Willow Creek se sont réjouis et la légende d'Anya, la fille au sac à dos magique, est transmise de génération en génération. Ainsi, Anya a poursuivi ses aventures, utilisant les merveilles de son sac à dos pour rendre le monde meilleur, pas à pas.
Utiliser Natural Language
Les grands modèles de langage sont des outils multitâches puissants. Souvent, il suffit de demander à Gemini ce que vous voulez, et il vous suffit.
L'API Gemini n'a pas de mode JSON. Vous devez donc tenir compte de certains points lorsque vous générez des structures de données de cette manière:
- Il arrive que l'analyse échoue.
- Le schéma ne peut pas être strictement appliqué.
Vous résoudrez ces problèmes dans la section suivante. Tout d'abord, essayez une requête simple en langage naturel avec le schéma écrit sous forme de texte. Cela n'a pas été optimisé:
model = model = model = genai.GenerativeModel(
model_name='models/gemini-1.5-pro-latest')
response = model.generate_content(
textwrap.dedent("""\
Please return JSON describing the the people, places, things and relationships from this story using the following schema:
{"people": list[PERSON], "places":list[PLACE], "things":list[THING], "relationships": list[RELATIONSHIP]}
PERSON = {"name": str, "description": str, "start_place_name": str, "end_place_name": str}
PLACE = {"name": str, "description": str}
THING = {"name": str, "description": str, "start_place_name": str, "end_place_name": str}
RELATIONSHIP = {"person_1_name": str, "person_2_name": str, "relationship": str}
All fields are required.
Important: Only return a single piece of valid JSON text.
Here is the story:
""") + story,
generation_config={'response_mime_type':'application/json'}
)
response.text
'{"people": [\n {\n "name": "Anya",\n "description": "A young girl who lives in the town of Willow Creek with her parents, Elise and Edward. She possesses a magical backpack that was handed down to her from her grandmother.",\n "start_place_name": "Willow Creek",\n "end_place_name": "Willow Creek"\n },\n {\n "name": "Elise",\n "description": "Anya\'s kind-hearted mother",\n "start_place_name": "Willow Creek",\n "end_place_name": "Willow Creek"\n },\n {\n "name": "Edward",\n "description": "Anya\'s wise-bearded father",\n "start_place_name": "Willow Creek",\n "end_place_name": "Willow Creek"\n },\n {\n "name": "Samuel",\n "description": "Anya\'s best friend, a curious and adventurous boy with a mischievous grin.",\n "start_place_name": "Willow Creek",\n "end_place_name": "Willow Creek"\n },\n {\n "name": "Monster",\n "description": "A massive beast with sharp teeth, glowing red eyes, and claws that could crush a human with ease.",\n "start_place_name": "Forest",\n "end_place_name": "Forest"\n }\n], "places": [\n {\n "name": "Willow Creek",\n "description": "A quaint town nestled amidst rolling hills and whispering willows."\n },\n {\n "name": "Forest",\n "description": "A shadowy place with rustling undergrowth and whispering trees."\n },\n {\n "name": "Schoolhouse",\n "description": "The only school in the town of Willow Creek."\n },\n {\n "name": "Anya\'s home",\n "description": "A modest cottage with a creaky wooden door."\n }\n], "things": [\n {\n "name": "Magic backpack",\n "description": "A magical backpack that was handed down to Anya from her grandmother. Its soft, emerald-green fabric shimmered with an ethereal glow, and its leather straps held secrets that only Anya knew.",\n "start_place_name": "Anya\'s home",\n "end_place_name": "Forest"\n },\n {\n "name": "Shimmering sword",\n "description": "A sword that shimmered in the sunlight and could strike with blinding light.",\n "start_place_name": "Magic backpack",\n "end_place_name": "Forest"\n },\n {\n "name": "Book of ancient spells",\n "description": "A book that contained ancient spells.",\n "start_place_name": "Magic backpack",\n "end_place_name": "Forest"\n },\n {\n "name": "Tiny compass",\n "description": "A compass that always pointed north.",\n "start_place_name": "Magic backpack",\n "end_place_name": "Forest"\n },\n {\n "name": "Magical key",\n "description": "A key that could open any lock.",\n "start_place_name": "Magic backpack",\n "end_place_name": "Forest"\n },\n {\n "name": "Shattered crystals",\n "description": "The remains of the monster after it was defeated by Anya\'s magic backpack.",\n "start_place_name": "Forest",\n "end_place_name": "Forest"\n }\n], "relationships": [\n {\n "person_1_name": "Anya",\n "person_2_name": "Elise",\n "relationship": "mother-daughter"\n },\n {\n "person_1_name": "Anya",\n "person_2_name": "Edward",\n "relationship": "father-daughter"\n },\n {\n "person_1_name": "Anya",\n "person_2_name": "Samuel",\n "relationship": "best friends"\n }\n]}'
Cela a renvoyé une chaîne JSON. Essayez de l'analyser:
import json
print(json.dumps(json.loads(response.text), indent=4))
{
"people": [
{
"name": "Anya",
"description": "A young girl who lives in the town of Willow Creek with her parents, Elise and Edward. She possesses a magical backpack that was handed down to her from her grandmother.",
"start_place_name": "Willow Creek",
"end_place_name": "Willow Creek"
},
{
"name": "Elise",
"description": "Anya's kind-hearted mother",
"start_place_name": "Willow Creek",
"end_place_name": "Willow Creek"
},
{
"name": "Edward",
"description": "Anya's wise-bearded father",
"start_place_name": "Willow Creek",
"end_place_name": "Willow Creek"
},
{
"name": "Samuel",
"description": "Anya's best friend, a curious and adventurous boy with a mischievous grin.",
"start_place_name": "Willow Creek",
"end_place_name": "Willow Creek"
},
{
"name": "Monster",
"description": "A massive beast with sharp teeth, glowing red eyes, and claws that could crush a human with ease.",
"start_place_name": "Forest",
"end_place_name": "Forest"
}
],
"places": [
{
"name": "Willow Creek",
"description": "A quaint town nestled amidst rolling hills and whispering willows."
},
{
"name": "Forest",
"description": "A shadowy place with rustling undergrowth and whispering trees."
},
{
"name": "Schoolhouse",
"description": "The only school in the town of Willow Creek."
},
{
"name": "Anya's home",
"description": "A modest cottage with a creaky wooden door."
}
],
"things": [
{
"name": "Magic backpack",
"description": "A magical backpack that was handed down to Anya from her grandmother. Its soft, emerald-green fabric shimmered with an ethereal glow, and its leather straps held secrets that only Anya knew.",
"start_place_name": "Anya's home",
"end_place_name": "Forest"
},
{
"name": "Shimmering sword",
"description": "A sword that shimmered in the sunlight and could strike with blinding light.",
"start_place_name": "Magic backpack",
"end_place_name": "Forest"
},
{
"name": "Book of ancient spells",
"description": "A book that contained ancient spells.",
"start_place_name": "Magic backpack",
"end_place_name": "Forest"
},
{
"name": "Tiny compass",
"description": "A compass that always pointed north.",
"start_place_name": "Magic backpack",
"end_place_name": "Forest"
},
{
"name": "Magical key",
"description": "A key that could open any lock.",
"start_place_name": "Magic backpack",
"end_place_name": "Forest"
},
{
"name": "Shattered crystals",
"description": "The remains of the monster after it was defeated by Anya's magic backpack.",
"start_place_name": "Forest",
"end_place_name": "Forest"
}
],
"relationships": [
{
"person_1_name": "Anya",
"person_2_name": "Elise",
"relationship": "mother-daughter"
},
{
"person_1_name": "Anya",
"person_2_name": "Edward",
"relationship": "father-daughter"
},
{
"person_1_name": "Anya",
"person_2_name": "Samuel",
"relationship": "best friends"
}
]
}
Cette méthode est relativement simple et fonctionne souvent, mais vous pouvez éventuellement rendre cette méthode plus stricte/robuste en définissant le schéma à l'aide de la fonctionnalité d'appel de fonction de l'API.
Utiliser l'appel de fonction
Si vous n'avez pas encore suivi le tutoriel Principes de base des appels de fonctions, assurez-vous de le faire en premier.
Lorsque la fonction appelle votre fonction et ses paramètres, ces derniers sont décrits à l'API en tant que genai.protos.FunctionDeclaration. Dans les cas de base, le SDK peut compiler le FunctionDeclaration à partir de la fonction et de ses annotations. Vous devrez donc les définir explicitement, pour l'instant.
Définir le schéma
Commencez par définir person en tant qu'objet avec les champs de chaîne name, description, start_place_name et end_place_name.
person = genai.protos.Schema(
type = genai.protos.Type.OBJECT,
properties = {
'name': genai.protos.Schema(type=genai.protos.Type.STRING),
'description': genai.protos.Schema(type=genai.protos.Type.STRING),
'start_place_name': genai.protos.Schema(type=genai.protos.Type.STRING),
'end_place_name': genai.protos.Schema(type=genai.protos.Type.STRING)
},
required=['name', 'description', 'start_place_name', 'end_place_name']
)
Définissez ensuite des personnes en tant que ARRAY d'objets person:
people = genai.protos.Schema(
type=genai.protos.Type.ARRAY,
items=person
)
Ensuite, répétez l'opération pour chacune des entités que vous essayez d'extraire:
place = genai.protos.Schema(
type = genai.protos.Type.OBJECT,
properties = {
'name': genai.protos.Schema(type=genai.protos.Type.STRING),
'description': genai.protos.Schema(type=genai.protos.Type.STRING),
}
)
places = genai.protos.Schema(
type=genai.protos.Type.ARRAY,
items=place
)
thing = genai.protos.Schema(
type = genai.protos.Type.OBJECT,
properties = {
'name': genai.protos.Schema(type=genai.protos.Type.STRING),
'description': genai.protos.Schema(type=genai.protos.Type.STRING),
}
)
things = genai.protos.Schema(
type=genai.protos.Type.ARRAY,
items=thing
)
relationship = genai.protos.Schema(
type = genai.protos.Type.OBJECT,
properties = {
'person_1_name': genai.protos.Schema(type=genai.protos.Type.STRING),
'person_2_name': genai.protos.Schema(type=genai.protos.Type.STRING),
'relationship': genai.protos.Schema(type=genai.protos.Type.STRING),
}
)
relationships = genai.protos.Schema(
type=genai.protos.Type.ARRAY,
items=relationship
)
Créez maintenant FunctionDeclaration:
add_to_database = genai.protos.FunctionDeclaration(
name="add_to_database",
description=textwrap.dedent("""\
Adds entities to the database.
"""),
parameters=genai.protos.Schema(
type=genai.protos.Type.OBJECT,
properties = {
'people': people,
'places': places,
'things': things,
'relationships': relationships
}
)
)
Appeler l'API
Comme vous l'avez vu dans la section Principes de base de l'appel de fonction, vous pouvez désormais transmettre cet élément FunctionDeclaration à l'argument tools du constructeur genai.GenerativeModel (ce dernier accepte également une représentation JSON équivalente de la déclaration de fonction):
model = model = genai.GenerativeModel(
model_name='models/gemini-1.5-pro-latest',
tools = [add_to_database])
Chaque fois que vous appelez l'API, le SDK envoie les outils avec votre requête, et le modèle doit appeler la fonction que vous avez définie:
result = model.generate_content(f"""
Please add the people, places, things, and relationships from this story to the database:
{story}
""",
# Force a function call
tool_config={'function_calling_config':'ANY'})
Il n'y a plus de texte à analyser. Le résultat est une structure de données.
'text' in result.candidates[0].content.parts[0]
False
'function_call' in result.candidates[0].content.parts[0]
True
fc = result.candidates[0].content.parts[0].function_call
print(type(fc))
<class 'google.ai.generativelanguage_v1beta.types.content.FunctionCall'>
La classe genai.protos.FunctionCall étant basée sur Google Protocol Buffers, convertissez-la en objet JSON plus familier:
print(json.dumps(type(fc).to_dict(fc), indent=4))
{
"name": "add_to_database",
"args": {
"things": [
{
"name": "Magical Backpack",
"description": "Anya's prized possession, the Magical Backpack, is no ordinary satchel. Its soft, emerald-green fabric shimmers with an ethereal glow, and its leather straps have secrets that only Anya knows. Within its capacious interior lay an enchanted world, filled with wonders that would ignite her imagination and change her life forever."
},
{
"name": "Shimmering Sword",
"description": "Among the wonders in Anya's Magical Backpack, lies a shimmering sword. With a determined gleam in her eye, she retrieved the shimmering sword and charged towards the monster."
},
{
"description": "Residing within the Magical Backpack, the Book of Ancient Spells holds secrets untold.",
"name": "Book of Ancient Spells"
},
{
"description": "Tucked away in the Magical Backpack is a tiny compass that always points north.",
"name": "Tiny Compass that Always Points North"
},
{
"description": "Hidden within the Magical Backpack is a magical key that can open any lock.",
"name": "Magical Key that Can Open Any Lock"
}
],
"relationships": [
{
"relationship": "Mother-Daughter",
"person_1_name": "Anya",
"person_2_name": "Elise"
},
{
"person_2_name": "Edward",
"relationship": "Father-Daughter",
"person_1_name": "Anya"
},
{
"person_2_name": "Samuel",
"person_1_name": "Anya",
"relationship": "Best Friends"
}
],
"people": [
{
"name": "Anya",
"description": "Anya, the main character of the story, is a young girl with a magical backpack.",
"start_place_name": "Willow Creek",
"end_place_name": "Unknown"
},
{
"name": "Elise",
"description": "Anya's mother, Elise is a kind-hearted woman.",
"end_place_name": "Unknown",
"start_place_name": "Willow Creek"
},
{
"start_place_name": "Willow Creek",
"end_place_name": "Unknown",
"name": "Edward",
"description": "Anya's father, Edward is a wise-bearded man."
},
{
"end_place_name": "Unknown",
"start_place_name": "Willow Creek",
"description": "Anya's best friend, Samuel is a curious and adventurous boy with a mischievous grin.",
"name": "Samuel"
}
],
"places": [
{
"description": "The quaint town of Willow Creek is nestled amidst rolling hills and whispering willows.",
"name": "Willow Creek"
},
{
"description": "The town's only schoolhouse.",
"name": "Schoolhouse"
},
{
"description": "A shadowy place filled with secrets and dangers, the Forest is home to a terrifying monster.",
"name": "Forest"
}
]
}
}
Conclusion
Bien que l'API puisse gérer les problèmes d'extraction de données structurées avec l'entrée et la sortie de texte pur, l'utilisation de l'appel de fonction est probablement plus fiable, car elle vous permet de définir un schéma strict et élimine une étape d'analyse potentiellement source d'erreurs.