PalM API: شروع سریع متنی با پایتون

مشاهده در ai.google.dev یک نوت بوک کولب را امتحان کنید مشاهده نوت بوک در GitHub

در این نوت بوک، یاد خواهید گرفت که چگونه با Palm API شروع کنید، که به شما امکان دسترسی به جدیدترین مدل های زبان بزرگ گوگل را می دهد. در اینجا، نحوه استفاده از ویژگی های تولید متن PalM API را یاد خواهید گرفت.

راه اندازی

ابتدا کتابخانه PALM API Python را دانلود و نصب کنید.

pip install -q google-generativeai
import pprint
import google.generativeai as palm

یک کلید API بگیرید

برای شروع، باید یک کلید API ایجاد کنید .

palm.configure(api_key='YOUR_API_KEY')

تولید متن

از تابع palm.list_models برای پیدا کردن مدل های موجود استفاده کنید:

models = [m for m in palm.list_models() if 'generateText' in m.supported_generation_methods]
model = models[0].name
print(model)

models/text-bison-001

برای تولید متن از روش palm.generate_text استفاده کنید:

prompt = """
You are an expert at solving word problems.

Solve the following problem:

I have three houses, each with three cats.
each cat owns 4 mittens, and a hat. Each mitten was
knit from 7m of yarn, each hat from 4m.
How much yarn was needed to make all the items?

Think about it step by step, and show your work.
"""

completion = palm.generate_text(
    model=model,
    prompt=prompt,
    temperature=0,
    # The maximum length of the response
    max_output_tokens=800,
)

print(completion.result)

There are 3 houses * 3 cats / house = 9 cats. So, 9 cats * 4 mittens / cat = 36 mittens were made. Also, 9 cats * 1 hat / cat = 9 hats were made. So, 36 mittens * 7m / mitten = 252m of yarn was used for the mittens. Also, 9 hats * 4m / hat = 36m of yarn was used for the hats. In total, 252m + 36m = 288m of yarn was used.
Thus, the answer is 288.

گزینه های بیشتر

تابع palm.generate_text چند آرگومان دیگر نیز دارد که قابل ذکر است.

توالی ها را متوقف کنید

از آرگومان stop_sequences برای توقف زودهنگام تولید استفاده کنید.

به عنوان مثال، LLM ها اغلب در محاسبات اشتباه می کنند. می توانید با قرار دادن معادلات در تگ <calc> از مدل بخواهید "از ماشین حساب استفاده کند".

مدل را در تگ بسته شدن متوقف کنید تا بتوانید دستور را ویرایش کنید:

calc_prompt = f"""
Please solve the following problem.

{prompt}

----------------

Important: Use the calculator for each step.
Don't do the arithmetic in your head. 

To use the calculator wrap an equation in <calc> tags like this: 

<calc> 3 cats * 2 hats/cat </calc> = 6

----------------

"""

equation=None
while equation is None:
    completion = palm.generate_text(
        model=model,
        prompt=calc_prompt,
        stop_sequences=['</calc>'],
        # The maximum length of the response
        max_output_tokens=800,
    )

    try:
        response, equation = completion.result.split('<calc>', maxsplit=1)
    except Exception:
        continue

print(response)

Chain-of-thought:

There are three houses, and each house has three cats, so there are 3 houses * 3 cats / house = 9 cats. Each cat has 4 mittens, so the cats need 9 cats * 4 mittens / cat = 36 mittens. Each mitten takes 7m of yarn, so 36 mittens * 7m / mitten = 252m of yarn. Each cat has a hat, and each hat takes 4m of yarn, so 9 cats * 4m / cat = 36m of yarn. So, in total, 36m + 252m = 288m of yarn were needed.

The answer should be

print(equation)

9 cats * 4 mittens / cat

از آنجا می توانید نتیجه را محاسبه کنید و یک دستور جدید برای ادامه مدل جمع آوری کنید. برای اجرای کامل کار ، مثال ماشین حساب متنی را ببینید.

نامزدها

به طور معمول، درجاتی از تصادفی بودن متن تولید شده توسط LLM وجود دارد. (در مورد چرایی در پرایمر LLM بیشتر بخوانید). این بدان معناست که وقتی بیش از یک بار با ورودی یکسان با API تماس می گیرید، ممکن است پاسخ های متفاوتی دریافت کنید. می توانید از این ویژگی به نفع خود برای دریافت پاسخ های مدل جایگزین استفاده کنید.

آرگومان temperature واریانس پاسخ ها را کنترل می کند. شی palm.Model مقدار پیش فرض temperature و سایر پارامترها را می دهد. 

models[0]

Model(name='models/text-bison-001', base_model_id='', version='001', display_name='Text Bison', description='Model targeted for text generation.', input_token_limit=8196, output_token_limit=1024, supported_generation_methods=['generateText'], temperature=0.7, top_p=0.95, top_k=40)

آرگومان candidate_count تعداد پاسخ های برگشتی را کنترل می کند: 

completion = palm.generate_text(
    model=model,
    prompt=prompt,
    # The number of candidates to return
    candidate_count=8,
    # Set the temperature to 1.0 for more variety of responses.
    temperature=1.0,
    max_output_tokens=800,
)

print(completion.result)

In each house there are 3 cats * 4 mittens / cat = 12 mittens. In total there are 3 houses * 12 mittens / house = 36 mittens. In total there are 36 mittens * 7m / mitten = 252m of yarn for the mittens. In total there are 3 houses * 3 cats / house * 1 hat / cat = 9 hats. In total there are 9 hats * 4m / hat = 36m of yarn for the hats. In total there are 36m yarn for the hats + 252m yarn for the mittens = 288m of yarn.
The answer: 288.

هنگامی که چندین نامزد درخواست می‌کنید، ویژگی Completion.result همچنان حاوی اولین مورد است. ویژگی Completion.candidates شامل همه آنها است: 

import pprint
pprint.pprint(completion.candidates)

[{'output': 'In each house there are 3 cats * 4 mittens / cat = 12 mittens. In '
            'total there are 3 houses * 12 mittens / house = 36 mittens. In '
            'total there are 36 mittens * 7m / mitten = 252m of yarn for the '
            'mittens. In total there are 3 houses * 3 cats / house * 1 hat / '
            'cat = 9 hats. In total there are 9 hats * 4m / hat = 36m of yarn '
            'for the hats. In total there are 36m yarn for the hats + 252m '
            'yarn for the mittens = 288m of yarn.\n'
            'The answer: 288.',
  'safety_ratings': [{'category': <HarmCategory.HARM_CATEGORY_DEROGATORY: 1>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_TOXICITY: 2>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_VIOLENCE: 3>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_SEXUAL: 4>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_MEDICAL: 5>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_DANGEROUS: 6>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>}]},
 {'output': 'Each house has 3 cats, so each house needs 3 * 4 = 12 mittens. '
            "With three houses, that's 3 * 12 = 36 mittens. And each house "
            'needs 3 * 1 = 3 hats. So in total, we need 3 hats + 36 mittens = '
            '39 items. Each mitten needs 7 meters of yarn, so 39 mittens need '
            '39 * 7 = 273 meters of yarn. Each hat needs 4 meters of yarn, and '
            "we need 3 hats, so that's 4 * 3 = 12 meters of yarn. So in total, "
            'we needed 12 + 273 = 285 meters of yarn.\n'
            'Thus, the answer is 285.',
  'safety_ratings': [{'category': <HarmCategory.HARM_CATEGORY_DEROGATORY: 1>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_TOXICITY: 2>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_VIOLENCE: 3>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_SEXUAL: 4>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_MEDICAL: 5>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_DANGEROUS: 6>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>}]},
 {'output': 'There are 3 houses * 3 cats / house = 9 cats. There are 9 cats * '
            '4 mittens / cat = 36 mittens. There are 9 cats * 1 hat / cat = 9 '
            'hats. The total amount of yarn for the mittens is 36 mittens * 7m '
            '/ mitten = 252m. The total amount of yarn for the hats is 9 hats '
            '* 4m / hat = 36m. The total amount of yarn is 252m + 36m = 288m.\n'
            'Thus, the answer is 288.',
  'safety_ratings': [{'category': <HarmCategory.HARM_CATEGORY_DEROGATORY: 1>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_TOXICITY: 2>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_VIOLENCE: 3>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_SEXUAL: 4>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_MEDICAL: 5>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_DANGEROUS: 6>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>}]},
 {'output': 'There are 3 houses * 3 cats / house = 9 cats. Each cat has 4 '
            'mittens + 1 hat = 5 items. So the total number of items is 9 cats '
            '* 5 items / cat = 45 items. Thus, 45 items * 7m / item = 315m of '
            'yarn was needed.\n'
            'Thus, the answer is 315.',
  'safety_ratings': [{'category': <HarmCategory.HARM_CATEGORY_DEROGATORY: 1>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_TOXICITY: 2>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_VIOLENCE: 3>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_SEXUAL: 4>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_MEDICAL: 5>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_DANGEROUS: 6>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>}]},
 {'output': 'Chain-of-thought:\n'
            'There are 3 houses * 3 cats / house = 9 cats.\n'
            'The cats need 9 cats * 4 mittens / cat = 36 mittens.\n'
            'The cats need 9 cats * 1 hat / cat = 9 hats.\n'
            'The mittens need 36 mittens * 7m / mitten = 252m of yarn.\n'
            'The hats need 9 hats * 4m / hat = 36m of yarn.\n'
            'Therefore, the total amount of yarn needed is 252m + 36m = 288m.\n'
            '\n'
            'The answer should be 288',
  'safety_ratings': [{'category': <HarmCategory.HARM_CATEGORY_DEROGATORY: 1>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_TOXICITY: 2>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_VIOLENCE: 3>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_SEXUAL: 4>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_MEDICAL: 5>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_DANGEROUS: 6>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>}]},
 {'output': 'First find the total number of cats: 3 houses * 3 cats / house = '
            '9 cats. Then multiply that number by the number of mittens per '
            'cat to find the total number of mittens: 9 cats * 4 mittens / cat '
            '= 36 mittens. Then multiply that number by the number of meters '
            'of yarn per mitten to find the total amount of yarn used for '
            'mittens: 36 mittens * 7 meters / mitten = 252 meters. Then do the '
            'same thing for hats: 9 cats * 1 hat / cat = 9 hats. Then multiply '
            'that number by the number of meters of yarn per hat to find the '
            'total amount of yarn used for hats: 9 hats * 4 meters / hat = 36 '
            'meters. Then add the amount of yarn used for mittens and hats to '
            'find the total amount of yarn used: 36 meters + 252 meters = 288 '
            'meters.\n'
            'Thus, the answer is 288.',
  'safety_ratings': [{'category': <HarmCategory.HARM_CATEGORY_DEROGATORY: 1>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_TOXICITY: 2>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_VIOLENCE: 3>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_SEXUAL: 4>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_MEDICAL: 5>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_DANGEROUS: 6>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>}]},
 {'output': 'The total number of cats is 3 houses * 3 cats / house = 9 cats. '
            'So, the total number of mittens is 9 cats * 4 mittens / cat = 36 '
            'mittens. The total number of hats is 9 cats * 1 hat / cat = 9 '
            'hats. The total length of yarn needed to make the mittens is 36 '
            'mittens * 7 m / mitten = 252 m. The total length of yarn needed '
            'to make the hats is 9 hats * 4 m / hat = 36 m. So, the total '
            'length of yarn needed is 252 m + 36 m = 288 m.\n'
            '\n'
            'The answer: 288',
  'safety_ratings': [{'category': <HarmCategory.HARM_CATEGORY_DEROGATORY: 1>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_TOXICITY: 2>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_VIOLENCE: 3>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_SEXUAL: 4>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_MEDICAL: 5>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_DANGEROUS: 6>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>}]},
 {'output': 'There are 3 houses with 3 cats each, so 3 * 3 = 9 cats. Each cat '
            'has 4 mittens and a hat, so 9 cats * 4 mittens / cat + 9 cats * 1 '
            'hat / cat = 36 mittens and 9 hats. Each mitten takes 7m of yarn '
            'and each hat takes 4m of yarn, so the total yarn needed is 36 '
            'mittens * 7m / mitten + 9 hats * 4m / hat = 252m + 36m = 288m.\n'
            'The answer: 288.',
  'safety_ratings': [{'category': <HarmCategory.HARM_CATEGORY_DEROGATORY: 1>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_TOXICITY: 2>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_VIOLENCE: 3>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_SEXUAL: 4>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_MEDICAL: 5>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_DANGEROUS: 6>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>}]}]

بنابراین، از آنجایی که پاسخ این مشکل را می دانید، بررسی نرخ حل آسان است: 

import numpy as np
np.mean(['288' in c['output'] for c in completion.candidates])

0.75
،

مشاهده در ai.google.dev یک نوت بوک کولب را امتحان کنید مشاهده نوت بوک در GitHub

در این نوت بوک، یاد خواهید گرفت که چگونه با Palm API شروع کنید، که به شما امکان دسترسی به جدیدترین مدل های زبان بزرگ گوگل را می دهد. در اینجا، نحوه استفاده از ویژگی های تولید متن PalM API را یاد خواهید گرفت.

راه اندازی

ابتدا کتابخانه PALM API Python را دانلود و نصب کنید.

pip install -q google-generativeai
import pprint
import google.generativeai as palm

یک کلید API بگیرید

برای شروع، باید یک کلید API ایجاد کنید .

palm.configure(api_key='YOUR_API_KEY')

تولید متن

از تابع palm.list_models برای پیدا کردن مدل های موجود استفاده کنید:

models = [m for m in palm.list_models() if 'generateText' in m.supported_generation_methods]
model = models[0].name
print(model)

models/text-bison-001

برای تولید متن از روش palm.generate_text استفاده کنید:

prompt = """
You are an expert at solving word problems.

Solve the following problem:

I have three houses, each with three cats.
each cat owns 4 mittens, and a hat. Each mitten was
knit from 7m of yarn, each hat from 4m.
How much yarn was needed to make all the items?

Think about it step by step, and show your work.
"""

completion = palm.generate_text(
    model=model,
    prompt=prompt,
    temperature=0,
    # The maximum length of the response
    max_output_tokens=800,
)

print(completion.result)

There are 3 houses * 3 cats / house = 9 cats. So, 9 cats * 4 mittens / cat = 36 mittens were made. Also, 9 cats * 1 hat / cat = 9 hats were made. So, 36 mittens * 7m / mitten = 252m of yarn was used for the mittens. Also, 9 hats * 4m / hat = 36m of yarn was used for the hats. In total, 252m + 36m = 288m of yarn was used.
Thus, the answer is 288.

گزینه های بیشتر

تابع palm.generate_text چند آرگومان دیگر نیز دارد که قابل ذکر است.

توالی ها را متوقف کنید

از آرگومان stop_sequences برای توقف زودهنگام تولید استفاده کنید.

به عنوان مثال، LLM ها اغلب در محاسبات اشتباه می کنند. می توانید با قرار دادن معادلات در تگ <calc> از مدل بخواهید "از ماشین حساب استفاده کند".

مدل را در تگ بسته شدن متوقف کنید تا بتوانید دستور را ویرایش کنید:

calc_prompt = f"""
Please solve the following problem.

{prompt}

----------------

Important: Use the calculator for each step.
Don't do the arithmetic in your head. 

To use the calculator wrap an equation in <calc> tags like this: 

<calc> 3 cats * 2 hats/cat </calc> = 6

----------------

"""

equation=None
while equation is None:
    completion = palm.generate_text(
        model=model,
        prompt=calc_prompt,
        stop_sequences=['</calc>'],
        # The maximum length of the response
        max_output_tokens=800,
    )

    try:
        response, equation = completion.result.split('<calc>', maxsplit=1)
    except Exception:
        continue

print(response)

Chain-of-thought:

There are three houses, and each house has three cats, so there are 3 houses * 3 cats / house = 9 cats. Each cat has 4 mittens, so the cats need 9 cats * 4 mittens / cat = 36 mittens. Each mitten takes 7m of yarn, so 36 mittens * 7m / mitten = 252m of yarn. Each cat has a hat, and each hat takes 4m of yarn, so 9 cats * 4m / cat = 36m of yarn. So, in total, 36m + 252m = 288m of yarn were needed.

The answer should be

print(equation)

9 cats * 4 mittens / cat

از آنجا می توانید نتیجه را محاسبه کنید و یک دستور جدید برای ادامه مدل جمع آوری کنید. برای اجرای کامل کار ، مثال ماشین حساب متنی را ببینید.

نامزدها

به طور معمول، درجاتی از تصادفی بودن متن تولید شده توسط LLM وجود دارد. (در مورد چرایی در پرایمر LLM بیشتر بخوانید). این بدان معناست که وقتی بیش از یک بار با ورودی یکسان با API تماس می گیرید، ممکن است پاسخ های متفاوتی دریافت کنید. می توانید از این ویژگی به نفع خود برای دریافت پاسخ های مدل جایگزین استفاده کنید.

آرگومان temperature واریانس پاسخ ها را کنترل می کند. شی palm.Model مقدار پیش فرض temperature و سایر پارامترها را می دهد. 

models[0]

Model(name='models/text-bison-001', base_model_id='', version='001', display_name='Text Bison', description='Model targeted for text generation.', input_token_limit=8196, output_token_limit=1024, supported_generation_methods=['generateText'], temperature=0.7, top_p=0.95, top_k=40)

آرگومان candidate_count تعداد پاسخ های برگشتی را کنترل می کند: 

completion = palm.generate_text(
    model=model,
    prompt=prompt,
    # The number of candidates to return
    candidate_count=8,
    # Set the temperature to 1.0 for more variety of responses.
    temperature=1.0,
    max_output_tokens=800,
)

print(completion.result)

In each house there are 3 cats * 4 mittens / cat = 12 mittens. In total there are 3 houses * 12 mittens / house = 36 mittens. In total there are 36 mittens * 7m / mitten = 252m of yarn for the mittens. In total there are 3 houses * 3 cats / house * 1 hat / cat = 9 hats. In total there are 9 hats * 4m / hat = 36m of yarn for the hats. In total there are 36m yarn for the hats + 252m yarn for the mittens = 288m of yarn.
The answer: 288.

هنگامی که چندین نامزد درخواست می‌کنید، ویژگی Completion.result همچنان حاوی اولین مورد است. ویژگی Completion.candidates شامل همه آنها است: 

import pprint
pprint.pprint(completion.candidates)

[{'output': 'In each house there are 3 cats * 4 mittens / cat = 12 mittens. In '
            'total there are 3 houses * 12 mittens / house = 36 mittens. In '
            'total there are 36 mittens * 7m / mitten = 252m of yarn for the '
            'mittens. In total there are 3 houses * 3 cats / house * 1 hat / '
            'cat = 9 hats. In total there are 9 hats * 4m / hat = 36m of yarn '
            'for the hats. In total there are 36m yarn for the hats + 252m '
            'yarn for the mittens = 288m of yarn.\n'
            'The answer: 288.',
  'safety_ratings': [{'category': <HarmCategory.HARM_CATEGORY_DEROGATORY: 1>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_TOXICITY: 2>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_VIOLENCE: 3>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_SEXUAL: 4>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_MEDICAL: 5>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_DANGEROUS: 6>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>}]},
 {'output': 'Each house has 3 cats, so each house needs 3 * 4 = 12 mittens. '
            "With three houses, that's 3 * 12 = 36 mittens. And each house "
            'needs 3 * 1 = 3 hats. So in total, we need 3 hats + 36 mittens = '
            '39 items. Each mitten needs 7 meters of yarn, so 39 mittens need '
            '39 * 7 = 273 meters of yarn. Each hat needs 4 meters of yarn, and '
            "we need 3 hats, so that's 4 * 3 = 12 meters of yarn. So in total, "
            'we needed 12 + 273 = 285 meters of yarn.\n'
            'Thus, the answer is 285.',
  'safety_ratings': [{'category': <HarmCategory.HARM_CATEGORY_DEROGATORY: 1>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_TOXICITY: 2>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_VIOLENCE: 3>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_SEXUAL: 4>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_MEDICAL: 5>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_DANGEROUS: 6>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>}]},
 {'output': 'There are 3 houses * 3 cats / house = 9 cats. There are 9 cats * '
            '4 mittens / cat = 36 mittens. There are 9 cats * 1 hat / cat = 9 '
            'hats. The total amount of yarn for the mittens is 36 mittens * 7m '
            '/ mitten = 252m. The total amount of yarn for the hats is 9 hats '
            '* 4m / hat = 36m. The total amount of yarn is 252m + 36m = 288m.\n'
            'Thus, the answer is 288.',
  'safety_ratings': [{'category': <HarmCategory.HARM_CATEGORY_DEROGATORY: 1>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_TOXICITY: 2>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_VIOLENCE: 3>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_SEXUAL: 4>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_MEDICAL: 5>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_DANGEROUS: 6>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>}]},
 {'output': 'There are 3 houses * 3 cats / house = 9 cats. Each cat has 4 '
            'mittens + 1 hat = 5 items. So the total number of items is 9 cats '
            '* 5 items / cat = 45 items. Thus, 45 items * 7m / item = 315m of '
            'yarn was needed.\n'
            'Thus, the answer is 315.',
  'safety_ratings': [{'category': <HarmCategory.HARM_CATEGORY_DEROGATORY: 1>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_TOXICITY: 2>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_VIOLENCE: 3>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_SEXUAL: 4>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_MEDICAL: 5>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_DANGEROUS: 6>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>}]},
 {'output': 'Chain-of-thought:\n'
            'There are 3 houses * 3 cats / house = 9 cats.\n'
            'The cats need 9 cats * 4 mittens / cat = 36 mittens.\n'
            'The cats need 9 cats * 1 hat / cat = 9 hats.\n'
            'The mittens need 36 mittens * 7m / mitten = 252m of yarn.\n'
            'The hats need 9 hats * 4m / hat = 36m of yarn.\n'
            'Therefore, the total amount of yarn needed is 252m + 36m = 288m.\n'
            '\n'
            'The answer should be 288',
  'safety_ratings': [{'category': <HarmCategory.HARM_CATEGORY_DEROGATORY: 1>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_TOXICITY: 2>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_VIOLENCE: 3>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_SEXUAL: 4>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_MEDICAL: 5>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_DANGEROUS: 6>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>}]},
 {'output': 'First find the total number of cats: 3 houses * 3 cats / house = '
            '9 cats. Then multiply that number by the number of mittens per '
            'cat to find the total number of mittens: 9 cats * 4 mittens / cat '
            '= 36 mittens. Then multiply that number by the number of meters '
            'of yarn per mitten to find the total amount of yarn used for '
            'mittens: 36 mittens * 7 meters / mitten = 252 meters. Then do the '
            'same thing for hats: 9 cats * 1 hat / cat = 9 hats. Then multiply '
            'that number by the number of meters of yarn per hat to find the '
            'total amount of yarn used for hats: 9 hats * 4 meters / hat = 36 '
            'meters. Then add the amount of yarn used for mittens and hats to '
            'find the total amount of yarn used: 36 meters + 252 meters = 288 '
            'meters.\n'
            'Thus, the answer is 288.',
  'safety_ratings': [{'category': <HarmCategory.HARM_CATEGORY_DEROGATORY: 1>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_TOXICITY: 2>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_VIOLENCE: 3>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_SEXUAL: 4>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_MEDICAL: 5>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_DANGEROUS: 6>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>}]},
 {'output': 'The total number of cats is 3 houses * 3 cats / house = 9 cats. '
            'So, the total number of mittens is 9 cats * 4 mittens / cat = 36 '
            'mittens. The total number of hats is 9 cats * 1 hat / cat = 9 '
            'hats. The total length of yarn needed to make the mittens is 36 '
            'mittens * 7 m / mitten = 252 m. The total length of yarn needed '
            'to make the hats is 9 hats * 4 m / hat = 36 m. So, the total '
            'length of yarn needed is 252 m + 36 m = 288 m.\n'
            '\n'
            'The answer: 288',
  'safety_ratings': [{'category': <HarmCategory.HARM_CATEGORY_DEROGATORY: 1>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_TOXICITY: 2>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_VIOLENCE: 3>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_SEXUAL: 4>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_MEDICAL: 5>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_DANGEROUS: 6>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>}]},
 {'output': 'There are 3 houses with 3 cats each, so 3 * 3 = 9 cats. Each cat '
            'has 4 mittens and a hat, so 9 cats * 4 mittens / cat + 9 cats * 1 '
            'hat / cat = 36 mittens and 9 hats. Each mitten takes 7m of yarn '
            'and each hat takes 4m of yarn, so the total yarn needed is 36 '
            'mittens * 7m / mitten + 9 hats * 4m / hat = 252m + 36m = 288m.\n'
            'The answer: 288.',
  'safety_ratings': [{'category': <HarmCategory.HARM_CATEGORY_DEROGATORY: 1>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_TOXICITY: 2>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_VIOLENCE: 3>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_SEXUAL: 4>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_MEDICAL: 5>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>},
                     {'category': <HarmCategory.HARM_CATEGORY_DANGEROUS: 6>,
                      'probability': <HarmProbability.NEGLIGIBLE: 1>}]}]

بنابراین، از آنجایی که پاسخ این مشکل را می دانید، بررسی نرخ حل آسان است: 

import numpy as np
np.mean(['288' in c['output'] for c in completion.candidates])

0.75