Any responsible approach to applying artificial intelligence (AI) should include
safety policies,
transparency artifacts, and
safeguards, but being responsible with AI means more than
following checklists.
GenAI products are relatively new and the behaviors of an application can vary
more than earlier forms of software. For this reason, you should probe the
models being used to examine examples of the model's behavior, and investigate
surprises.
Prompting is the ubiquitous interface for interacting with GenAI, and
engineering those prompts is as much art as it is science. However, there are
tools that can help you empirically improve prompts for LLMs, such as the
Learning Interpretability Tool (LIT). LIT is an open-source
tool for visually understanding and debugging AI models, that can be used as
a debugger for prompt engineering work. Follow along with the
provided tutorial using the Colab or Codelab.
Figure 1. The LIT's user interface: the Datapoint Editor at the top allows
users to edit their prompts. At the bottom, the LM Salience module allows them
to check saliency results.
Include non-technical teams in model probing and exploration
Interpretability is meant to be a team effort, spanning expertise across
policy, legal, and more. LIT's visual medium and interactive ability to examine
salience and explore examples can help different stakeholders share and
communicate findings. This approach can enables more diversity of perspective in
model exploration, probing, and debugging. Exposing your teammates to these
technical methods can enhance their understanding of how models work. In
addition, a more diverse set of expertise in early model testing can also help
uncover undesirable outcomes that can be improved.
[[["Easy to understand","easyToUnderstand","thumb-up"],["Solved my problem","solvedMyProblem","thumb-up"],["Other","otherUp","thumb-up"]],[["Missing the information I need","missingTheInformationINeed","thumb-down"],["Too complicated / too many steps","tooComplicatedTooManySteps","thumb-down"],["Out of date","outOfDate","thumb-down"],["Samples / code issue","samplesCodeIssue","thumb-down"],["Other","otherDown","thumb-down"]],["Last updated 2024-10-23 UTC."],[],[],null,["# Prompt debugging with LIT\n\n\u003cbr /\u003e\n\nAny responsible approach to applying artificial intelligence (AI) should include\n[safety policies](/responsible/docs/design#define-policies),\n[transparency artifacts](/responsible/docs/design#transparency-artifacts), and\n[safeguards](/responsible/docs/safeguards), but being responsible with AI means more than\nfollowing checklists.\n\nGenAI products are relatively new and the behaviors of an application can vary\nmore than earlier forms of software. For this reason, you should probe the\nmodels being used to examine examples of the model's behavior, and investigate\nsurprises.\n\nPrompting is the ubiquitous interface for interacting with GenAI, and\nengineering those prompts is as much art as it is science. However, there are\ntools that can help you empirically improve prompts for LLMs, such as the\n[Learning Interpretability Tool](https://pair-code.github.io/lit/) (LIT). LIT is an open-source\ntool for visually understanding and debugging AI models, that can be used as\na [debugger for prompt engineering work](https://pair-code.github.io/lit/documentation/components.html#sequence-salience). Follow along with the\n[provided tutorial](https://pair-code.github.io/lit/tutorials/sequence-salience/) using the Colab or Codelab.\n\n**Analyze Gemma Models with LIT**\n\n|---|-------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------|\n| | [Start Codelab](https://codelabs.developers.google.com/codelabs/responsible-ai/lit-gemma) | [Start Google Colab](https://colab.research.google.com/github/google/generative-ai-docs/blob/main/site/en/gemma/docs/lit_gemma.ipynb) |\n\n\u003cbr /\u003e\n\n*Figure 1.* The LIT's user interface: the Datapoint Editor at the top allows\nusers to edit their prompts. At the bottom, the LM Salience module allows them\nto check saliency results.\n\nYou can use LIT on your [local machine](https://pair-code.github.io/lit/documentation/getting_started.html#installation), in\n[Colab](https://colab.research.google.com/github/google/generative-ai-docs/blob/main/site/en/gemma/docs/lit_gemma.ipynb), or on [Google Cloud](https://codelabs.developers.google.com/codelabs/responsible-ai/lit-on-gcp).\n\nInclude non-technical teams in model probing and exploration\n------------------------------------------------------------\n\nInterpretability is meant to be a team effort, spanning expertise across\npolicy, legal, and more. LIT's visual medium and interactive ability to examine\nsalience and explore examples can help different stakeholders share and\ncommunicate findings. This approach can enables more diversity of perspective in\nmodel exploration, probing, and debugging. Exposing your teammates to these\ntechnical methods can enhance their understanding of how models work. In\naddition, a more diverse set of expertise in early model testing can also help\nuncover undesirable outcomes that can be improved."]]
Start Codelab
Start Google Colab
