20 Evaluating AI output
GenAI output should always be evaluated for validity, accuracy and bias.
Before you prompt GenAI:
- Think about the questions (prompts) that you want to pose to a GenAI tool -how will you benefit from the capabilities that it offers?
- GenAI is not a search engine – so don’t use it like a search engine. Sometimes Googling, or using a library database is a more effective step to find an answer.
- Recognize that GenAI has limitations. It may not be the tool that you need for the task at hand.
Evaluating the response
- Fact-check everything. Because GenAI can produce outputs that sound normal, but also are factually incorrect or subject to bias, you’ll want to fact-check outputs. Every time.
- Compare outputs with reliable sources. Examples of reliable sources may be peer-reviewed journals, authoritative websites, and recognized textbooks. If two or more independent sources disagree, you may need to dig further.
- Assess timeliness: Some models have knowledge cutoff points. AI-generated information may be out of date due to the training data. Use current, reliable sources to determine if there are new events, discoveries, or practices that would impact the credibility of the information.
- Verify any citations or studies provided to you: these may or may not exist.
- Check for bias by seeking balanced perspectives from diverse sources outside of what the GenAI tool tells you to check.
- Verify the logic behind the output. Look at each step of the reasoning when available and replicate it on your own, whether it’s algebra, programming, or philosophical reasoning. If the AI won’t show you its reasoning, that should be a concern that it isn’t a trustworthy output.
