16 Output reliability and GenAI
GenAI is very confident, and also sometimes very wrong
Sometimes, GenAI tools will create information that sounds confident and grammatically correct but is actually wrong or misleading. We call this a hallucination or a fabrication.
GenAI models do not think; they are designed to predict the next word in a sentence based on patterns it has seen in huge amounts of data. It doesn’t actually know facts like a human does. It just knows what a fact looks like based on those patterns. Sometimes, it confidently gives you information that is completely false or made up because its algorithm tells it to change up the response and predict a word or pixel that isn’t the most probable.
Fabricated citations
Hallucinations and fabrications often show up when you ask GenAI for academic research sources. You might get real-looking titles and authors listed in perfectly formatted citations that don’t actually exist. Always verify citations by searching for the DOI or pasting each citation into Google Scholar or another library database.
Prompt: Give me five good research sources on fast fashion
- Kumar, Vikas, et al. “Sustainable Fashion Supply Chain: Lessons from H&M.” Journal of Cleaner Production, vol. 112, no. 4, 2016, pp. 2377-2385. ScienceDirect, doi: 10.1016/j.jclepro.2015.09.048.
- Hancock II, Joseph H. “Fast Fashion, Sustainability, and the Ethical Appeal of Luxury Brands.” Fashion Theory: The Journal of Dress, Body & Culture, vol. 17, no. 3, 2013, pp. 351-368. JSTOR, doi: 10.2752/175174113X13673404903122.
- Netter, Sarah, et al. “Sustainable Fashion Consumption and the Fast Fashion Conundrum: Fashionable Consumers and Attitudes to Sustainability in Clothing Choice.” International Journal of Consumer Studies, vol. 41, no. 3, 2017, pp. 290-298. Taylor & Francis Online.
- Liu, Tingchi, et al. “A Circular Economy Approach to the Fashion Industry: A Review.” Sustainability, vol. 12, no. 5, 2020, pp. 1788. MDPI, doi: 10.3390/su12051788.
- Rissanen, Timo, and Holly McQuillan. “Textile Waste Minimisation Through Design.” Journal of Textile Design Research and Practice, vol. 5, no. 1, 2017, pp. 1-22. Taylor & Francis Online, doi: 10.1080/20511787.2017.1281423.
Any output from GenAI should be evaluated for accuracy, but especially for things like citations. In this example, we’d want to either
- follow each DOI link in the citation, or
- copy and paste each citation into Google Scholar or another database.
Is this just with text output?
Fabrications do not just occur with text output. GenAI image creators are also prone to hallucinations and fabrications. Image generators don’t understand physical anatomy or language mechanics. They frequently generate humans or animals with incorrect numbers of fingers, toes, or limbs, and they struggle with spelling embedded text or creating accurate infographics.
In the example image of the University of Minnesota students, there are several elements that are concerning about it being a real image:
- Duluth is spelled incorrectly,
- the mascot for Rochester is incorrect, and
- it doesn’t show the University of Minnesota colors as much as it could.
Since AI-generated images aren’t always accurate, it’s really important to evaluate them.
You can do this by using tools like Google’s reverse image search to see if the image (or parts of it) exist elsewhere or to find reliable sources for the information it supposedly represents.
Do I trust the information that GenAI produces?
It’s becoming more difficult to discern reality from GenAI in both text and images. This is where those essential critical thinking skills come into play. Pay attention to what kind of output you are expecting from GenAI and if it could be prone to misinformation.
- Risky use: Relying on AI image tools to produce accurate diagrams, anatomical charts, or data infographics or asking GenAI for specific stats, historical dates, or scientific data without independent verification.
- Safe use: Using GenAI to turn bulleted notes into an essay outline or asking GenAI for alternative terms, synonyms, or related concepts to broaden your search in real academic databases like JSTOR or PubMed.
