2 - Assess Content: Assessing AI-Based Tools for Accuracy

Sometimes an AI will confidently return an incorrect answer. This could be a factual error, or – like in the example below – inadvertently omitted information. Vanuatu and Vatican City are both real countries, but these are not all the countries that start with the letter V.

Sometimes, rather than simply being wrong, an AI will invent information that does not exist. Some people call this a “hallucination,” or, when the invented information is a citation, a “ghost citation.”

These are trickier to catch, because often these inaccuracies contain a mix of real and fake information. In the screenshot above, none of the listed sources on The Great Gatsby exist. While the authors are all real people, and the collections are all real books, none of the articles listed here are actually real.

When ChatGPT gives a URL for a source, it often makes up a fake URL, or uses a real URL that leads to something completely different. It’s key to double-check the answers AI gives you with a human-created source. You can find out how to fact-check AI text in sections and video at the bottom of this page.

Currently, if you ask an AI to cite its sources, the results it gives you are very unlikely to be where it is actually pulling this information. In fact, neither the AI nor its programmers can truly say where in its enormous training dataset the information comes from.

As of summer 2023, even an AI that provides real footnotes is not providing the places information is from, just an assortment of webpages and articles that are roughly related to the topic of the prompt. If prompted, the AI will provide the exact same answer but footnote different sources.

.   

For example, the two screenshots above are responses to the same prompt. In first screenshot, the user specified to use only peer-reviewed sources. When you compare the two, you can see that the AI cites different sources for word-for-word identical sentences. This means that these footnotes are not where the AI sourced its information. (Also note that the sources on the right are all either not peer-reviewed or not relevant. Plus, artsy.net, history.com, and certainly theprouditalian.com are not reliable enough for you to source from in your assignments.)

This matters because an important part of determining a human author’s credibility is seeing what sources they draw on for their argument. You can go to these sources to fact-check the information they provide, and you can look at their sources as a whole to get insight into the author’s process, potentially revealing a flawed or biased way of information-gathering.

You should treat AI outputs like fact-checking a text that provides no sources, like some online articles or social media posts. You’ll determine its credibility by looking to outside, human-created sources (see lateral reading below).

AI can accidentally ignore instructions or interpret a prompt in a way you weren’t expecting. A minor example of this is ChatGPT returning a 5-paragraph response when it was prompted to give a 3-paragraph response, or ignoring a direction to include citations throughout a piece of writing. In more major ways, though, it can make interpretations that you might not catch. If you’re not too familiar with the topic you’re asking an AI-based tool about, you might not even realize that it’s interpreting your prompt inaccurately. 

The way you ask the question can also skew the response you get. Any assumptions you make in your prompt will likely be fed back to you by the AI. 

For instance, when ChatGPT was prompted: 

“Write a 5 paragraph essay on the role of elephants in the University of Maryland's sports culture. Be sure to only include factual information. Provide a list of sources at the end and cite throughout to support your claims.”

It returned an answer full of false information about elephants being a symbol of UMD sports alongside Testudo, making up some elephant-related traditions and falsely claiming that elephants helped build U.S. railroads during the Civil War. It generated a list of non-existent news articles and fake website links supporting both of these claims.

By contrast, when ChatGPT was prompted:

“Does UMD's sports culture involve elephants? Give a detailed answer explaining your reasoning. Be sure to only include factual information. Provide a list of sources at the end and cite throughout to support your claims.”

It returned a correct answer with information about our real mascot, Testudo the terrapin. 

However, the sources it provided were both dead links, either to out-of-date pages on the UMD website, or to real pages with a muddled URL. 

ChatGPT interpreted the first prompt as “taking it as a given that UMD’s sports culture involves elephants, write an answer justifying this.” However, with the way the second prompt was phrased, the AI was free to answer the question based on its training data, and returned the correct answer. 

Depending on how we phrased the question, ChatGPT either reinforced a mistake we made in the prompt or corrected that same mistake. Paying attention to your prompt phrasing can make a key difference!

You can see both the conversations in full here:

• Elephants at UMD Sports Links to an external site.
• UMD Sports: No Elephants Links to an external site.

Lateral reading can (and should) be applied to all online sources, but you will find fewer pieces of information to assess through lateral reading when working with AI. While you can typically reach a consensus about online sources by searching for a source’s publication, funding organization, author or title, none of these bits of information are available to you when assessing AI output. As a result, it is critical that you read several sources outside the AI tool to determine whether credible, non-AI sources can confirm the information the tool returned. 

With AI, instead of asking “who’s behind this information?” we have to ask “who can confirm this information?” In the video above, lateral reading is applied to an online source with an organization name, logo, URL, and authors whose identities and motivations can be researched and fact checked from other sources. AI content has no identifiers and AI output is a composite of multiple unidentifiable sources. This means you must take a look at the factual claims in AI content and decide on the validity of the claims themselves rather than the source of the claims.

Since AI output is not a single source of information but rather drawn from multiple sources that could be both factual and false, you will find it useful to break apart AI output into smaller components of information that can be evaluated independent of each other. For instance, let’s see what happens when we ask ChatGPT to write an essay on Jim Henson’s undergraduate studies at The Ohio State University.

Of course, many of you know that Jim Henson completed his undergraduate at UMD, not Ohio State, which illustrates a critical distinction to be made about AI; it will take what you provide it and try to answer your question as best it can, but it will NOT fact check you or spot incorrect assumptions in the prompt you give it. In paragraph 2 from the example above, the AI correctly states that Henson was involved in puppetry activities on campus, but is incorrect in stating this was at Ohio State. If one were to assume AI output is accurate throughout, they would not realize their own error in prompting the AI with the incorrect university. 

The issues in this example are not just based on the flawed prompt. In paragraph 3, the AI describes a Dr. Richard Lederer who was a mentor of Henson’s. A quick Google search reveals no mentions of a Dr. Richard Lederer at Ohio State or UMD. Further, the only Richard Lederer of note to come up is an author, linguist and speaker who seems to have no connection to Henson himself. This is an example of the AI “hallucinating” a seemingly factual answer that sounds quite plausible, but is unfounded upon some quick lateral reading.

Links to an external site.

Critical thinking about AI responses goes beyond determining whether the specific facts in the text are true or false. We also have to think about bias and viewpoint – two things we keep in mind when reading human authors, but you might be surprised to learn we have to keep in mind with AI as well. 

Any text implicitly contains a point of view, influenced by the ideologies and societal factors the author lives with. When we critically think about news articles, books, or social media posts out in the wild, we think about the author’s viewpoint and how that might affect the content we’re reading. These texts that all of us produce every day are the foundation of generative AI’s training data. While AI text generators don’t have their own opinions or points of view, they are trained on datasets full of human opinions and points of view, and sometimes those viewpoints surface in its answers. 

AI can be explicitly prompted to support a particular point of view (for instance, “give a 6-sentence paragraph on ramen from the perspective of someone obsessed with noodles”). But even when not prompted in any particular way, AI is not delivering a “neutral” response. For many questions, there is not one “objective” answer. This means that for an AI tool to generate an answer, it must choose which viewpoints to represent in its response. It’s also worth thinking about the fact that we can’t know exactly how the AI is determining what is worth including in its response and what is not.

AI also often replicates biases and bigotry found in its training data (as you learned on page 1 of this module). It is very difficult to get an AI tool to arrive at the fact that people in positions of authority, like doctors or professors, can be women, without explicit prompting from a human. AI image editing tools have edited users to be white when prompted to make their headshot look “professional,” Links to an external site. and can sexualize or undress women, particularly women of color Links to an external site., when editing pictures of them for any purpose.

AI also replicates biases by omission. When asked for a short history of 16th-century art, ChatGPT and Microsoft Copilot invariably only include European art. This is the case even if you ask in other languages, like Chinese and Arabic, so the AI tool is not basing this response on the user’s presumed region. China and the Arabic-speaking world were certainly producing art during the 16th century, but the AI has decided that when users ask for “art history,” they mean “European art history,” and that users only want information about the rest of the world if they specifically say so.

These are more obvious examples, but they also reveal the decision-making processes that the AI is using to answer more complex or subtle questions. The associations that an AI has learned from its training data are the basis of its “worldview,” and we can’t fully know all the connections AI has made and why it has made those connections. Sometimes these connections lead it to decisions that reinforce bigotry or give us otherwise undesirable responses. When this happens in ways we can see, it prompts the question: how is this showing up in ways that aren’t as obvious?

Now let’s try lateral reading for a second time, with a focus on the response’s perspective:

1. We can start with fractionation again, but this time we’re thinking about what claims and perspectives are being represented in the AI response.
   
   • Brainstorm the groups who might be invested in this issue and who might have a discrete perspective (stakeholders, corporations, governments, demographic groups, nationalities, regions, etc.)
   • Think about the argument as a whole. What perspective(s) can you find here? Which might be missing?
   • Now break down the response into individual claims. What perspective(s) can you find in these claims? Which might be missing?
2. Time to start your lateral reading. Think about what sources might provide the perspectives above, both the ones in the AI’s response and the ones missing from it.
   • Try publications like newspapers or well-established magazines, like the Atlantic or Scientific American.
   • You can find perspectives you’re looking for in news articles, opinion pieces, speeches, etc. Remember to think critically about these perspectives – some may be based on incorrect facts or a distortion of the issue.
   • Check Wikipedia to get a sense of each publication’s reputation.
3. Next, think deeper about what assumptions are being made here.
   
   • What did your prompt assume?
   • What did the AI assume?
   • Who would know things about this topic? Would they have a different perspective than what the AI is offering? Where could you check to find out?
4. Finally, make a judgment call. What here is true, what is misleading, and what is factually incorrect? Can you re-prompt the AI to try to get a different perspective? Can you dive deeper into one of the sources you found while fact-checking? 

Again, the key is remembering that the AI is not delivering you the one definitive answer to your question. 

For more information on lateral reading and a worksheet to track your work, see below:

• Teaching Lateral Reading Links to an external site.
• Worksheet: Lateral Reading Graphic Organizer Links to an external site.

• Buell, S. (2023, July 19). An MIT student asked AI to make her headshot more ‘professional.’ it gave her lighter skin and blue eyes. Boston Globe . Retrieved August 2, 2023, from https://www.bostonglobe.com/2023/07/19/business/an-mit-student-asked-ai-make-her-headshot-more-professional-it-gave-her-lighter-skin-blue-eyes/
• Cesari, L. [@bee_in_the_library]. (2022, January 12). A fav strategy of professional fact checkers #infoliteracy #librariansoftiktok #teachersoftiktok [Video]. TikTok. https://www.tiktok.com/@bee_in_the_library/video/7052430272150719790
• Heikkilä, M. (2022, December 12). The viral AI avatar app Lensa undressed me—without my consent. MIT Technology Review . Retrieved August 2, 2023, from https://www.technologyreview.com/2022/12/12/1064751/the-viral-ai-avatar-app-lensa-undressed-me-without-my-consent/
• Stanford History Education Group. (2020, Jan 16). Sort fact from fiction online with lateral reading [Video]. YouTube. https://www.youtube.com/watch?v=SHNprb2hgzU
• Stanford History Education Group. (2020). Teaching lateral reading . Civic Online Reasoning. https://cor.stanford.edu/curriculum/collections/teaching-lateral-reading/