Artificial Intelligence

Generative AI, or GenAI, are tools which can generate new text, audio, images, or other media by analyzing patterns within data they have been trained on. ChatGPT is a popular GenAI chatbot that responds to user questions and prompts in an ongoing conversation.

ChatGPT and similar chatbots can help you brainstorm research topics or suggest search terms for finding scholarly sources in library databases. However, these tools have significant limitations you need to be aware of. This page will discuss some concerns and best practices for using ChatGPT to help with the research process.

Core Concepts 

• Training: Process of teaching AI by feeding it large amounts of data (like text, images, or audio); the AI learns to recognize patterns to make predictions or generate responses.
• Prompts: Any command or question a user gives an AI tool—whatever you type into the box.
• Hallucinations: False or made-up information that an AI tool might confidently present as true.
• Leading questions: Questions that suggest a specific answer or influence the response, leading to confirmation bias.
• Confirmation bias: The tendency to seek out or interpret information in a way that supports existing beliefs.
• Open-ended questions: Questions that can't be answered with a simple yes or no, encouraging fuller exploration of multiple sources.

Further Reading

• CIRCLS. (n.d.). AI glossary. Center for Integrative Research in Computing and Learning Sciences.  
• Murad, A. (2023, September 6). LLMs for dummies. Digital Native.  
• University Library. (n.d.). Generative AI: Overview. University of Illinois Urbana-Champaign. 

ChatGPT can be a very helpful tool, but understand its limitations. Review the explanations below to understand why these tools are starting points to help you explore an idea and get tips on how to begin your research. Always remember, ChatGPT is a starting point, not the end point in your research process.

What to Look Out For

• AI Wants To Please You: Be careful how you word your prompts. Make them specific and detailed, but keep questions open-ended. Targeted (or leading) questions will result in ChatGPT confirming your assumptions rather than showing other possible answers (confirmation bias).
• Inaccuracy: GenAI tools can fabricate entirely made-up information (hallucinations). ChatGPT is only as good as its training data. If the AI has been trained with incomplete, outdated, or incorrect information, it will replicate inaccuracies in its results. With niche topics, it's more likely to produce inaccurate information. With well-researched topics, it may prioritize certain perspectives over others.
• Bias and Assumptions: AI replicates human biases present in its training data. Academic publications have historically lacked diversity in its authorship and subject content, which AI responses will reflect. Good research requires multiple perspectives and authorities and because you cannot know what "sources" ChatGPT is using.
• Censorship and Community Guidelines: Library databases don't censor search results, but ChatGPT may not respond to certain prompts that violate its usage policies, which frequently change. You also can't be sure if it's not selectively censoring the responses it does generate. 
• Verify Everything: ChatGPT is a starting point to explore ideas and get research tips. Always verify information through library resources and scholarly sources.

Let's Experiment

I asked ChatGPT for information on the etymology and changing usage of the word "dude" in the English language and to provide me 5 sources of information, so that I could look more into the topic. ChatGPT provided me with the following sources:

• "The Birth of the 'Dude': From Dandy to Surfer, and Beyond" by Ben Zimmer, published in The Atlantic: https://www.theatlantic.com/entertainment/archive/2010/06/the-birth-of-the-dude-from-dandy-to-surfer-and-beyond/57985
• "Dude: The Evolution of an American Word" by Jay Heinrichs, published in Smithsonian Magazine: https://www.smithsonianmag.com/arts-culture/dude-the-evolution-of-an-american-word-47576079/
• "The Dude Abides: The Cultural Significance of the Word 'Dude'" by Kate Abbott, published in JSTOR Daily: https://daily.jstor.org/the-dude-abides-the-cultural-significance-of-the-word-dude/
• "The Rise of 'Dude' in American English" by Anne Curzan, published in the Oxford Research Encyclopedia of Linguistics: https://oxfordre.com/linguistics/view/10.1093/acrefore/9780199384655.001.0001/acrefore-9780199384655-e-456

Go to the library catalog and see if these sources are real or hallucinations.

Let's Experiment

Pick a GenAI tool from the list at the bottom of the page and ask it the same ethically complex question (e.g., "Who makes the best leaders?") multiple times. Notice how responses might contain subtle biases or assumptions. Then try rephrasing your question to encourage a more balanced response. How does the AI's answer change?

Further Reading

• Center for Teaching Excellence, Addressing bias in AI-generated writing. University of Kansas. 
• Goodwin, J. (2024, April 16). Google is hallucinating idioms — these are the five most hilarious we found. Tom's Guide. 
• Jones, E. (2024, February 26). ChatGPT hallucinates fake but plausible scientific citations at a staggering rate, study finds. PsyPost. 
• Mather, M. (2023, May 18). Confirmation bias in the era of large AI. Psychology Today.
• Tiryakioglu, L. (2023, May 27). Lawyers sanctioned after submitting fake court citations generated by ChatGPT. CNN Business. 

ChatGPT isn't perfect and its responses should never be blindly trusted, but it can still be an incredibly helpful tool during the early stages of the research process such as exploring a topic, brainstorming research ideas, and generating search terms.

Selecting a Topic & Developing Your Research Question

Ever had a great idea but its too broad for your assignment and you're not sure how to move forward? ChatGPT can help with that initial exploration and brainstorming process. You want to be specific with your prompt and include:

• type of assignment (research paper, argumentative essay, literature review…)
• course (history, economics, sociology...)
• student grade level (freshman, lower division, capstone…)
• your general topic

Suggested template and example: 

• "I’m working on a [type of assignment] for a [grade level] [course type] class and I need help coming up with a topic. I’m interested in topics related to [general topic/subject]. Please give me a list of 10 topic ideas related to that."
• "I’m working on a research paper for an upper division marine biology class and I need help coming up with a topic. I’m interested in topics related to marine mammals. Please give me a list of 10 topic ideas related to that." (ChatGPT conversation)

You can ask for additional ideas or refinement of topics that interest you. The goal is to generate narrower, more developed sub-topics suitable for research. 

Generating Search Terms

Now that we have an interesting topic, we need to figure out how to find credible information to use in our assignment. To do this, ask ChatGPT to "act like a librarian" and provide you with a list of keywords for searching library databases.

Suggested template and example: 

• "I'm interested in [your research topic]. [Assign a role] and show me a list of search terms I could use to find academic papers on this topic in my college's library catalog and database."
• “I’m interested in how climate change affects dolphin lifespans and population sizes. Act like a librarian and show me a list of search terms I could use to find academic papers on this topic in my college's library catalog and database.” (ChatGPT conversation)

Look over and try out a combination of the suggested search terms in the library catalog. If searching these terms doesn’t lead you to relevant sources, reach out to a librarian for help or head back to ChatGPT and continue prompting for additional search terms. You could try something like “these keywords returned results about x, I want keywords that return results about y.”

Let's Experiment

Choose a general topic that interests you (like "sustainable fashion" or "digital privacy"). Open ChatGPT and use the template provided above to ask an AI tool for 10 narrower research topic ideas. Then ask it to suggest search terms for your favorite topic from the list. Take these search terms to your library's database and see what kinds of scholarly sources you find.

Further Reading

John Spoor Broome Library. (n.d.). Choosing and refining research topics. California State University Channel Islands. 

Minimizing Confirmation Bias

Prompt writing is an art and you want to be as specific as possible: assign AI a role, describe the task, explain context, and designate an audience. For example, a prompt could be "Act as a fantasy novelist. Write a description introducing the main character of a quest story that follows the typical Hero’s Journey structure.” However, don't confuse writing a detailed prompt with asking leading questions, which will lead to confirmation bias. 

Follow the tips below to help minimize the risk of confirmation bias when asking ChatGPT questions.

• Start with open-ended questions:  
  • Leading questions suggest a specific answer or influence the response, leading to confirmation bias (the tendency to seek out or interpret information in a way that supports existing beliefs). So instead ask open-ended questions that can't be answered with a simple yes or no, encouraging fuller exploration of multiple sources.
  • Leading Question: "How does human activity contribute to climate change?"
  • Open-ended Question: "What are the different factors contributing to climate change?"
• Evaluate and ask follow-up questions:
  • Read GenAI responses closely and look for inconsistencies, missing perspectives, bias, or if the responses seem too similar. Remember, ChatGPT replies with hallucinations all the time. Ask follow-up questions that challenge its response or ask for additional information on a particular aspects. 
• Ask questions from multiple perspectives:
  • Comparing and evaluating these answers helps you identify confirmation bias, explore inconsistencies, and review possible opposing viewpoints on your topic. Regarding our examples, after comparing the responses and asking follow-up questions, we understand that while baseball seems to play a significant role in some Star Trek episodes of one Star Trek television show, it may not be particularly significant to the franchise as a whole. With this baseline understanding, we can now decide if this topic is worth further research or if you'd be better off exploring a different topic.
  • "What human activities contribute to climate change?"
  • "What natural factors contribute to climate change?"
  • "What are the economic implications of combating climate change?"
  • "Why is the evidence for climate change inconclusive?"

Examples: Star Trek and Baseball

Let's explore a couple of prompts and responses in ChatGPT to test out the advice and warning above. This is a course on Artificial Intelligence, did you think we wouldn't bring up Star Trek?

Example: Leading Question

Prompt: “How is baseball used as a symbol for human creativity in Star Trek?” (ChatGPT)

The prompt assumes a few things: baseball is in Star Trek, baseball holds symbolic value in Star Trek, and that baseball is a symbol for human creativity in Star Trek. This might not appear a leading question but it includes a lot of assumptions, which ChatGPT will use in its response. With this leading of a question, do you think ChatGPT responds that baseball acts as multiple symbols in Star Trek or that it only has very limited symbolic meaning within the Star Trek franchise because it only appears in one show? Nope. Because you’ve told ChatGPT exactly how you want it to think about baseball in relation to Star Trek (i.e. as a symbol for human creativity), it generates an answer that confirms your initial assumption that baseball symbolizes human creativity in Star Trek. 

Example: Leading Question

Prompt: “I’m working on an argumentative essay for a lower-division American Studies course. I’m interested in how baseball is used as a symbol for human creativity in Star Trek. Show me 5 possible research questions related to that topic.” (ChatGPT)

This question follows the suggested template above, but it still includes a leading assumption that baseball symbolizes human creativity in Star Trek. How do you think ChatGPT will respond? Just like with the general question, because you’ve led ChatGPT to a specific answer, it simply generates variations on questions that confirm your initial assumption. But what if there are other possible interpretations of what baseball symbolizes in Star Trek? Or what if someone could argue that baseball doesn’t have a symbolic meaning in Star Trek? If you don’t research these additional possibilities, you won’t be prepared to address potential counter arguments to the claims you make in your research paper or project. 

Example: Open-ended Question

Prompt: “How is baseball represented in Star Trek?" (ChatGPT)

Let's start with open-ended questions and not lead the AI to one specific answer. This prompt allows for multiple perspectives; however, remember that even with an open question, you need to read the response critically and challenge ChatGPT with follow-up questions - because this answer still assumes that baseball is important in the entire Star Trek franchise.

Example: Ask questions from multiple perspectives

Prompt: "Does baseball play any significant symbolic role in Star Trek?" (ChatGPT)

ChatGPT's states that baseball plays a symbolic role in Star Trek and then provides numerous examples. However, after a close reading, it’s clear that all the examples come from one tv show, Star Trek: Deep Space Nine. This seems odd and it took a follow-up question for ChatGPT to clarify that baseball only has deep symbolic meaning in Deep Space Nine, which is a very important distinction if this was your research topic.

Ask questions from multiple different perspectives.

Go against all my advice and ask very leading questions but from different perspectives. For example:

• “Since baseball plays a major role in Star Trek, how does the franchise use the sport to explore themes of tradition and nostalgia?”
• “Since baseball is barely mentioned in Star Trek and has little impact on the franchise, why do you think the writers largely ignored the sport?”
• "How is baseball explored in Star Trek?"
• "Why isn't baseball explored in Star Trek?"

Let's Experiment: Pick an Interest

Let's try out what you've learned. 

1. Pick a topic. Pick a special interest or something covered in-depth in class, the more niche the better. Your pre-existing knowledge will help you quickly fact-check and the more niche the topic, the more likely ChatGPT will struggle with its responses.
2. Open ChatGPT in a separate tab on your browser.
3. Ask an open-ended question on your topic.
4. Ask 2 additional contradictory open-ended questions on your topic.
5. Ask 2-3 leading questions on your topic.

Compare the responses, then ask yourself:

• Did any of the responses contradict others?
• Do you notice any incorrect or incomplete information?
• Does anything about the responses surprise you?
• Does this change how you feel about using ChatGPT?

Further Reading

Apostrophe. (2023, August 9). The art of asking questions in the age of AI. ApostropheCMS.  
Gasc, C. (2023, June 7). Is ChatGPT your biggest fan? Avoiding confirmation bias & biased AI. Atos. h
Harvard University Information Technology. (2024, February 13). AI prompts: How to get the best results from generativeAI.  
MIT Sloan Educational Technology Office. (n.d.). Effective prompts. MIT Sloan EdTech. 
University of Waterloo Library. (n.d.). Bias in ChatGPT and generative AI. University of Waterloo. 

Here’s a list of AI resources and tools you may find helpful in your research process. Always ask your professor’s permission before using them. Navigate to the second tab in this box for video tutorials on how to use these tools. Find detailed information on additional AI research tools on the Generative AI Product Tracker created by ITHAKA S+R.

Canvas AI Resources

• AI Commons - Platform offering AI tools, training programs, and CSU-developed solutions
• Academic Applications of AI (AAAI) Student Microcredential - Comprehensive program for all CSU students to enhance understanding of AI

CSUCI AI Resources