We often view generative AI (GenAI) in binary terms: something to simply ban or allow. That frame is too narrow. Even when students use it in line with institutional policy, they may mistake fluent responses for reliable accounts. So we must also teach students to judge the output.
Fact checking is important, but it is not enough. An AI tool’s response may contain no obvious factual errors yet still be misleading because it obscures, oversimplifies or includes plausible-sounding references that do not exist. Unesco’s AI competency framework for students places judgement and human agency at the centre of AI literacy. We can translate that principle into a simple seminar activity.
1. Begin with an answer that sounds convincing
Choose an answerable yet highly contested question relevant to your discipline. For example, in a communication class, “Do social media platforms polarise society?” Ask students to enter the prompt into an approved, free GenAI tool and save the unedited response. Choose a tool that doesn’t require students to purchase a subscription or disclose personal information.
Before checking anything, ask students to mark the sentences they find most compelling and explain why. Was it the confident tone, a specific statistic, a named scholar or a clear causal link? This step makes an important distinction visible: credibility is based on impressions; reliability is based on evidence-backed judgement.
2. Run a reality check, not just a fact-check
Divide the response into four layers. Students should identify:
- verifiable claims
- identifiable sources or authorities
- conceptual claims that depend on definitions, which students must state
- implicit assumptions that shape the answer.
Give small groups 20 minutes to audit one layer each using library databases, course readings and original sources.
- Spotlight guide: Critical thinking in teaching and research
- Boosting AI literacy across your institution
- Reaching for the pizza? How to help students make informed choices with GenAI
For every claim central to the answer’s conclusion, students must use one of four labels: supported, contradicted, unverifiable or contested. They must record what evidence led to the decision. If a cited study exists, does it make the claim attributed to it? If the AI tool says platforms “cause” polarisation, does the evidence show causation, correlation or simply plausibility? Which groups, countries or platforms does the tool prioritise?
This stage moves students beyond hunting for hallucinated citations. It shows that accuracy also depends on scope, definition, inference and omission. It also reflects assessment reform guidance from Australia’s Tertiary Education Quality and Standards Agency: learning activities should prepare students to engage critically and actively with AI, while assessment should make the learning process visible.
End the audit with cross-group comparisons. Two groups may assign different labels to the same claim because they used different evidence or interpreted a concept differently. Do not rush to resolve the disagreement. Ask which source most directly presents the underlying study or data, which definition is being used and what additional information would change the judgement. This models scholarly disagreement as a reasoned and evolving process rather than a contest between equally valid opinions.
3. Have students rewrite the response
Now ask each group to rewrite the AI tool’s response in 150 to 200 words. The goal is not merely to correct errors or improve style. Students must distinguish interpretation from concrete evidence, highlight discrepancies and errors and include at least one credible alternative explanation. They must also remove any source they cannot verify.
Require a brief change log with the rewrite. It should include one retained claim, one qualified claim and one rejected claim, as well as a missing perspective they have added. Students should also provide a reason for each decision. This change log is valuable because it reveals the judgement process.
Assess students using four criteria: quality of verification, recognition of uncertainty and inaccuracies, openness to alternative explanations and justification of revisions. Do not reward students simply for disagreeing with the AI output. Scepticism means developing judgement, not assuming that every machine-generated statement is false.
Make the exercise reusable
The activity can fit into a 50-minute seminar or become a longer assignment. In large classes, assign each group one paragraph and combine the audits. In disciplines that require students to handle sensitive data that cannot be entered into the tools, generate the sample answer yourself. Repeat the exercise later with a stronger prompt so that students can test whether better prompting improves not only fluency but also the quality of the output.
Students are asking for practical support to develop critical thinking when using AI, a Jisc report found. A ban may remove AI from the equation, but this leaves students ill-prepared to critically engage with AI-generated content. By turning AI output into an object of collective enquiry, educators can teach a valuable habit: rather than simply asking whether an answer is correct, asking what makes it credible.
Mert Can Atar is an assistant professor in the department of new media and communication, Faculty of Communication, Istanbul Aydın University, Turkey.
If you would like advice and insight from academics and university staff delivered direct to your inbox each week, sign up for the Campus newsletter.
comment