In AI-enabled healthcare education, critical thinking comes first

“AI is revolutionising healthcare”; “AI can enhance learning”, “AI is degrading students’ ability to think”. If you work in health professions education, you’ll likely have read such headlines more than a few times. As the evidence for AI’s impact remains mixed, it’s easy to feel confused and overwhelmed. So, what is a healthcare educator to do? 

Perhaps a greater focus on AI literacy help us steer a path. But that leads to a whole host of other questions, leading with: what does that look like in practice? 

• Support medical students to form their identity as doctors
• Generation Z presents new challenges for medical education
• How reflective remediation can set medical students up for success

Medical students need to know enough about generative AI to make good decisions about if, when and how to use it. This requires a broad understanding of what it is and how it works, its strengths and limitations, its ethical and legal implications, how to get the best out of it, and how to evaluate its outputs. They should also be aware of the fundamental importance of data and know how to interrogate it for bias in ways that are task-appropriate.

Healthcare students need to understand the resource implications of the AI they use so they can evaluate the environmental and other trade-offs involved. This means realising that the cloud infrastructure on which the AI enterprise relies is not of the white and fluffy variety but an energy- and water-hungry metal and concrete behemoth that is gobbling up more and more critical resources as the scale of AI development accelerates. 

Knowing this can help them make more ethically formed decisions about which uses are worth that price and the part they can play in steering future development towards a more sustainable future.

The Unesco AI competency framework for students offers a useful starting point for all of the above. However, as future healthcare professionals, students will over time need to layer in additional information pertinent to the healthcare context. This will include developing a more fine-grained awareness of the types of AI used in healthcare, and the ways in which they work. Ideally, they should know enough about AI to recognise the opportunities it presents and be able to ask the right questions, and even collaborate with AI developers to solve real problems that healthcare systems face.

They should be able to apply their knowledge of data bias to interrogate what it means for the patients they serve. They will require the communication skills to deal with patients who come to them with their condition pre-diagnosed by a chatbot, treatment plan in hand. They will need to recognise that AI is not infallible and have developed the knowledge and competence to question its outputs, and the confidence and leadership attributes to overrule it when appropriate. 

To do this, they must develop the fundamental architecture of clinical skills, competence and evaluative judgement that will enable them to make evidence-informed decisions. This will require them to learn to deal with ambiguity and uncertainty while formulating solutions to frequently complex problems. Along the way they will have to make the choice to struggle with the difficult and messy thinking processes that this entails, forgoing the temptation to reach first for an LLM-derived solution. And critically as students, they must develop the metacognitive awareness to recognise that this is a choice worth making.

So, what are the foundational AI literacy skills that will set the stage for these future-focused abilities? They really aren’t that much different than for every student. Focusing on the fundamental AI literacy competencies identified by Unesco and laser-like attention to developing students’ ability to think should help provide a strong foundation for them to thrive in the AI era. 

Here, I’ll identify five strategies educators can use to help students bring critical thinking and metacognitive awareness to their use of AI.

1. Explicitly teach and model critical thinking and inductive reasoning

• Make reasoning processes explicit
• Teach learners to generate and test hypotheses before consulting LLMs
• Focus on mechanisms, principles and causal understanding

2. Support students to distinguish between types of LLM use

Does outsourcing certain tasks to an LLM undermine critical thinking, or foster it? Provide basic instruction on negative cognitive offloading and encourage student reflection on their own practices.

3. Include opportunities for students to critically interrogate LLM outputs

Allow use of LLMs in some case-based learning sessions to explore issues of bias, hallucination and superficial reasoning.

4. Promote a ‘think first’ approach to the use of LLMs

• Encourage students to use LLMs to provide feedback or alternative perspectives on work rather than completing it for them. 
• Design assessments that reward process, reasoning and explanation.

5. Scaffold the development of metacognition 

• Support students to monitor and manage their thinking when using generative AI. 
• Encourage them to use structured reflective questioning, for example, could I explain this work without AI support?
• What did I learn from doing it?
• Am I improving in this subject over time?

The concept of “human in the loop” underpins many regulatory frameworks governing AI in healthcare, but a human in the loop is only as effective as their preparation for that role. Health professions educators have a responsibility to ensure that those humans have developed the foundational knowledge and critical capacities to fulfil that role in a meaningful way.