Save 30% with code HTAI25 at press.jhu.edu.

Buy the Book at Amazon
Buy the Book at JHUP

Designing Learning with AI

There is research that AI can both improve and impede learning. The same research suggests important design features that can allow AI to support learning. 

1. Student use of AI continues to climb

2. Students SAY they use it for learning (and understanding instructions)

3. Students use of AI to do academic work is also on the rise.

4. Students believe that AI has improved their academic performance 

5. Students who use AI find it more ethical than those who use it less.

6. When student use is UNGUIDED, homework improves and exam scores fall.

All of this probably means that unguided student us of AI leads to less learning, but these studies mostly measure exam scores and we are need longer-term studies that measure durable and transferable learning and under what conditions, BUT…

7. Pre-AI Tutoring Systems were effective when well-designed.

8. Gen AI tools can improve learning when they are well-designed.

KEY FINDING: Who does the cognitive work matters.

Design Principles for Teachers using AI

Classroom time should center human interactions. AI offers an opportunity (maybe even an imperative) to raise standards: What can students do with AI that they could not do on their own? Now is a good moment to refocus assignment design on learning rather than catching cheaters. In the same way that video games offer a customized learning experience, designed to keep every user pleasantly frustrated, AI tools can be customized. The key is to keep students at the right (customized) level of friction. In practice that means teachers designing prompts and secure custom bots rather than allowing students to use open tools. You can find complete templates, examples and instructions at https://weteachwithai.com/creating-ai-simulations-and-custom-bots/  

    Design AI tools with constraints. For example: Never complete the user’s thinking for them. Ask questions that lead toward the insight rather than delivering it.  Ask more than you tell. Your purpose is to create conditions under which students demonstrate, deepen, and reflect on what they have already read. Praise specificity, not correctness.  If a student pastes in a plot summary or SparkNotes-style text, then say this…

    Specify a clear process for interactions. For example:  Begin by doing this. Then analyze my submissions for conventional thinking, absent sources, perspectives, arguments, or data and faulty assumptions. Begin every conversation with this exact sequence. Do not skip steps. Step 1, say this “Hi, I am your Ai tutor.” Step 2…  Then choose ONE of the following simulation frames, appropriate to the student’s complexity tier. Then…

    AI tools customize easily when provided with context so include both a calibration step (to set the difficulty or friction level) and a customization step (that creates relevance and motivation). 

    –For motivation: Ask about interests in outside of this class. What do you want to do after graduation?

    Last time you told me X; has anything shifted in how you’re thinking about that? Record student’s interests. You will use it to create analogies, frame questions, and build relevance bridges throughout the session. Do this organically.

    –For friction and complexity level:  Where is one place you have struggled with this material? Can you explain concept X to me? How experienced are you with this topic? How would you interpret or explain Y? Your role is to build a picture of the student’s blind spots, engagement patterns and intellectual strengths (specifically abstract argument or other) and push them into moderate discomfort at least once per session. Do not announce that you are doing this. Simply do it. The user should feel intellectually stretched.

    These two ideas are connected because too much feedback at once is demotivating. AI is patient so you can focus it to do both at once. For example: Encourage students to engage in self-reflection by providing other perspectives and ideas. Stimulate deeper student thinking by providing thought experiments, second opinions, alternative views, and even contrarian scenarios to broaden horizons. Help students discover nuance, innovative ideas, and new ways of thinking. HOWEVER, give only one suggestion or hint at a time. Focus on where I could most improve. Do not overwhelm me with too much information and guidance at once. Ask me if I want more, but stop when the recommendations have little true value. 

    Creating instructions that require the AI to ask about reasoning pushes students to do the thinking, but it also makes their thinking more visible for grading. For example: Ask students to explain their thinking and continue pushing them to go deeper. Ask “why did you choose that strategy?” and “Can you explain each of those steps?” Ask students to provide alternative explanations, sources and counter-arguments. Interrogate students in a way that leads them to distinguish between verifiable facts and speculation. Require commitment: if the student gives a vague or hedging answer, push back: “If you had to bet on ONE interpretation, what would it be? Commit to a reading.” Ask “help me make sense of what you mean.”

    Since student agency is important for the development of critical thinking, provide opportunities for students to critique AI output. For example: Ask students to dissect your position and tell you why it is wrong. Ask students to verify information you provide. Make “find and fix the AI’s mistake” a recurring task.

    Students easily come to rely on AI, so it is important to make sure there are unaided work and different (and AI-off) assessments frequently. Some of those could might be AI assessment bots, but then with even more guardrails. 

    AI has tremendous potential to increase access but also to widen equity gaps. Do not assume everyone uses AI well. Provide equal access and explicit AI-literacy support. Paid tools are often much better, but AI tools are well-designed then can often run on free models effectively.  All of the example simulations here were stressed tested on the best models but set to run and interact with students using only free models.