Introduction
We have seen that AI detection is not very reliable, however, we clearly want to ensure that student work is, indeed student work. While students may be using AI to support the creation of their work, it’s important to ensure that it’s not easy for students to use AI to do all of their work. Many staff use the same, or similar assessments over the years, and while it is not easy to radically change an assessment within the academic year, it is often possible to tweak assessments
Could AI answer my assessment now?
The first thing to do is to use a current or recent item of coursework and paste it into Bing (or another Generative AI of your choice). Would the answer pass? Does it meet all of the learning outcomes at an appropriate level? Has it included references? If so, are they all accurate, or are some of them hallucinations (i.e. looking convincing, but don’t actually exist).
If you think it is a low pass or a minor fail, do you think that with a few extra prompts it could generate something that would be a clear pass. Remember, your students may not know the subject as well as you do, so think about how they might adjust the prompts.
How could I change the assessment?
Immediate changes
While a significant change in the assessment type requires QA approval, small changes may be possible. Could you think of any changes that you could do now, to meet the existing learning outcomes. If you’d like some ideas, have a look at the suggestions that UCL have made. In a recent AI and Assessment workshop, one idea that was suggested by some staff was
“Use more evaluative elements like compare and contrast in essays”
We tested that type of question and found that, while Bing Chat was able to evaluate the strengths and weaknesses of two different approaches to public health when asked individually, it was not able to compare them with each other. Another suggestion was
“Set case studies involving watching a video”
What might you add? What does Generative AI suggest as options for assessments that can’t easily be answered by genAI?
In the future
It seems that increasingly, essay-based coursework is likely to be used in the same way it currently is. We already have many areas that are using authentic assessment (which takes many forms, as a quick search will show). Students will be expected to be able to use AI tools when they start their careers, in almost all fields, so future assessments are likely to use AI in some way or another.
Universities globally are looking at this, and so you may well be involved in discussions examining the role of assessment in your subject areas, and how they should be tweaked. Professional, Statutory and Regulatory Bodies (PSRBs) are also involved in discussions in this area.
JISC’s National Centre for AI has a range of useful resources, including a set of postcards (PowerPoint) to think about different approaches to assessment. This is, however, a rapidly changing field, so what is a suitable assessment approach today, might not be next week.
Resources to give you further ideas.
As well as JISC’s postcards, you may find these interesting.
- 10-minute chats on generative AI. These are hosted by Tim Fawns at Monash. While they are all interesting, Dave Cormier’s (21 Sept 2023) looks at the role of the essay in today’s world. They’re about 10 minutes long.
- “AI + AI = AI“. Martin Compton (who presented in one of our webinars), wrote a post in May 2023, looking at AI and Academic Integrity, and how they lead to the need for Assessment Innovation.
- Assessment Makeovers in the AI age, (44 minutes) a podcast from Teaching in Higher Education featuring Derek Bruff.
To think about
What challenges do you foresee when trying to design assessments that are effective in an AI world, yet also allow our students to demonstrate their knowledge and skills? How comfortable do you feel developing assessments that may be very different from those you did as a student (or even got students to do 2 or 3 years ago)?
Do share both your ideas and any other useful resources you have found in the comments.
