ÂÒÂ×Ðã

XClose

Teaching & Learning

Home
Menu

Designing assessments for an AI-enabled world

This resource provides guidance on how to adapt assessment to better support learning. It also explores how to accommodate the required changes that might arise from the global use of generative AI.

On this page:

This resource focusses on the pedagogical impact of AI on our assessment practices. It is designed to promote an environment of trust and responsibility with students through enhanced assessment practice and the fostering of a strong sense of academic integrity.

The suggestions, both in the short and longer term, are designed to enable students to understand –Ìýand work effectively with – Ìýthe capabilities, limitations and ethical implications of using AI in assessment.

Some first steps

  1. Try out aÌýgenerative AI tool for yourself.ÌýÂÒÂ×Ðã staff and students can now accessÌý.ÌýMicrosoft Copilot is a more secure tool thanÌýthe consumer-oriented generative AI services, provided you log in with your ÂÒÂ×Ðã credentials.ÌýIf you want to test exam questions (current or otherwise), this is the only safe tool to use.

  1. Consider your current assessments critically.ÌýDoes the current design really measure what you want it to measure?ÌýHow else might you ascertain students’ learning?Ìý

  2. If needed, consider amending your exam questions or written assessment tasks to make them more difficult for an AI tool to answer (see below).Ìý

  3. Make clear to your students what is acceptable use of AI in the context of your assignments, and what is not. Tell your students that they must not try to pass off work created by such tools as their own as this would constitute academic misconduct. Point them to ÂÒÂ×Ðã’s guidance on academic integrity and to the Academic Manual (section 9.2.1 g, h and m). Read ourÌýguidance for staff on using AI tools in assessment.

  1. Contact your Faculty Learning Technology Lead or yourÌýArena Faculty contactÌýif you would like further advice.Ìý

Before you makeÌýchanges


Changes you can make now

You can use this series of six videos to guide changes to policy and practice at programme or module level – they have been selected to align with ÂÒÂ×Ðã’sÌý Assessment Change Timeline.ÌýÌýÌý

1. Discuss Academic Integrity and AI with your students

YouTube Widget Placeholder

Key pointsÌý

  • Tell students when and how use of AI is permitted in assessment
  • Signpost where to find guidance (see below under 'Additional resources')
  • Explore capabilities and fallabilities of AI with students and colleagues in a context of transparency
  • Build critical AI literacy

Additional resources

For studentsÌý

For staffÌý


2. Feedback and formative assessment

Formative assessments need to be part of an integrated assessment plan where they are linked to success in the summative assessment and where there is a clear rationale for student engagement.ÌýThese Ìýprovide some excellent ways to engage students in formative activities.ÌýÌýÌý

YouTube Widget Placeholder

Key pointsÌý

  • Create a shared understanding of the conception and purposes of feedback
  • Encourage students to be active participants in feedback process
  • Consider combining tutor, peer and students' self-assessment (the latter could be informed by AI)


3. Revise exam questions

YouTube Widget Placeholder

Key points

  • Pose hypothetical questions with no simple answer
  • Pose scenario-based questions for problem solving
  • Ask students to interpret information from a range of sources
  • Ask students to evaluate AI responses
  • Consider authentic restrictions to assessment that reflect those in the 'real-world'

Tip: Speak toÌýyour Faculty Learning Technology LeadÌýor yourÌýArena Faculty contactÌýif you would like further advice


4. Revise essay questions Ìý

YouTube Widget Placeholder

Key points

  • Switch the format
  • Students to focus on their personal experience and record development in thinking
  • Ask students to generate, critique and improve AI responses
  • Ask students to write for a particular audience


5. Convert generic questions to scenario-based questionsÌý

YouTube Widget Placeholder

Key pointsÌý

  • What are the learning oucomes?
  • What action is required of the student?
  • What is the object of the action?
  • What is the context?
  • What is the scope or remit of the action?
  • What evidence does the student need to provide?


6. Upgrade your Multiple Choice Questions (MCQs)Ìý

This video includesÌýexamples of how ChatGPT3.5 deals with a range of MCQÌýquestions.Ìý

Detailed timings
  1. Overview of types of MCQs and performance with ChatGPT (0 mins)Ìý
  2. ChatGPT and maths questions (3 mins, 40 seconds) Ìý
  3. Examples of ChatGPT responses to a range of MCQ type questions (4.36 mins)Ìý
  4. Summary of strategies for upgrading your MCQs (10 mins)Ìý

YouTube Widget Placeholder

Key pointsÌý

  • MCQs, single best answers or extended matching questions that have factual answers may be bestÌýbe assessed in an invigilated context
  • Consider includingÌýimages in the question
  • Ask students to assess evaluation of AI responses
  • Reduce the number but increase the complexity of your questions
  • Ask for 'workings out'
  • Ask students to submit audio responses including a rationale for their thinking


Plan for larger changes to assessmentÌýÌý

Download the Assessment Menu [pptx]

The Assessment Menu provides you with different options to consider when making changes to your assessment. They are aligned to the Assessment Framework for taught programmes.Ìý

The menu includes 40 different cards. Each card illustratesÌýan idea for an assessment that either engages students directly with AI or for which AIÌýfind it difficult to generate a response. The suggestions here are starters for discussion and application within your discipline.Ìý

Please note:

  • This is an evolving resource as things are developing at pace.
  • This resource was developed before ÂÒÂ×Ðã staff and students couldÌýaccess Microsoft CoPilot, a more secure generative AI tool.Ìý
  • Issues of access, equity and transparency with students need to be considered if using AI is an integral part of an assessment task. We need to be attuned to inclusive practice and provide options where appropriate. Ìý Ìý
  • Learning outcomes and associated criteriaÌýwill likely need reviewÌýto reflect capabilities that graduates will need in an AI-enabled world. For this reason you may find entries such ‘emotional intelligence’ or ’contextual intelligence’ that do not directly map onto Bloom’s taxonomy as we know it.Ìý

The Assessment Menu is licencedÌýunder a .ÌýÌý

The have created an interactive PowerPoint version of the slides that can be .

Additional resources

Content for the Assessment Menu is drawn from range of sources and is largely based on Lydia Arnold’s .

You can also get ideas from the guide written by expert academics in the IOE, ÂÒÂ×Ðã’s Faculty for Education and Society. This book includes a resource bank with models, links and ideas that are ready to use and based on evidence.ÌýÌý

Back to the top of this page