Teaching Quality Enhancement Group
A department of the Academic Development Unit, Corporate Services
A department of the Academic Development Unit, Corporate Services
Embedding Support Processes for e-Assessment to ensure the reliable and secure large-scale implementation of CAA.
The National Student Survey has identified assessment methods and assessment feedback as important issues across the sector. At the University of Bradford these issues are now part of a debate which will lead to more comprehensive policy development regarding assessment. Based on a series of pilots, we believe that innovative e-assessment in general and computer-assisted assessment in particular can make an important contribution. However, while we have invested in the underpinning technology (both hardware and software) which should support the expanding demand for this service over the next few years, we do not have the physical facilities to make this easy to administer, and we have not yet developed the integrated administration which will enable this on a large scale. Projects in the e-strategy have provided the support for physical facilities; the focus of the pathfinder project is developing the adminstrative and support systems.
This project addresses several key issues that affect all HEIs considering the use of online assessment, namely:
John Dermo, Adviser on Computer Assisted Assessment, Teaching Quality Enhancement Group:
John supports academic, administrative and technical staff in the ongoing development of e-assessment and coordinates all CAA-related activity at the University to ensure that we provide a coherent and comprehensive support service for online assessment. Building on the Pathfinder work, John also provides specialist support to external projects, such as the current JISC Institutional Exemplar project “Integrating thin client systems and smart card technology to provide integrated, flexible, accessible and secure E-assessment”.
Official university guidelines for CAA: rules and regulations document specifying administrative and support functions to ensure resilient and manageable online formative and summative assessment routes.
Pedagogical guidelines (“best practice”) document for recommended use of software for formative and summative assessment:
Training sessions for instructors (in Sept 07, Jan 08 and May 08) in use of Questionmark Perception (QMP) and other software. The following sessions were delivered: Come and Create High Quality Multiple Choice Questions, Come and Create Online Assessments in Questionmark Perception, Online Assessment in Blackboard.
In-house training materials for use of QMP.
Web-based resources for use of QMP: support webpages also a special community site has been created in the VLE for users of QMP.
Specification of CAA systems at University of Bradford: these 2 reports describe the support systems and technical systems in place for CAA at the University of Bradford, established through liaison with academics, ASSU and IT services.
CAA induction and training sessions for existing and new invigilators: for the first time at the University of Bradford, for the academic year 2007-8, invigilator induction and training incorporated Computer-Assisted Assessment. At the general invigilator induction and briefing in May, a specific slot was devoted to CAA. In addition to this, ASSU have identified a core team of CAA specialist invigilators, who will focus on online exams in their invigilation duties. A special Invigilator training session was held for this team in December.
Research evidence (quantitative and qualitative) on use of CAA in undergraduate courses to inform the sector; two surveys have been carried out at UoB that provide evidence that relate to this project. The questionnaire SUTTL (staff use of technology in teaching and learning) provides quantitative and qualitative data on current staff use of various technologies, including Questionmark Perception. In addition, SPEAQ (student perceptions of e-assessment questionnaire) provides valuable quantitative and qualitative data on student attitudes towards e-assessment. Quantitative data on increased staff and student use of QMP in 2007-8 has also been obtained from the Questionmark database.
Detailed case studies of institutional implementation of CAA; See briefing paper (forthcoming from HEA)
Development of banks of online assessment materials in QMP: Evaluations of JORUM, SWAP and QM Shared Repository were carried out, with the latter adopted as the best alternative for the present.
Baseline study and needs analysis report on use of CAA at UoB. At the start of the project, a baseline study and needs analysis was carried out.
Dermo, J. (2007) “Benefits and Obstacles: Factors Affecting the Uptake of CAA in Undergraduate Courses” in Khandia, F. (ed) Proceedings of the 11th International Conference on Computer Assisted Assessment: Research into e-assessment. pp 155-162
Dermo, J. (2008) “Implementing Online Assessment: Finding the Right Path for an HE Institution” in Ladwa, A (ed) E-Learning in HE available online at http://www.rsc-yh.ac.uk/Documents/HEbooklet2_000.pdf (last accessed 12 June 2008)
June 5th 2007: University of Bradford 6th Annual LTA conference: Looking Ahead: Presentation by C.Alder and J. Dermo (TQEG), G.Harel (School of Informatics) and L.Breen (School of Management) “Developments in Computer-Aided Assessment”
July 10th 2007: 11th International CAA conference, Loughborough University: Presentation by J. Dermo “Benefits and Obstacles: Factors Affecting the Uptake of CAA in Undergraduate Courses”
February 26th 2008: JISC RSC YH Symposium for FE/HE Strategists & Practitioners, University of Sheffield: Presentation by J.Dermo “Implementing Online Assessment: Finding the Right Path for an HE Institution”
April 30th 2008 : Questionmark Breakfast Briefing, Manchester: Case Study Presentation for Questionmark users by J.Dermo “Implementing Online Assessment: choosing the right Path for an HE Institution”
May 13th 2008 Blackboard WorldEurope Conference, Manchester: Presentation by J.Dermo “Formative and Summative Assessment: Integrating Blackboard Software and Questionmark Perception”
June 3rd 2008: University of Bradford 7th Annual Learning, Teaching & Assessment Conference: Meeting New Challenges: Presentation by J. Dermo, “Implementing Online Assessment: Choosing the Right Path for an HE Institution”
July 9th 2008: 12th International CAA conference, Loughborough University: Presentation by J.Dermo and S.Eyre “Secure, reliable and effective institution-wide e-assessment: paving the way for new technologies” (proposal accepted)
September 15-16 2008: Questionmark European Users Conference, London: Case Study Presentation by J. Dermo “Implementing Online Assessment: choosing the Correct Path for an HE Institution” (proposal submitted)