Innovative Approaches to Assessing Clinical Skills of Large Groups using Video

What was the learning and teaching challenge you faced?

There were a number of teaching and learning challenges with the paper-based method

¡ñ Students had no flexibility as to when they did the face-to face exam. They were allocated a specific date and time to attend.

¡ñ Pressure, nerves, anxiety and stress for students can affect or enhance their usual performance. ¡ñ Fatigue and boredom can potentially affect assessor¡¯s concentration and objectivity leading to inconsistency in marking.

¡ñ The mark sheet is the only record of the student performance so there is no way to verify results if there is later disagreement.

¡ñ Paperwork can go missing.

¡ñ Manually entering marks on the computer system for the face to face exam left a lot of room for error.

¡ñ The face to face practical exams were extremely resource intensive requiring a lot of time and human and material resources

¡ñ Few students made appointments to receive feedback so valuable teaching and learning opportunities were missed.

What was done?/What did you do?

In order to address these problems, I replaced the paper-based face-to-face assessment with online video submission in 2013/2014. Students now submit a video of themselves performing the nursing skill. They buddy up and use their own devices (i.e. smart phone, IPad) to record each other performing the required clinical skills and then upload the videos to Google Classroom. The module co-ordinator and one other person independently mark each video on Loop according to a marking guide. This format is now embedded in the module and has been used each year since then with ongoing refinements.

How did it work for you?

This had overcome the problems identified with the paper-based face-to-face format.

¡ñ It offers more flexibility as students can submit their recordings anytime within defined timespans.

¡ñ Having the option to record multiple attempts reduces the negative emotions associated with having one attempt in front of an examiner. They can have multiple attempts and submit what they feel is their best attempt. This can encourage self-assessment and possibly peer-assessment ¡ñ Fatigue and loss of concentration are no longer issues as examiners can take breaks and rewind the videos. Consistency is further enhanced by having 2 markers for each video. If they have different results for a student, they can watch the video together and agree a mark.

¡ñ If results are disputed the student and the assessor can review the performance together to see exactly what was done.

¡ñ There is no risk of paperwork going missing because all videos are marked online. This also eliminates the potential of errors inputting results.

¡ñ There are significant time and resource savings

o time required per student reduced from roughly 10 minutes to less than 4 minutes.

o number of academic staff required from 8 to 2 over 2 days

o no paper or photocopying needed

¡ñ Online feedback is available once all submissions are marked. Students can read the feedback while looking at their video recording rather than trying to remember the exam. In addition to these benefits this pioneering change in the assessment for this module supports the five pillars of academic integrity. Students self and peer assess videos and submit their best attempt thus giving them responsibility for their own learning. The ability for assessors to review video recordings ensures consistency and brings transparency to the assessment. Also students can revisit video recordings with feedback and see that their grade is an honest and fair reflection of their performance thus building trust and fostering mutual respect. 

Tips for implementing this practice

Preparation is key and includes the following;

? Ensure there is a designated point-of-contact for student with queries or problems

? Ensure appropriate facilities and equipment are available

? Have good IT support and appropriate platform to allow uploading and marking videos

? Use ¡°Bitesize chunks¡±. It may be necessary to deconstruct skills to their basic components. For example, vital signs are considered one skill but students submit a video for each of its four components of temperature, pulse, respirations and blood pressure.

? Give students clear guidelines in terms of; o what is being assessed and needs to be visible on the video o the technological requirements for recording and uploading the video

? Consider providing a video demonstrating the application of the guidelines.

? Have a marking checklist that clearly defines the skill requirements and marking structure

? Consider double-marking videos

Reflections and future plans?

It was hard work to introduce this new assessment format as it was a venture into the unknown that required team effort and commitment, and buy-in and support from IT Support and Teaching Enhancement Unit. It has been extremely positively evaluated by students

¡ñ The vast majority of them preferred the online submission format.

¡ñ They valued the opportunity to review their recordings, have repeated attempts, and submit their best effort. ¡ñ They reported being less stressed, anxious and nervous.

¡ñ They liked the flexibility it offered and acknowledged the opportunity for self and peer assessment during which they could observe and reflect on their performances enabling them to identify areas for improvement and take responsibility for their own practice and learning. I also found a number of benefits that I did not anticipate;

¡ñ Students have started recording themselves during independent practice and reviewing the videos among themselves.

¡ñ In cases where students challenged their results they were able to discuss application of the theory to practice thus demonstrating higher order thinking skills.

¡ñ Students who are now doing face-to-face practical exams in other modules are requesting the option of video submission. As a result, module co-ordinators are enquiring about it. My future plans for this module assessment include;

¡ñ All first year practical exams are now by video submission. In response to student and module co-ordinator requests we will start rolling out to second years in 2018/2019.

¡ñ As more students complete the online video submission we are building a bank of data which can be analysed to compare online video submission with face to face practical exams in terms of students results and learning. ¡ñ Video submission being used for some in service training for qualified nurses in partner services showing that this is a transferrable strategy.

A useful video can be accessed here https://www.youtube.com/watch?v=Oj_FQQSzN3I&feature=youtu.be