Reliability of online assessment methods
Physical exam taking in the classroom has been established as a reliable method of learning assessment. Nevertheless, organizations have been looking into alternative online assessment methods, which might become necessary in case of an emergency, such as a pandemic or a natural disaster.
A brief review of recent national responses to a pandemic reveals that online assessment might not be the best choice in high stakes exams. For example, France decided to replace the baccalaureate exams with semester averages, which Greece decided to hold the physical panhellenic exams that control the access to higher education.
At the same time, respectable universities (e.g., Berkeley, Cambridge, Harvard) when forced to adopt online exams for a variety of their courses, they have adopted a variety of assessment methods. Moreover, those universities have chosen to grade online exams with a pass-fail, instead of a detailed grade, in order to reflect a lack of confidence in those methods.
In the following table, alternative assessment methods are organized according to reliability. In additon to the traditional final exam, we consider contemporary assessment methods, such as midterm deliverables, oral assessments, and plagiarism detection methods.
Assessment method | Reliability |
---|---|
Physical exam + midterm deliverables | Very high |
Physical oral exam | High |
Physical written exam | High |
Take home exam + midterm deliverables | Medium |
Take home exam + distance oral exam | Medium |
Take home exam + plagiarism detection | Low |
Multiple choice + camera proctoring | Low |
In conclusion, it is very difficult to achieve anything more than medium reliability in distance assessments, because there is no way to control the presence of third parties. The employment of a single distant exam is discouraged, while the employment of elaborate distance assessment methods can only achieve medium reliability. Therefore, distance exams are only suitable when the trade-off against reliability is acceptable.