AI Proctoring: What You Need to Know

The global pandemic resulted in recruits and students being displaced from their institutions. Quarantine forced everyone to carry out their duties and responsibilities of daily life, from the confines of their home. This included work and examinations too. Educational and professional organisations were faced with the issue of conducting exams to test potential candidates.

Doing so in person would risk infecting every individual present and thus spreading the virus, while doing it online meant cutting the spread of the virus but tampering with the credibility for the test, in order words cheating! There were advocates for the restructuring or complete removal of tests in order to accommodate for this shortcoming but the issues arising from this became a burden.

Many institutions were reliant upon their examination patterns for testing the calibre of their candidates and upheaving it all would take a considerable amount of time. This led to the inevitable delay of examinations and job placements throughout the world.

Subsequently, companies began recalling placement offers and colleges postponed graduations. There seemed to be no way to reliably gauge how skilled one was in fields that required displaying said skill for selection. The economy was suffering from an inability to satisfy the growing demand in the job market, with an oversupply of candidates unable to prove their capabilities to potential employers.

Here is where we witnessed the entry and rise of AI-based Proctors.

AI Proctoring, also known as automated proctoring, came to the rescue of recruiters and educators alike by promising a reliable method of testing candidates while ensuring that no malpractices would ensue.

Wait, what are AI proctors?

Consider this your very own online invigilator! An AI proctor is essentially an algorithm, trained using data on the behavioural patterns of individuals taking tests. When paired with the live feed of a test-taker, the algorithm compares the examinee's behaviour with what it considers as “normal” test-taking behaviour.

In case of any deviation from the norm, the algorithm tracks or “flags” the behaviour and informs the examiners. This combined with machine learning means that the algorithm gets better over time as it gathers more and more data on behavioural patterns.

The algorithm needs access to the examinee's screen and microphone as well as their camera. It tracks visual data and any sounds audible in the room, as well as all virtual information that the examinee has displayed on their screen. These reduce malpractices during tests.

How does it track behaviour for unethical practices?

• Eye tracking: The proctoring software uses the examinee’s camera to track their eye movements. It can ascertain whether they are looking away from the screen or at something out of the camera’s visual range. The algorithm can also track the motion of the eyes for specific indicators in the patterns of its motion. This allows it to gauge their behaviour and scrutinise it for consistency with regular test-takers.

• Visual cues: The AI proctor uses the camera for much more than just eye tracking. It can use facial recognition software to determine whether the person giving the test is the same as the one registered to do so. It can also track any items present in the visual field of the camera such as smartphones, calculators, and books, and check for the presence of additional people in the room.

• Audio cues: The algorithm tracks the sounds present in the room using the examinee’s microphone and compares it to its database of speech patterns to gauge the presence of any suspicious activity.

• Virtual Cues: An auto proctor will also request access for screen sharing, which allows it to scan the information displayed in front of the examinee and determine if the test is all they have open at the time. It can also be coded to not allow for the switching of tabs, copy-pasting and file sharing. This helps in cases where examinees might try to open another tab to research or access resources that are not intended to be used during the assessment. Any unnecessary changes or attempts to switch out of the exam will be reported to the proctor in charge. Like we said! You’re very own online invigilator!

What's wrong with human proctors?

Human proctors are effective when there are multiple examinees in one consolidated location. The need for a one-to-one ratio of invigilator to the examinee is removed, and the presence of a perceived authority figure dissuades any potential misconduct.

However, when faced with the dilemma of online tests, human invigilation falls behind for a few reasons. As individuals are not present in person, every second of their exam needs to be monitored. Constantly tracking student behaviour is challenging, regardless of whether a live feed or a recording is used, as momentary lapses of attention or distractions can dramatically impact the results of the test.

Auto proctors are not replacing human proctors. At least not yet. An auto proctor in effect makes the process of proctoring much more like having an invigilator for every student who reports any potential wrongdoings to the one in charge; (i.e. the human proctor) for confirmation or punishment.

It's a simple numbers game really We can take an example to better understand the magnitude of the benefit of using AI proctoring. We assume that there are twenty examinees taking a one-hour test in person with one proctor. The proctor takes one hour to invigilate the exams of all twenty participants. Now if this were to be an online exam, the twenty examinees translate into twenty one-hour recordings to be reviewed or twenty individual screens that a single proctor has to monitor while ensuring that none engage in any malpractice.

The former requires nearly a whole day to verify what initially consumed a single hour. The latter requires the proctor to essentially be on a group video call with twenty people while listening and paying attention to all of them at the same time. This is practically impossible for a single person, and the only solution would be to increase the number of proctors per exam. Here's where AI proctoring can step in.

As it is already trained to know what to look for, an AI merely requires adequate processing power to effectively monitor as many examinees as desired. An AI can reliably monitor multiple examinees simultaneously and let the invigilator take over if there's any dubious behaviour.

This cuts down on both time and costs for the examiners while maintaining the credibility of the test. Auto proctor services also provide additional benefits such as streamlining the tally and collection of test answers while also disclosing the results to the candidates extremely conveniently.

Are there any drawbacks?

Auto proctors, while extremely useful, are still developing in their versatility and accessibility. Using an auto proctor is not as simple as sitting back and letting the algorithm do all the work as the most severe “punishment” that they are assigned is a cancellation of the test unless an examiner intervenes. Thus an entirely AI-proctored test would only lead to a semi-credible test at best.

Currently, the major issues faced by auto proctors include:

• Concerns regarding data collected: Certain automated proctors came under fire for the collection of student data without disclosing how that data would be ultimately used. This raised concerns because it included sensitive information including biometric data, home addresses, phone numbers, etc. Many trusted AI proctor services do provide a breakdown of how the data collected is used and kept anonymous, but due to the lack of proper legislation, it makes sense that many might be wary of online proctors.
• False flagging: At the current stage, false flagging seems to be the biggest issue that plagues the algorithm. Bad lighting, unavoidable noise and slow devices can hamper the test by randomly flagging the participant for activities that the AI interprets as unethical behaviour. In reality, this can be something as innocent as a yawn from the examinee. If an examinee is flagged multiple times in a short period of time their test might be automatically cancelled. Therefore, human intervention is still needed to some degree. However, this should solve itself over time as the algorithm becomes more adept at gauging examinee behaviour.
• Lack of infrastructure: There are still many parts of the world where internet connectivity and access to the tools needed for AI proctoring are limited. Rural populations throughout the world suffer from a lack of infrastructure that prevents them from utilising these tools to their full extent. Having a consistent internet connection can be out of control for many, which means that appearing for an auto-proctored test would mean taking on the risk that if the connectivity goes down then so do your chances of passing the exam.

Auto proctoring in Different Industries

With the worldwide pandemic, we saw a boost in the number of online courses from prestigious universities and organisations that were available to the general public as well as an increase in the percentage of people that were enrolling in these courses.

A portion of these courses provided certification to those who passed them. To ensure the credibility of these courses, modules or sections of the course involved AI-based proctors to certify that no unfair means were employed in the attainment of the certification. AI proctors come with uses beyond proctoring exams. They have carved a niche in the domain of work-from-home employment where they are ideal for monitoring employees working out of the office.

They can track employee engagement by charting the time they spend on their home devices working on assignments and projects. It can also track whether the employee is at the device, paying attention or on another window doing anything apart from work.

This is usually integrated into the software installed on their devices so it can send them notifications, and log their hours worked. These are usually set to record employee screens once they sign in during their expected working hours. AI is also being implemented in medical remote proctor services. Remote proctor services have existed in the medical sector for a while. They allowed consultants from around the world to provide their expertise for medical cases remotely, which was crucial for time-sensitive cases where a patient would not have had the time to wait for the consultant to arrive.”

ExplORer Surgical” is such a remote proctor service, that is applying AI to assist the consultant by recording and analysing a video feed during a procedure, and providing an overview of the entire operating room. This includes the medical supplies present, data presented in medical charts and the equipment in use, to name a few.

In summary AI Proctoring is set on a growth path that will continue as long as there is a need for remote monitoring or assessment. By eliminating the issue of distance between the test-giver and the test-taker, we can expect auto proctors to lead the change towards greater accessibility of jobs and skills throughout the global market.

With an entry into the medical sector, auto proctors are also poised to help save lives all over the world. There are definitely drawbacks, as they are not absolute in their accuracy and do not entirely eliminate the need for a human proctor. However, they do also provide a slew of benefits that assist the human proctor in carrying out their duties much more effectively. By now you know what AI proctoring is, the essence of how it functions, what the pros and cons are, and the various uses it can have.

Do you think this is the direction the future is headed in a post-pandemic world? Or do you believe in a few years we will outgrow the need for these services and human proctors will solely remain the norm as we return to the pre-pandemic routine of in-site office work and offline examinations?