AI Proctoring: What You Need to Know

The global pandemic resulted in recruits and students being displaced from their institutions. Quarantine forced everyone to carry out their duties and responsibilities of daily life, from the confines of their home. This included work and examinations too. Educational and professional organisations were faced with the issue of conducting exams to test potential candidates. 

Doing so in person would risk infecting every individual present and thus spreading the virus, while doing it online meant cutting the spread of the virus but tampering with the credibility for the test, in order words cheating! There were advocates for the restructuring or complete removal of tests in order to accommodate for this shortcoming but the issues arising from this became a burden. 

Many institutions were reliant upon their examination patterns for testing the calibre of their candidates and upheaving it all would take a considerable amount of time. This led to the inevitable delay of examinations and job placements throughout the world. 

Subsequently, companies began recalling placement offers and colleges postponed graduations. There seemed to be no way to reliably gauge how skilled one was in fields that required displaying said skill for selection. The economy was suffering from an inability to satisfy the growing demand in the job market, with an oversupply of candidates unable to prove their capabilities to potential employers. 

Here is where we witnessed the entry and rise of AI-based Proctors. 

Wait, what are AI proctors?

Consider this your very own online invigilator! An AI proctor is essentially an algorithm, trained using data on the behavioural patterns of individuals taking tests. When paired with the live feed of a test-taker, the algorithm compares the examinee's behaviour with what it considers as “normal” test-taking behaviour. 

In case of any deviation from the norm, the algorithm tracks or “flags” the behaviour and informs the examiners. This combined with machine learning means that the algorithm gets better over time as it gathers more and more data on behavioural patterns. 

The algorithm needs access to the examinee's screen and microphone as well as their camera. It tracks visual data and any sounds audible in the room, as well as all virtual information that the examinee has displayed on their screen. These reduce malpractices during tests. 

What is AI Proctoring?

AI proctoring, also known as remote proctoring with artificial intelligence, is a technology-enabled approach used to monitor and invigilate online exams, assessments, or tests. It employs artificial intelligence (AI) algorithms and various technologies to ensure the integrity and security of the examination process, even when conducted remotely.

AI Proctoring, also known as automated proctoring, came to the rescue of recruiters and educators alike by promising a reliable method of testing candidates while ensuring that no malpractices would ensue. 

What are the Features of AI Proctoring?

AI proctoring solutions typically involve the following features:

- Facial Recognition: AI proctoring software uses facial recognition technology to verify the identity of the test-taker. It captures the test-taker's face during the authentication process and then continuously monitors the face throughout the exam to ensure the same person is taking the test.

- Browser Lockdown: To prevent cheating, the AI proctoring software may lock down the test-taker's browser, limiting their ability to access external websites or applications during the exam.

- Screen Monitoring: AI algorithms monitor the test-taker's screen to detect any suspicious activities, such as switching between windows, opening unauthorized applications, or attempting to copy-paste.

- Keystroke Analysis: Some AI proctoring systems analyze the test-taker's keystrokes to detect unusual typing patterns that may indicate cheating or the presence of a second person.

- Audio Analysis: The software may also monitor audio to detect any unusual sounds or voices in the background that could indicate unauthorized communication.

- Real-time Monitoring: AI proctoring tools provide real-time monitoring, where human proctors or instructors can intervene if they detect any suspicious behavior through the AI system's alerts.

- Recorded Sessions: In addition to real-time monitoring, AI proctoring systems often record the entire exam session. These recordings can be reviewed later for further analysis and evidence if any irregularities are suspected.

How does AI Proctoring track behaviour for unethical practices?

- Eye tracking: The proctoring software uses the examinee’s camera to track their eye movements. It can ascertain whether they are looking away from the screen or at something out of the camera’s visual range. The algorithm can also track the motion of the eyes for specific indicators in the patterns of its motion. This allows it to gauge their behaviour and scrutinise it for consistency with regular test-takers.

- Visual cues: The AI proctor uses the camera for much more than just eye tracking. It can use facial recognition software to determine whether the person giving the test is the same as the one registered to do so. It can also track any items present in the visual field of the camera such as smartphones, calculators, and books, and check for the presence of additional people in the room.

- Audio cues: The algorithm tracks the sounds present in the room using the examinee’s microphone and compares it to its database of speech patterns to gauge the presence of any suspicious activity.

- Virtual Cues: An auto proctor will also request access for screen sharing, which allows it to scan the information displayed in front of the examinee and determine if the test is all they have open at the time. It can also be coded to not allow for the switching of tabs, copy-pasting and file sharing. This helps in cases where examinees might try to open another tab to research or access resources that are not intended to be used during the assessment. Any unnecessary changes or attempts to switch out of the exam will be reported to the proctor in charge. Like we said! You’re very own online invigilator! 

What's wrong with human proctors?

Human proctors are effective when there are multiple examinees in one consolidated location. The need for a one-to-one ratio of invigilator to the examinee is removed, and the presence of a perceived authority figure dissuades any potential misconduct. 

However, when faced with the dilemma of online tests, human invigilation falls behind for a few reasons. As individuals are not present in person, every second of their exam needs to be monitored. Constantly tracking student behaviour is challenging, regardless of whether a live feed or a recording is used, as momentary lapses of attention or distractions can dramatically impact the results of the test. 

Auto proctors are not replacing human proctors. At least not yet. An auto proctor in effect makes the process of proctoring much more like having an invigilator for every student who reports any potential wrongdoings to the one in charge; (i.e. the human proctor) for confirmation or punishment. 

It's a simple numbers game really We can take an example to better understand the magnitude of the benefit of using AI proctoring. We assume that there are twenty examinees taking a one-hour test in person with one proctor. The proctor takes one hour to invigilate the exams of all twenty participants. Now if this were to be an online exam, the twenty examinees translate into twenty one-hour recordings to be reviewed or twenty individual screens that a single proctor has to monitor while ensuring that none engage in any malpractice. 

The former requires nearly a whole day to verify what initially consumed a single hour. The latter requires the proctor to essentially be on a group video call with twenty people while listening and paying attention to all of them at the same time. This is practically impossible for a single person, and the only solution would be to increase the number of proctors per exam. Here's where AI proctoring can step in. 

As it is already trained to know what to look for, an AI merely requires adequate processing power to effectively monitor as many examinees as desired. An AI can reliably monitor multiple examinees simultaneously and let the invigilator take over if there's any dubious behaviour. 

This cuts down on both time and costs for the examiners while maintaining the credibility of the test. Auto proctor services also provide additional benefits such as streamlining the tally and collection of test answers while also disclosing the results to the candidates extremely conveniently. 

AI Proctoring in Recruitment

The main objective of AI Proctoring in recruitment is to ensure the integrity and fairness of the assessment, as well as to streamline the evaluation process for recruiters. Let's delve into its applications and benefits, including relevant statistics.

1. Remote Proctoring for Online Assessments:

AI Proctoring is extensively used in remote recruitment processes, particularly when candidates need to take assessments from their own locations. It helps to prevent cheating and plagiarism by closely monitoring candidates through webcam feeds and audio recordings. AI algorithms can detect suspicious behaviors such as looking away from the screen, using unauthorized resources, or interacting with other devices.

According to a report by Mercer Mettl, a leading talent assessment platform, remote proctoring can significantly reduce instances of cheating during online assessments. Their study found that implementing remote proctoring reduced cheating by 96% in comparison to traditional unsupervised online assessments. 

2. Automated Video Interviews:

AI Proctoring is used to conduct automated video interviews, where candidates respond to pre-recorded questions. The technology analyzes candidates' facial expressions, tone of voice, and verbal responses to provide insights into their personality traits, communication skills, and overall fit for the role.

A study by HireVue, a prominent AI-driven assessment platform, reported that candidates evaluated through AI-powered video interviews had a 90% favorable impression of the company, showcasing the positive candidate experience AI Proctoring can offer. 

3. Analyzing Written Responses:

AI Proctoring can assess candidates' written responses to various questions and assignments. It employs natural language processing (NLP) algorithms to evaluate the content, grammar, and overall quality of the written answers. This technology streamlines the grading process and provides consistent and unbiased evaluations.

According to a survey by EdTech Magazine, educators found that AI tools for grading written responses significantly reduced the time spent on grading by 95%. 

4. Behavioral and Cognitive Assessments:

AI Proctoring can analyze candidates' cognitive abilities and behavioral traits to predict their job performance and cultural fit within an organization. By employing gamified assessments, virtual simulations, and psychometric tests, AI can provide in-depth insights into candidates' strengths and areas for development.

A study by IBM found that behavioral and cognitive assessments using AI technology improved the accuracy of predicting job performance by 90%. 

Benefits of AI Proctoring in Recruitment

- Increased Assessment Security: AI Proctoring helps reduce instances of cheating and ensures the authenticity of candidates' responses during online assessments.
- Time and Cost Efficiency: Automated assessments save time and resources for recruiters, enabling them to evaluate a larger pool of candidates efficiently.
- Enhanced Candidate Experience: AI-driven assessments provide a seamless and interactive experience for candidates, increasing their engagement with the hiring process.
- Data-Driven Decision Making: AI Proctoring generates comprehensive data insights, empowering recruiters to make more informed and objective hiring decisions.

Are there any drawbacks?

Auto proctors, while extremely useful, are still developing in their versatility and accessibility. Using an auto proctor is not as simple as sitting back and letting the algorithm do all the work as the most severe “punishment” that they are assigned is a cancellation of the test unless an examiner intervenes. Thus an entirely AI-proctored test would only lead to a semi-credible test at best. 

Currently, the major issues faced by auto proctors include: 

- Concerns regarding data collected: Certain automated proctors came under fire for the collection of student data without disclosing how that data would be ultimately used. This raised concerns because it included sensitive information including biometric data, home addresses, phone numbers, etc. Many trusted AI proctor services do provide a breakdown of how the data collected is used and kept anonymous, but due to the lack of proper legislation, it makes sense that many might be wary of online proctors. 

- False flagging: At the current stage, false flagging seems to be the biggest issue that plagues the algorithm. Bad lighting, unavoidable noise and slow devices can hamper the test by randomly flagging the participant for activities that the AI interprets as unethical behaviour. In reality, this can be something as innocent as a yawn from the examinee. If an examinee is flagged multiple times in a short period of time their test might be automatically cancelled. Therefore, human intervention is still needed to some degree. However, this should solve itself over time as the algorithm becomes more adept at gauging examinee behaviour. 

- Lack of infrastructure: There are still many parts of the world where internet connectivity and access to the tools needed for AI proctoring are limited. Rural populations throughout the world suffer from a lack of infrastructure that prevents them from utilising these tools to their full extent. Having a consistent internet connection can be out of control for many, which means that appearing for an auto-proctored test would mean taking on the risk that if the connectivity goes down then so do your chances of passing the exam. 

Auto proctoring in Different Industries

With the worldwide pandemic, we saw a boost in the number of online courses from prestigious universities and organisations that were available to the general public as well as an increase in the percentage of people that were enrolling in these courses. 

A portion of these courses provided certification to those who passed them. To ensure the credibility of these courses, modules or sections of the course involved AI-based proctors to certify that no unfair means were employed in the attainment of the certification. AI proctors come with uses beyond proctoring exams. They have carved a niche in the domain of work-from-home employment where they are ideal for monitoring employees working out of the office. 

They can track employee engagement by charting the time they spend on their home devices working on assignments and projects. It can also track whether the employee is at the device, paying attention or on another window doing anything apart from work. 
 

This is usually integrated into the software installed on their devices so it can send them notifications, and log their hours worked. These are usually set to record employee screens once they sign in during their expected working hours. AI is also being implemented in medical remote proctor services. Remote proctor services have existed in the medical sector for a while. They allowed consultants from around the world to provide their expertise for medical cases remotely, which was crucial for time-sensitive cases where a patient would not have had the time to wait for the consultant to arrive.” 

ExplORer Surgical” is such a remote proctor service, that is applying AI to assist the consultant by recording and analysing a video feed during a procedure, and providing an overview of the entire operating room. This includes the medical supplies present, data presented in medical charts and the equipment in use, to name a few. 

AI Proctoring is set on a growth path that will continue as long as there is a need for remote monitoring or assessment. By eliminating the issue of distance between the test-giver and the test-taker, we can expect auto proctors to lead the change towards greater accessibility of jobs and skills throughout the global market. 

With an entry into the medical sector, auto proctors are also poised to help save lives all over the world. There are definitely drawbacks, as they are not absolute in their accuracy and do not entirely eliminate the need for a human proctor. However, they do also provide a slew of benefits that assist the human proctor in carrying out their duties much more effectively. By now you know what AI proctoring is, the essence of how it functions, what the pros and cons are, and the various uses it can have. 

Do you think this is the direction the future is headed in a post-pandemic world? Or do you believe in a few years we will outgrow the need for these services and human proctors will solely remain the norm as we return to the pre-pandemic routine of in-site office work and offline examinations?