Gabe Teninbaum, a professor at Suffolk University Law School, is appealing to ExamSoft to fix a serious bug with the testing software: failure to recognize faces. This is a problem that can delay test takers – or prevent them from starting their exams altogether – and according to reports, it makes people with dark skin color excessive.
ExamSoft’s software records students as they take remote exams and monitors for signs of academic dishonesty. Teninbaum’s report deals with an ExamSoft feature called ExamID, which aims to verify that test takers are who they say they are. The first time a student logs on to their exam portal, they upload a photo of themselves (their “baseline image”); then they are asked to take another selfie before embarking on future exams, checking the software against the original photo.
Research has found that face recognition algorithms consistently make more mistakes in identifying black faces than white. And while these studies did not focus specifically on ExamSoft, this does not appear to be an exception. Back in September, several non-white examiners said New York Times that the software could not identify it due to ‘poor lighting’ – a problem that Teninbaum, with light skin, could not repeat.
The @ExamSoft software cannot “recognize” me due to “poor lighting” even though I am sitting in a well-lit room. To think it has nothing to do with lighting. Pretty sure we all predicted that their face recognition software would not work for coloreds. @ DiplomaPriv4All
– Alivardi Khan (@uhreeb) 8 September 2020
@ExamSoft I FINALLY figured out MY OWN 4 hours later (still waiting for a SOFT chat representative from the exam) HOW TO GET TO MY MOCK EXAM … ONLY TO TELL THE SYSTEM CANNOT ACKNOWLEDGE. I AM IN A BOARD ROOM WITH ALL THE LIGHTS ATTACKED AND THE SUN WAS STILL. MAKE IT SENSE. pic.twitter.com/7K3yKT1y4n
– Futureesq1990 (@ futureesq1990) 17 September 2020
Early in the fall, Teninbaum tried to find a solution. He believes such mistakes add unnecessary stress to an already stressful period. “These are students who are about to take a high-stakes exam with a lot going on, and that is very unwelcome,” Teninbaum said in an interview with The edge.
“If you’re going to do an exam, you just want to focus on the exam,” he added. “You do not want to feel that you have these extra challenges.”
Teninbaum also believes that optics matter; schools owe it to marginalized students not to rely on a category of software known as discriminatory. “Students deserve to feel that their institution is doing what they can to protect their rights, interests and dignity,” he says.
In his report, which in The Journal of Robotics, Artificial Intelligence, and LawTeninbaum gives an account of the solution he found.
He suggests that schools give each student an identical generic base image. He then suggests that they should ask ExamSoft to enable ‘delayed identification’, a feature built into the software that enables students to continue with exams even if identification fails. This feature is basically hidden – it is not mentioned anywhere on ExamSoft’s website (at least not what I could find). Only ExamSoft can enable it.
Along with these adjustments, ExamSoft will identify each test taker incorrectly. But they will still be able to continue with their exams – ExamSoft then sends the selfies to the school, and instructors can manually verify all of them. “We know who our students are,” Teninbaum says. “We can make sure the students are who they say they are and avoid subjecting students to these kinds of challenges.”
He also suggests that ExamSoft make the ‘deferred identification’ feature accessible to customers. “The reporter urges ExamSoft to build it into a feature that allows institutions to easily turn on / off and thus bypass ExamID until the technology expires to one that does not discriminate,” his report reads.
Teninbaum hopes these changes will last longer than the COVID-19 pandemic, and that it can help students feel more comfortable taking remote classes. “It’s going to be a growing problem as people get more and more online for their training,” he says.
Nevertheless, he only solved part of the problem. Students had a variety of issues with the proctoring software from ExamSoft. The videos of more than 3,000 people who used the platform to take the California bar exam in October were marked for possible violations – nearly 36 percent of applicants who took the online exam. Users reported audio issues and other technical errors.
A group of six U.S. senators – including Richard Blumenthal, Elizabeth Warren and Corey Booker – wrote an open letter to ExamSoft in December highlighting the potential harm to students of color and students with disabilities.
ExamSoft did not immediately respond to a request for comment.