Remote exam software is crashing when the stakes are the highest
And raising more and more privacy concerns
“Barmageddon”
In 2015,the company agreed to paymore than $2 million to settle a class action suit after software failures affected the bar exams in 43 states. At the time, the situation spawned a catchy epithet: “barmageddon.” The company declined to comment on those incidents.
And this year, two states with contracts with the proctoring company ILG Technologies, Indiana and Nevada,postponed their remote bar examsafter technical glitches were revealed during practice testing. ILG Technologies did not respond to a request for comment. Officials offering the exams in Michigan, Indiana, and Nevada also did not reply to requests for comment.
But that hasn’t stopped more and more state bar associations, along with a growing number of universities, from turning to companies that produce remote proctoring software—theUniversity of Washington,University of Missouri, and the University of California system are just a few of the many schools that have turned to the tools.
One executive from a company called Examityrecently told Inside Higher Edthat the company’s growth was up 35 percent over what was expected in its quarterly forecast. Another, from a company called Proctorio,told The Washington Posthis company would “probably increase our value by four to five X just this year.”
Even before the pandemic, the tools were being widely adopted. In 2018, Examityreported that it had seen five straight yearsof more than 50 percent growth. That growth is likely to continue unabated as schools remain remote.
The states The Markup contacted did not respond to requests for the dollar amounts of contracts with exam companies, but ExamSoft, for example, has been known to charge each exam-taker$134.50 in licensing feesas part of its contract. Normally, thousands of people take the bar around the country each year.
Tracking eyes and faces
Meanwhile, privacy advocates say the technology, which purports to track eyes, scan faces, and monitor typing, is troubling.
“A bar exam that forces our future colleagues to use invasive and unproven technology … and potentially endure discrimination is antithetical to everything the law is meant to uphold,” Albert Fox Cahn, executive director of the Surveillance Technology Oversight Project, wrote this month ina letter to New York officialsasking them to reconsider the use of ExamSoft for the state’s bar exam, citing concerns of the potential for racial bias in tools like face-scanning. (Some test-takers have recently said they facedexactly that problemwith the software—that it appeared to struggle to recognize darker features.)
ExamSoft’s Sandberg said, “[A]ll exams are first reviewed by AI, flagged for potential incidents and then also re-reviewed by professional human proctors.”
The Electronic Frontier Foundationraised similar concernsabout the California state bar exam. Both that exam and the New York exam were given remotely through ExamSoft last week.
Open Book Tests or No Tests at All?
The question is, in a pandemic’s world, what the alternatives really are.
One option, “diploma privilege,” is used in Wisconsin. It’s a process that allows some in-state law graduates to become licensed to practice without taking the bar exam. A few other states have given2020 law school graduateswho complete certain requirements such privileges, laying a path to practice as lawyers without taking the exam.
But other states have been more reluctant. In Florida, a group of law graduates filed a petition in the state’s Supreme Court courtagainstthe use of ILG software for the state’s exam this year. The graduates asked to have the exam waived for the year and to be allowed to practice after working with a licensed attorney for six months. The courtultimately shot downthe request. The Florida Board of Bar Examiners did not respond to a request for comment.
At the University of California, Santa Barbara, faculty senta letterto the administration objecting to the broad categories of data, from identifying physical details to keystrokes, used by ProctorU, another provider of software used for remote exams. “We recognize that in our collective race to adapt our coursework and delivery in good faith, there are trade-offs and unfortunate aspects of the migration online that we must accept,” the faculty wrote. “This is not one of them.” (The companythreatened legal actionagainst the faculty over the letter and said it only collects data for proctoring purposes. Use of the software is currently optional.)
“On campus, we are there to protect our students; we feel responsible for our students,” Jennifer Holt, a film and media studies professor at UC Santa Barbara and part of the faculty association, told The Markup. “That,” she said, “doesn’t change because instruction has moved online.” The use of the software isn’t required, but there’s little alternative beyond changing the format of tests.
Some experts in education question whether the problems go beyond the use of any specific software.
“This is a case where we’re trying to use the technology Band-Aid for a problem that’s much more fundamental,” said David Rettinger, a professor of psychological science at the University of Mary Washington, who studies ethics in education.
Rettinger said educators should consider relying more on different forms of testing, like open-book exams, which may better demonstrate how much students have learned. That could also create a less adversarial learning environment, he said, one that’s built more soundly on trust. It’s not exactly a silver lining, he said, since these are problems that suddenly have to be solved immediately, instead of over time.
Holt, for one, said she’d given the first open-book exam of her career recently. “These are crazy times,” she said. “We’ve got to make adjustments.”
But, she added, “we have to do it in a way that has these students in mind first.”
This article wasoriginally published on The Markupby Colin Lecher and was republished under theCreative Commons Attribution-NonCommercial-NoDerivativeslicense.
