A Good Show, But Questions Remain: Sequoia AVC Advantage Recertification Exam by PA Office of Secretary of State

by Marybeth Kuznik, VotePA

sequoiaAdvantage.jpgOctober 11, 2006 -- Entertaining showmanship was the order of the day as Pennsylvania voting system examiner Dr. Michael I. Shamos cast his eye over the Sequoia AVC Advantage and its WinEDS ballot installation and tabulation software in Harrisburg last week The testing was being redone after the WinEDS software failed so badly in March, 2006 on the AVC Advantage machine that the examination then had to be suspended in the middle. Dr. Shamos noted that Sequoia had advanced the software approximately 60 revision numbers since March, and had added features as well as claiming to have corrected problems.

Dressed in a colorful orange polka-dot tie and orange and white striped shirt, Shamos strolled about the exam room, posturing frequently in front of a large LCD projection of the screen under scrutiny newly added for those in attendance to observe. At one point he even rendered a mime of a bound Harry Houdini escaping from chains to make his point that locks on a voting machine do not equate with security. The effect was not so much of a thorough scientific examination, but rather that of a polished performance designed to show an audience how thoroughly the testing was being done.

After a rather rough beginning for Sequoia with cartridges early-on failing several times to correctly load the software, the updated WinEDS program did appear to display some improvement although questions remained. One glaring problem was that event logs could be altered to remove evidence that file changes had been made or tampering had been done. Another problem showed up when an audit trail printed out from two different memory sources in the system looked very different each time it was printed.

"We know that logically it is equivalent, but one gets nervous," said Shamos.

As usual, the exam consisted basically of checking the system against seventeen points required by Pennsylvania law, and a scripted vote of twelve ballots. Dr. Shamos moved through the tests in about four hours and fifty-five minutes of actual exam time. During that period he tested the basic functions of the software that failed in March as well as several new features. Performing his 113th examination of a voting system, Shamos was not shy about expressing his displeasure at various characteristics displayed by the Sequoia materials. He repeatedly took the Sequoia equipment, software, and in some instances personnel to task.

"If I haven't worked for Sequoia for 12 years, how would I know what to do, " Shamos pondered when a communication problem between hardware and software manifested itself. And discovering later that a screen designed to display a list of improperly altered files instead implied that they were normal, he snapped that "flashing red and a few sirens going off would certainly be nice here."

But despite the numerous difficulties that occurred during this retest, when citizen observers asked what assurance the re-exam gave that problems similar to those discovered in March would not crop up later, Shamos appeared totally confident that WinEDS was now "healthy".

"We have tested every function on every menu of the system," he said. "Everything worked today."

Notwithstanding Shamos' confidence, this observer left with a conflicting feeling that she had just seen a fine performance by a skilled showman, but many questions still remain unanswered about the stability and security of WinEDS and the Sequoia AVC Advantage.