Sequoia Advantage Recertification Exam

by Steve Strahs, Montgomery County Election Reform Network

October 11, 2006 -- At long last, just a month before the election and after at least three postponements, Sequoia came in for its recertification test last week in Harrisburg for the Advantage AVC vote machine and - especially - its Win EDS software.  Win EDS, the ballot installation and vote tabulation software, had flunked the test last March and wasn't used to tabulate the votes for the May primary.  This time around the examination went much more smoothly for Sequoia, although there were still some problems.   Marybeth Kuznik and I attended, wearing our white hats.

All in all, though, examiner Michael Shamos, while not issuing a recommendation on-the-spot, seemed pleased and praised Sequoia for the enormous effort that he said went into making the software improvements.  More tellingly, perhaps, was the cheery disposition of the four Sequoia staffers, including VP Paul Nulty, upon conclusion of the exam.  Despite some pretty anxious moments, the Sequoia people and the Secretary of State's office staffers walked out happy.  Shamos estimates his official report will take about two weeks to issue, but Montgomery County has already started upgrading the machines, downloading the software and starting on the Logic and Accuracy tests.

Two big questions loom: 1) How significant are the problems that did arise (see below) and when and how will they be addressed? 2) What about the questions that Shamos didn't ask?  On the latter, election integrity activists know very well that experts have been able to demonstrate how to hack into some of the same vote machines that Shamos has approved for PA.  Then when these security threats surface based on vulnerabilities that Shamos never tested for, he professes shock and grave concern.  Or he has the SoS issue some directive with apparent little effect.  This report - highly non-technical - is in the interest of keeping the process as open as possible and helping others who are taking on such questions.

Another issue, before turning to the problems that arose on Wednesday, is the defects that Sequoia admits they found before making the improvements.  The most significant that they mentioned seemed to be their admission of long-standing problems related to straight party voting.  That is, when a voter chose a straight party vote for all offices and then wrongly went back and also chose a specific candidate from the other party - inconsistent with his straight party choice - the entire straight party vote was nullified. Yet there was no way for the voter to know that.  That's because the light that went on over the straight party choice when the voter selected it never went off!  According to Sequoia, this is a defect that may well have existed since the machine was first introduced or at least back to 1990.  Other problems Sequoia says they fixed that were never detected in certification exams involved the write-in option switch and a "boot light" turning on before the machine would allow the voter to start. 

What is disturbing about these problems, especially the one involving the straight party vote, is that the Montgomery County Election Board apparently wasn't any more interested in uncovering them over the life of these machines than was Sequoia.  Montco has never done an undervote analysis for their machines or any other type of performance evaluation of them, as far as is known, and has no interest in doing so.  This is despite the latest research that says that full-face DRE systems are plagued by "an unacceptably high residual vote rate." (Brennan Center, The Machinery of Democracy, 2006)

After a promise at a public meeting prior to the primary that they would do a report on the overall functioning of the May election, the Montgomery County Department of Voter Services issued almost 5 months later a 12-line table indicating phone calls received by election district and time-of-day with three or four-word problem identifications next to each.  That was it for the research and performance evaluation for a countywide election. 

One issue raised by Shamos at the Wednesday exam was how to ensure that the firmware on any machine corresponds to what was actually tested and approved by the ITA.  There was talk of a tamper-proof seal, which may be required by the feds in the future, but, basically, Sequoia was stumped.  Shamos was also "astounded" at the level of difficulty in determining what files are actually in a machine and actually dumping out what shouldn't be there. 

A problem that took considerable time was the inability to successfully insert ballot installation software.  It took three different cartridges and a number of unusual error messages to load the software and there was talk of communication errors between  the operating system and the path find for removable devices.  At one point Shamos asked, "If I haven't worked for Sequoia for 12 years (referring to one of the Sequoia staffers), how would I know what to do?"   Sequoia was supposed to come up with an explanation for the problem later on, but we never heard it.

When he changed vote tallies by manipulating WinEDS, Shamos noted that a "tally validation" window, a new function, showed that numbers had, indeed, been changed.  However, its presentation was so innocuous that it seems fair to say that unless you were looking for that precise information, it could be easily overlooked.  There was not even a warning, red alert or anything that made it clear that the security of the system had been breached.  A siren was suggested, jokingly.   After his manipulations, Shamos went back to see if he could "cover his tracks" by erasing evidence on the activity log.   In fact, there were no tracks to cover - no evidence of any tampering on the log.  Shamos suggested that a mechanism to detect such acts should be applied to the event log.  You think?

Overall, Shamos seemed confident that there was adequate security to foil tampering with the tabulations of the kind that he tested for last March.  Unfortunately, I can't say how he arrived at that conclusion.  He went quickly and I may have missed it. 

One of the most significant issues raised - just discussed rather than exhibited - was the question of the security of the Local Area Network used by Montgomery County.  Shamos briefly described a threat scenario: It's hectic on election night.  Someone gets a copy of WinEDS and does manual entry of votes and sends it to the data base server.  When the cartridge is inserted, it can't be read, but it is assumed that the cartridge was read earlier so it is overlooked.  What's to prevent this?  How is the problem uncovered?    Shamos asked Director of Voter Services Joe Passarella to have the County IT Director call him to talk about what's on the network and what the protocols are.   Question: how can this "test" be interpreted if it is discussed only in a private phone call?   Also, anyone know if this question was ever asked in other exams? 

One question that I raised in the brief Q&A at the end was about the keys for getting into the machines.  I pointed to the problem with the Diebold machines and asked in a general way how secure the locks are in back of the machines.  Shamos responded with a round-about explanation involving Houdini's escapades (no kidding), something about since he was all tied up, you know that that couldn't have had any impact on his ability to escape.  Entertaining, but what it came down to, as far as I could tell, was that, heck, we're not really putting the trust of our democratic franchise in crumbly little keys that are probably available all over the place.  Not exactly persuasive or reassuring, but probably in keeping with Shamos's stated conviction that the real threats to the security of the electronic vote stem from insiders, especially people with authority and access.  I wonder what Voter Services Director Passarella thought about that comment!