Sequoia Voting Systems had been one of the "big three" e-voting providers, along with Diebold (Premiere) and ES&S. All three companies were notorious for massive amounts of secrecy and many, many, many reports of faulty machines with weak security. Sequoia's biggest problem -- which showed up in election after election after election after election -- was that it seemed to count the votes differently every time. That seems like a rather big flaw. The company also threatened computer security expert Ed Felten after the State of New Jersey asked him to look at Sequoia's code.
Just last week there were reports that Sequoia had accidentally revealed some of its source code -- but this week Sequoia has surprised a lot of people by announcing plans for a new e-voting system which will have open source e-voting software included. The code will be released to the public next month.
This is definitely a big (and surprising) step forward. The Wired link above tries to speculate why -- but I'd argue the most obvious reason (not mentioned in that article) is that Sequoia's two largest competitors, ES&S and Diebold/Premiere merged last month, suddenly making Sequoia a much smaller player in the space (I believe it was already the number three player...). Going open source isn't just a way to improve its code and improve trust in the machines, but also a way to stand out against a much larger competitor.
For years, the big e-voting firms have refused to share their source code, repeatedly insisting all sorts of awful things would happen if the code was revealed. Of course, in the few instances where people actually did get access to the code, the only "awful things" that turned up were pretty massive security holes and weak programming. However, it looks like Sequoia may have inadvertently revealed its source code (found via Slashdot) due to an incompetent attempt to "remove" trade secret info:
The Election Defense Alliance filed a public records request under California law for a copy of the final election databases from recent elections in Riverside County California. Riverside coughed them up, after sending them first to Sequoia for "redaction of trade secrets" and forcing EDA to pay a substantial amount for this "service."
As near as we can tell, instead of stripping out proprietary stuff of any sort, Sequoia simply committed vandalism: they stripped the Microsoft SQL header data off the top, expecting that this would ruin access to the data under any possible database utility and making the contents unreadable. [Note: confirming this is a high-priority task!]
While they succeeded in ruining the files as data, they didn't realize what a Linux user could do with the "strings" command: strip out unreadable characters and leave everything left as readable plain text. This in turn revealed thousands of lines of Microsoft SQL code that appear to control the logical flow of the election.
So now there's a project underway to analyze the code, which can't make Sequoia very happy. But what may be even more interesting is that the folks hosting the code are suggesting that the way Sequoia buried its code in data files may violate federal election law concerning e-voting systems.
It violates the federal rulebook on voting systems on several levels: the rules require that code be hash-checked to prove authenticity in the field for obvious reasons. If the real working code is buried in with the data, no such hash-checks are possible. The federal rulebook is also clear that code can't be interpreted, apparently to avoid modification "in the field" (generally county or city election offices). There is also a rule barring "machine generated code" and since these data files are allegedly created (and managed) by the WinEDS application, the code in these files has to be "machine generated"?
That can't be good. Though it might further explain the resistance to ever sharing the code.
You may recall earlier this month that a judge in New Jersey barred some researchers from releasing their report into the security vulnerabilities found in e-voting machines from Sequoia that were being used in the state. Sequoia had fought hard to stop the research from even being done in the first place, let alone released, even threatening the researchers with lawsuits. Now, one of the researchers who did the research, Andrew Appel, has released a long report detailing a ridiculous number of security problems with Sequoia's machines. To be honest, it's not clear from the blog post about the report if this is the same one that's being suppressed or not, but it's pretty damning. Because this is an important issue that doesn't necessarily get enough attention, I'm reposting Appel's executive summary of just how screwed up these machines are:
Executive Summary
I. The AVC Advantage 9.00 is easily "hacked" by the installation of fraudulent firmware. This is done by prying just one ROM chip from its socket and pushing a new one in, or by replacement of the Z80 processor chip. We have demonstrated that this "hack" takes just 7 minutes to perform.
The fraudulent firmware can steal votes during an election, just as its criminal designer programs it to do. The fraud cannot practically be detected. There is no paper audit trail on this machine; all electronic records of the votes are under control of the firmware, which can manipulate them all simultaneously.
II. Without even touching a single AVC Advantage, an attacker can install fraudulent firmware into many AVC Advantage machines by viral propagation through audio-ballot cartridges. The virus can steal the votes of blind voters, can cause AVC Advantages in targeted precincts to fail to operate; or can cause WinEDS software to tally votes inaccurately. (WinEDS is the program, sold by Sequoia, that each County's Board of Elections uses to add up votes from all the different precincts.)
III. Design flaws in the user interface of the AVC Advantage disenfranchise voters, or violate voter privacy, by causing votes not to be counted, and by allowing pollworkers to commit fraud.
IV. AVC Advantage Results Cartridges can be easily manipulated to change votes, after the polls are closed but before results from different precincts are cumulated together.
V. Sequoia's sloppy software practices can lead to error and insecurity. Wyle's Independent Testing Authority (ITA) reports are not rigorous, and are inadequate to detect security vulnerabilities. Programming errors that slip through these processes can miscount votes and permit fraud.
VI. Anomalies noticed by County Clerks in the New Jersey 2008 Presidential Primary were caused by two different programming errors on the part of Sequoia, and had the effect of disenfranchising voters.
VII. The AVC Advantage has been produced in many versions. The fact that one version may have been examined for certification does not give grounds for confidence in the security and accuracy of a different version. New Jersey should not use any version of the AVC Advantage that it has not actually examined with the assistance of skilled computer-security experts.
VIII. The AVC Advantage is too insecure to use in New Jersey. New Jersey should immediately implement the 2005 law passed by the Legislature, requiring an individual voter-verified record of each vote cast, by adopting precinct-count optical-scan voting equipment.
Remember the election mess in Palm Beach, Florida from last month? The one where votes seemed to be randomly disappearing, and each recount came up with different results? Originally the blame was put on the fact that different scanning machines from e-voting firm Sequoia, would somehow count the votes differently. That seemed scary enough, and Sequoia protested, insisting that it was all human error. However, when human errors happen every time the machines are used, it's time to suggest that the real problem is with the machines.
Wired is running a long, and somewhat scary, report about the ongoing situation in Palm Beach, where every time the votes are counted, a different vote count comes out. A test was set up by the local newspaper to scan a sampling of ballots, and every time the results of those tests were different -- sometimes in extreme ways. Quite often, the machines seemed to count perfectly marked ballots as invalid, while at other times it accepted votes from invalid ballots. In other words, the machines basically don't work. And we're relying on them in many areas for the election coming up in a month. Isn't that comforting?
from the it's-not-like-we-have-an-election-coming-up-that-use-these-machines dept
You may recall that earlier this year, after some serious problems were discovered with Sequoia's e-voting machines in New Jersey, that the state asked a group of independent researchers to investigate the machines and prepare a report. Sequoia threatened to sue the researchers though. Luckily, a court allowed the researchers to investigate the machines, and said that 30 days after the court had received the report, it could be released. However, Sequoia, in its usual "It can't be our fault, no, no!" fashion, has convinced the judge to suppress the report.
Despite the fact that we're a month away from an election that will use these machines that time and time again have been shown to have problems accurately and reliably counting votes, no one is allowed to see the report. Voters in New Jersey won't be told the results of the report until after it's too late to request absentee ballots. As the head researcher on the report notes, even New Jersey's governor and secretary of state are not allowed to read the report and use it to make public policy decisions that would more likely create a fair election. For so many years now, the e-voting companies have dismissed concerns, blocked attempts to investigate, threatened investigators and almost never admitted any fault, despite tons and tons of evidence that the machines simply do not work that well. It's a travesty that this report is being suppressed.
from the it's-not-like-we've-got-computers-that-can-count dept
You know, the one thing that computers are supposed to be good at is counting things accurately. So why is it so hard to do so when it comes to counting votes? We recently wrote about the case in Washington DC's primaries where election officials were struggling to figure out the source of an awful lot of votes for a non-existent write-in candidate. Sequoia, the makers of the e-voting machines in question, were quick to deny any and all responsibility with the hilariously "thou dost protest too much" statement: "There's absolutely no problem with the machines in the polling places. No. No."
Either way, it appears that officials in DC still can't properly add up the votes properly, and are noting that 13 separate races all show the exact same number of overvotes: 1,542, though no one can explain why. Sequoia continues to stand by its original statement that the problem must be one of human error -- though it fails to explain how simple human error would create 1,542 extra votes in 13 entirely separate races -- and why it didn't design a system that would prevent the ability for "human error" to create such votes.
Last week, we wrote about yet another problem with Sequoia e-voting equipment where the company was vehemently denying the problem was with the machines, even saying: "There's absolutely no problem with the machines in the polling places. No. No." Of course, this came right after a report revealing how easy it was to hack their machines, as well as numerous other problems with Sequoia machines. Yet the company consistently employs the same exact strategy: it couldn't possibly be the fault of the machines.
You may recall the story earlier this month about the Sequoia optical scanning machines in Palm Beach County that supposedly couldn't reach the same vote tally if different counting machines were used. At least that was the original claim -- but it was later changed when election officials admitted they had simply misplaced some ballots. Well, the latest report claims that the recount is now not showing lost ballots -- it's showing too many ballots. Fantastic. Election officials think they've traced the problem to the fact that some votes on Sequoia's e-voting machine cartridges weren't properly transferred, which kicks off Sequoia's standard PR response:
The company's representative, Phil Foster says "the cartridge is fine. Why it didn't read I do not know," suggesting another human error made on election night.
You know, when you keep saying that, and the problems keep occurring, at some point, people are going to stop believing you. Even if the problem really is human error every one of these times, people might begin to wonder why you don't design your systems to avoid such human errors.
It seems like every few months, well respected security researchers come out with yet another report about just how insecure various e-voting machines are. The amazing thing is how hard the various e-voting companies have fought against allowing these researchers to look at their machines, always insisting that the federal certification process (the one that's were later shown to have not done a very good job testing the machines) was fine. Of course, even the Government Accountability Office has admitted that the federal certification process sucks.
One of the complaints that the e-voting firms have had about having independent security researchers testing the machines is that those tests are not in real world conditions. In fact, we had a commenter from one of the e-voting companies who insisted that these independent tests were useless because:
The point people often miss, which is left off of the conspiracy blogs, is that all of these 'hacking' attempts that are requested are made to do so in some sort of vacuum. In some obscure room where a gang of hackers get together and try to penetrate the system with unlimited resources. In any election, paper or fully electronic, there are procedural and security measures taken that complement and supplement the security features of the system itself. This is in addition to internal and system-independent, pre- and post-election audit features.
That's really rather meaningless, because if it were true, then that info would also come out in those independent research reports. However, even that comment turns out to be untrue. As a few folks have submitted, some security researchers at UCSB have demonstrated not just how insecure Sequoia's e-voting systems are, but they've shown how easy it is to hack an election with a pair of videos that you can watch right here (if you're in the RSS feed, click through to see them):
What this shows is that the hack that the researchers shows demolishes that comment from the insider. All it required was for those wishing to change the results of the election to drop a USB key into the pile of USB keys used to set the system up. All of the security measures that the insider talks about are then bypassed with ease. The video shows it getting buy the procedural security measures, as well as the pre- and post-election audit features.
The video also shows why paper ballots are hardly a solution, as the second video shows how the malware included in the software can be set to void out legitimate votes and replace them with fake votes, in a variety of different scenarios, almost all of which are likely to go undetected. This is a hugely damning report -- and it comes against a company that has fought so hard against having its machines tested by independent security experts. While some may say that this shows why they didn't want it tested -- it should concern anyone who believes in free and fair democratic elections that we're using such insecure voting machines.
For all the trouble surrounding e-voting, some folks believe that optical scan technologies that simply count the paper ballot votes are a decent solution. Of course, those optical scan technologies are often made by the same companies that make the e-voting equipment, and have been shown to have numerous problems going back many years. And, as per usual with these e-voting companies, they've been highly resistant to independent inspection of the systems. Perhaps that's because the machines can't do the one thing they're supposed to do properly: count the votes.
Down in Palm Beach County, Florida (yes, the home of the infamous 2000 election year "butterfly ballot" with its hanging chads), officials are admitting that they've somehow lost about 3,400 ballots. But they don't seem to be saying they physically lost the ballots -- they're saying that the optical scan machines, provided by Sequoia Voting Systems (no stranger to e-voting counting problems) count the ballots differently when the same ballots are run through different machines. In trying to explain how come a "recount" showed 3,400 fewer ballots than the original count, a county official explained:
The seven high-speed tabulating machines used in the recount are much more "unforgiving" than those that process votes on election day
Does that not seem highly problematic to people? Isn't part of the point of these optical scan machines that they'll count the ballots consistently? If everyone seems to admit that there's an element of near total randomness (chalked up to how "unforgiving" the machines are) in these machines, isn't that reason enough to question their usage at all? As for the election in question, it appears that officials have decided to throw up their hands at the controversy and certify the election, despite the fact that this "unforgiving" recount changed the results of the election. Update: Well, now officials are claiming that it wasn't a technology problem but that they simply didn't feed ballots into the machine. That's not particularly comforting either -- and it's still troublesome that they would suggest that machines would count the votes differently in the first place.
Last month, e-voting firm Sequoia threatened both independent researchers and New Jersey election officials if those independent researcher were allowed to inspect Sequoia's e-voting machines. This seemed like a very odd threat for a variety of reasons. Why wouldn't Sequoia want its machines inspected? The very fact that it was threatening legal action seemed like grounds to simply never use Sequoia e-voting machines. Sequoia claimed that existing inspections were enough, despite a history of problems in those inspections. Furthermore, Sequoia's own explanations for the problems with its machines in the primary elections this year were wrong. Ed Felten found that Sequoia's explanations didn't actually explain many of the problems. Unfortunately, though, with the threat of legal action, New Jersey agreed not to have Felten test the machines.
However, a New Jersey state judge has now ruled that it's perfectly reasonable for independent inspectors to review the machines. Unfortunately, she pushed back the date for such inspections until September, meaning that it won't affect this year's presidential election -- which will still use machines that may have problems. So while Sequoia didn't succeed in stopping independent examination of its machines, it did stall the process long enough so that the existing machines will stay in use for this year's elections -- despite the long list of problems that have been discovered with them. Apparently, we're still in beta when it comes to democracy.
You may recall that last month, the state of New Jersey asked some top notch computer security researchers, including Ed Felten, to do an independent study of Sequoia's e-voting machines. That's because there were some worrisome discrepancies in the voting totals that the machines released. When Sequoia found out about this it threatened to sue, which seems fairly odd. If the company were confident in the quality of its e-voting machines, why wouldn't it want well-respected security researchers to take a look? However, Sequoia's legal threats worked, and the state of New Jersey nixed plans for that independent review. Sequoia also offered an explanation, claiming that it was all a minor bug, where the machine merely got mixed up about party affiliation -- but the vote totals would match up in the end. Guess what? That turns out to not be true.
Ed Felten has received a bunch of "summary tapes" from the last election in New Jersey, and while many of them do have the vote totals matching up correctly at the end at least two of the summary tapes simply don't add up, meaning that Sequoia's explanation of what went wrong is incorrect. Given how often the company has denied or hidden errors in its machines, despite a ton of evidence, we shouldn't be surprised that it was inaccurate in explaining away this latest problem as well. However, we should be outraged that the company refuses to allow third party researchers to investigate these machines. It's a travesty that any government would use them when they've been shown to have so many problems and the company is unwilling to allow an independent investigation.
Just as e-voting firm Sequoia is resisting having its machines reviewed independently, the Brookings Institute has put a bunch of e-voting machines to the test, and found error rates around 3% on some of the machines. These weren't errors due to software problems, but usability problems, where the design of the system resulted in people voting for a candidate they did not want. 3% is a huge number, and could easily change the results of an election. While the study found that people generally like e-voting technology, that still doesn't mean it's particularly effective. One other interesting part of the finding: when there was a voter-verified paper trail, it didn't cut down on errors. This suggests that many voters were either confused or didn't even bother to verify their vote. This should all be very worrisome. Even ignoring the technology problems that these machines have been shown to have, the fact that the design tends to create so many mistake votes should lead people to seriously question the use of e-voting machines.
Yesterday we covered the threats that e-voting firm Sequoia had sent to Ed Felten and to various officials in New Jersey. Unfortunately, it appears those threats worked: the election officials have backed down and agreed not to send Felten the machine to test. News.com has more details on both the reason for the test and Sequoia's response to the whole mess. The reason? Shockingly enough, Sequoia's e-voting machines malfunctioned during the primary in a way that should scare you: it gave two different vote counts. You would think that's a pretty good reason for allowing a qualified, well-respected researcher like Felten to check out the machines. No such luck. Sequoia has tried to explain it away as a bug, but that doesn't explain why the machines shouldn't be tested by a third party.
Sequoia's response to that question is disingenuous, claiming that the company "supports third party reviews and testing of its election equipment." If that's so, then why not Ed Felten? Well, because Sequoia says that the machines have already been through a "rigorous" independent review from an accredited Voting System Test Labs. Ah? Would that be one of the accredited Voting System Test Labs that was barred from further testing for not having proper controls in place and having no evidence that tests were actually conducted? Most of those tests have very limited real-world applicability -- which is what Felten is good at testing. Sequoia also lists out some independent tests in other states that the company was forced into accepting, as if it willingly took part in them. Yet, what the company doesn't explain is what it's so scared of in having Felten test its machine. If the company is confident in the machines, then where's the problem? As a last resort, Sequoia appeals to the fact that such a test would break a licensing agreement, noting that "Licensing agreements are standard practice in the technology industry." That's clearly a cop out. While it may be legally correct, it's no reason not to let a researcher try to figure out if there are any problems with its machines. This isn't some random technology here. This is the technology we're trusting with providing a free and fair election. Sequoia should be ashamed of pulling out legal threats and weak excuses.
Many of the folks around here are surely aware of the name Ed Felten, the Princeton professor who runs the fantastic blog Freedom To Tinker, and who has been involved in a number of important technology news stories over the years. One of the first that brought him to much wider attention in the tech community happened back in 2001. The recording industry had set up a contest, asking anyone to try to hack its SDMI DRM offering. The idea was to prove that SDMI was a perfectly good DRM. But, of course, like every other DRM, it had its faults, and Felten and some of his researchers figured them out. That's where things got ridiculous. Despite the fact that the recording industry had told people to try to hack SDMI, when Felten went to present the paper, he was threatened with a lawsuit for breaking the anti-circumvention clause of the DMCA. Eventually, after a ton of public pressure, the recording industry backed down, but Felten's name was cemented in the minds of many in the tech industry as a fighter for freedom of speech and, more importantly, the freedom to tinker.
It would appear that the folks at Sequoia, one of the big three e-voting firms out there, is somewhat unaware of this aspect of Felten's past. In the past few years, Felten has been one of a few top computer science experts who have been picking apart the problems with e-voting machines. His freedom to tinker with such machines has broken numerous stories revealing serious problems with the machines that many suspected, but were unable to confirm, since the e-voting firms kept the machines so under wraps. In publicizing these flaws, Felten has become one of the go-to guys when various governments are reviewing e-voting machines, so it should come as no surprise that election officials in New Jersey (where Felten lives and works) would be interested in having him run some tests on a Sequoia e-voting machine that they're looking at using in future elections.
This seems perfectly reasonable -- and if you're an e-voting company like Sequoia, it should also be a perfect way to build more trust in your machines, telling people that they've been reviewed by some of the top experts in the field who found nothing wrong with them. Except... that's not how execs at e-voting companies seem to think. Sequoia has, instead, sent a threatening email to Felten, saying that election officials who sent a machine to Felten would be breaking the state's terms of service with Sequoia, and that the company has:
"retained counsel to stop any infringement of our intellectual properties, including any non-compliant analysis. We will also take appropriate steps to protect against any publication of Sequoia software, its behavior, reports regarding same or any other infringement of our intellectual property."
Yes, this is quite reminiscent of the recording industry's threats to Felten in 2001. Hopefully this situation ends similarly -- with Sequoia backing down quite publicly and apologizing. It's disgusting that such a firm would threaten a well-respected researcher with lawsuits just for checking on the security of an e-voting machine. This is worse than the recording industry situation. This is about the sanctity of our democratic elections. For Sequoia, a firm entrusted with our elections, to threaten someone for merely testing its product to make sure it lives up to necessary standards is terribly worrisome. It should call into question any locality that chooses to make use of Sequoia e-voting machines.
Just days after Ohio announced problems with all of the e-voting machines used in that state, Colorado has decertified e-voting machines from all four major vendors in the space, noting serious problems with them all, including a 1% error rate in counting ballots (1%!). So at what point do the e-voting companies stop stonewalling and finally just admit that they need to start again from scratch? At this point, it's beyond clear that none of these firms is even the least bit trustworthy -- and yet, they continue to protest these decertifications, despite piles upon piles of evidence that these machines have serious problems.
Another day, another security problem with e-voting machines. Obviously, one of the biggest requests from people who were nervous about the security of e-voting machines was that all e-voting machines have a verifiable paper trail. Then, at least, there's a way to recount the votes if there are any questions. Unfortunately, even when the e-voting companies finally do add a paper trail, it seems that they muck up the process. As was noted in the recent security analysis of these machines, many of the problems are because they weren't designed from the ground up with security in mind, but rather have security procedures slapped on as extras.
In this case, some Ohio activists discovered that the paper trail coming from e-voting firm Election Systems and Software (ES&S) happen to have time and date stamps on them. Those ballots are available for anyone to look at, based on election law in Ohio. Also available for anyone to peruse are the voter sign-in logs. With both of those in hand, it's not hard to put together a pretty decent list of who voted for what. You just match up the names in the order they signed in with the timestamp on the ballots.
Of course, rather than responding to this as they should, by admitting it was a bad idea, ES&S sends out their PR people to say it's no big deal. While ES&S is right that it might not always be possible to do an exact match person to person, you can come pretty close -- and that should be seen as a huge concern. Furthermore, as Ed Felten points out, the other e-voting firms aren't much better, and Diebold (or Premiere, or whatever its new name is) appears to be outright lying skirting the truth when it claims that its paper trail doesn't include timestamps (update:: Ed Felten points out that the Diebold ballots don't have a time stamp, but the electronic records do). It's not hard to see how this happened, but the continued denial and stonewalling from the e-voting companies, rather than admitting a mistake was made and explaining how they're going to fix things, really is troubling.
Following the release of the various independent security reports last week on the e-voting machines used in Californa, the Secretary of State needed to decide by Friday night whether to keep the machines in use in the state. At 11:45pm, she decided to decertify the machines only to immediately recertify them if they made some security changes. Of course, it seems like the changes are simply patches, and as the original report noted, many of the security problems the machines have is because all of the security they've implemented was patched on as an afterthought. Until the machines are designed from the ground up with security in mind, it's not likely to really fix many of the vulnerabilities. But, in the meantime, there's an election coming up, and apparently a bunch of major security problems are no reason to get rid of the expensive e-voting machines the state has already purchased.
Back in March, California decided that after years of negative publicity about the security of e-voting machines (and certainly enough evidence to suggest they weren't very secure) that it would allow independent security experts to try to hack into any machine before it got approval to be used in California elections. Those researchers have gone ahead and found that every machine they tested was hackable -- often very easily. The researchers were able to hack into Diebold, Sequoia and Hart InterCivic machines. They didn't get a chance to test ES&S machines because, as you may recall, ES&S stalled before handing over their source code (and included a nasty threatening letter with it). To be fair, these machines were tested in non-normal conditions, where the researchers had access to all sorts of documentation, the full source code and no election going on where people might spot them tampering with a machine. That is, this doesn't mean that it's necessarily easy to hack an election. It just means that all of the machines have some insecurities -- most of which we didn't know about before. The key here is that we can now understand these insecurities and whether or not they're adequately protected by other measures. What still doesn't make sense is why the e-voting firms are so against this process. All it's really doing is helping those companies improve their products to make them more secure. Of course, one key reason is that the researchers found that many of the security problems are because the machines weren't built with security in mind -- but only had it added as an afterthought. In other words, these companies probably should be redesigning their machines from scratch, which they don't want to do. Of course, does it worry anyone else that the machines weren't designed with security in mind in the first place?
RSS
Twitter
