Making sure the votes count

Arapahoe County is pilot site
The Colorado Statesman

Arapahoe County elections officials worked out the kinks this week on a new way to verify the vote count, one that should increase confidence in election results and will be adopted by most counties in Colorado within three years.

The so-called risk-limiting audit uses statistical methods to examine ballots until a pre-specified level of confidence is reached that the election turned out the way officials declared after election night. A variant also piloted by Arapahoe County brings the same confidence to the performance of ballot-counting machines. In both cases, auditors examine ballots until they’re confident that the reported results are right, pulling far fewer ballots for contests decided by wide margins and potentially plenty of ballots for close races.

Arapahoe County Clerk and Recorder Matt Crane pulls a roll of vote tabulations from the county’s electronic voting equipment cast during the June primary election at a risk-limiting audit performed this week in Littleton.

“Our voters deserve to know the process is fair, and accurate and transparent,” said Arapahoe County Clerk and Recorder Matt Crane during a demonstration of the risk-limiting audit at the county’s election headquarters in Littleton on Wednesday.

Dr. Philip Stark, the statistician who designed the risk-limiting audit system tested by Arapahoe County this week, inputs variables into a simple, web-based calculator to determine how many ballots to pull for a hand count on Aug. 13 at the county’s election facilities.

Arapahoe County tried the method after last November’s combined election and ran the pilot audit again this week on results from the two contested races in June’s primary — the Republican contests for governor and the House District 37 seat.

Arapahoe County Clerk and Recorder Matt Crane and Dr. Philip Stark, chair of the UC-Berkeley statistics department, select a ballot chosen at random from among nearly 40,000 cast in the June Republican primary on Aug. 13 at the county election facilities in Littleton.

“That’s why we volunteered again, to try to get us closer to get us ready to roll in 2017,” Crane said.

Former Secretary of State Donetta Davidson and Arapahoe County Clerk and Recorder Matt Crane pause for a moment during a pilot risk-limiting audit of the county’s ballots.

Bipartisan legislation passed in 2009 mandates that every county has to adopt risk-limiting audits by 2017, though the deadline has already been extended and waivers could be granted by the Secretary of State’s office if the necessary equipment isn’t on-line by then.

Dr. Philip Stark brought with him a set of colorful, 10-sided dice used to generate random numbers.
Photos by Ernest Luning/The Colorado Statesman

Philip Stark, who chairs the Department of Statistics at the University of California at Berkeley, designed the audit system and was on hand in Littleton this week to guide the demonstration audit. He brought with him a computer security virtuoso and a ballot-counting expert — “all on their own dime,” Crane pointed out — along with a set of colorful, 10-sided dice used to generate random numbers.

“What distinguishes a risk-limiting audit is it has a large, pre-specified chance of correcting the outcome if the outcome is wrong,” Stark said. “These don’t just do a quick spot-check of the function of machines and then stop; instead, they keep looking at more and more evidence until you have convincing statistical evidence that if you keep looking at more, you’re just going to confirm the answer that you’ve already got. Otherwise, if you never get evidence like that, you keep counting until you’ve counted all the ballots, and then you know who won because you have a full manual tally.”

Risk-limiting audits have been piloted in roughly a dozen counties in California and demonstrated in Ohio, officials said. Colorado is the first state to mandate their use in statute.

Currently, clerks audit election results by pulling at random 500 ballots or 5 percent of the vote, whichever is fewer, running them through the ballot counters again and then comparing those results to a hand count. While that kind of audit verifies something — that the machine count conducted for the audit aligns with a hand count — it doesn’t reveal whether the machines read the ballots the same way when they were counted the first time, for the official election results, Stark noted.

Election-integrity advocates praised the pilot audit and said they were looking forward to Colorado’s statewide adoption of the method.

“We do so much to help voters cast their ballots, it’s heartbreak if, at the end of the day, something happens with the machines or the outcome isn’t what voters intended,” said Susannah Goodman, director of the Voting Integrity Program at Common Cause, who attended the pilot audit. “That’s why this is so exciting to us is because it reinforces all the work we’ve done at the front end to get voters to the poll, to get voters registered, to make sure they’ve cast their votes. This is a way of saying, we’re going to check the outcome again with an audit.”

“Risk-limiting audits are extremely efficient but must be performed with exacting precision and only when a measurable high standard of quality characterizes the collection of evidence (ballots) brought to the audit,” vote watchdog Harvie Branscomb told The Colorado Statesman. “Officials will be able to defend their elections as ‘evidence-based’ once an excellent risk limiting audit is in place across Colorado.”

In practice, auditors start with a statistically determined number of ballots to pull and then use a random-number generator (the dice help with this) to decide which ones to examine. Then they tally those results until the official winner is ahead by a wide enough margin in order to conclude that he or she won the election. (In races with razor-thin margins, auditors might have to keep pulling ballots at random until they’ve hand-counted all the ballots cast, which would effectively be a full recount by hand, officials noted.)

In the House District 37 Republican primary, for instance, Stark’s formula determined that auditors would have to pull 58 ballots in order to verify that Jack Tate was the winner — he defeated Michael Fields roughly 2-to-1, with 66 percent of the vote.

Using Stark’s formula, auditors determined they would have to pull just 15 ballots to perform the second kind of risk-limiting audit that Arapahoe County was demonstrating this week, the comparison audit, which tests not whether Tate won the race but whether the ballot-counting equipment correctly recorded the votes.

Because the governor’s primary was both close and statewide — former U.S. Rep. Tom Tancredo won in Arapahoe County by a couple percentage points, about the same margin former U.S. Rep. Bob Beauprez won statewide — the formula determined that auditors would have to hand-count some 7,000 ballots out of the roughly 40,000 cast in the county, too many for the staff on hand. In an actual statewide audit, that number would be spread among all 64 counties, officials noted, spreading the burden thinly.)

To accomplish the other type of audit Arapahoe officials piloted, a risk-limiting comparison audit, clerks must have a complete record showing how the tabulation system — optical scanners, for instance, to count mail ballots — interpreted each vote on each ballot, and then auditors pull a number of actual ballots at random and determine whether their interpretation of the result for individual ballots is the same as what the voting system decided. After reaching a sufficient number of matches — the auditors decide this ballot was a vote cast for Tancredo, for example, and that’s what the system recorded — then there can be sufficient confidence that the ballots were counted correctly.

This type of audit is helpful discovering problems with tabulation systems — either software errors, as happened in a recent Florida municipal count when a glitch led to votes allotted to the wrong candidate and the wrong race, which was discovered in a post-election audit, or physical errors, such as dust specks on the scanner lens or inadvertent markings on ballots that lead to incorrect tallies. This audit can also determine whether the machines are identifying over-votes and under-votes correctly.

For this week’s Arapahoe County demo, the clerk’s office purchased scanning equipment and acquired open-source ballot-scanning software to produce the initial records showing how the system recorded each ballot’s vote. (Currently, once a ballot has been counted, there is no record tying that individual ballot to the results read by the counting equipment, although the scanners and software used in the pilot audit was able to do this.) Counties will have to have the right kind of ballot-readers and software in place — they exist now but aren’t certified for actual vote counts, though work fine for demonstrations — before the 2017 rollout in Colorado, current and former election officials told The Statesman.

“The equipment’s just not ready,” said former Secretary of State Donetta Davidson, who served as a commissioner on the U.S. Election Assistance Commission and has been monitoring the development of Stark’s audit method. A past Arapahoe County clerk and recorder, she has also been executive director of the Colorado County Clerks Association and observed the pilot audit in Littleton this week. “The venders need to step up and realize this is something the state needs and start moving in that direction so it can be certified by the federal (authorities) and then certified by the state,” she added.

Ernest@coloradostatesman.com

See the August 15 print edition for full photo coverage.