Dartmouth says 90% chance Dem won in Sarasota, Fl, but innocent ballot design error is the

[papau]

mostly likely problem (shades of the butterfly ballot) - although they do not rule out programs being messed with (“machine malfunction”) .

=========================================================

Ballot Formats, Touchscreens, and Undervotes

http://www.dartmouth.edu/~herron/cd13.pdf

The 2006 midterm elections in Florida have focused attention on undervotes, ballots on which no vote is recorded on a particular contest. This interest was sparked by the high undervote rate— more than 18,000 total undervotes out of 240,000 ballots cast—in Florida’s 13th Congressional District race, a race that, as of this paper’s writing, was decided by 369 votes. Using precinct-level voting returns, we show that the high undervote rate in the 13th Congressional District race was almost certainly caused by the way that one county’s (Sarasota’s) electronic touchscreen voting machines placed the 13th Congressional District race above the Florida Governor election on a single screen. We buttress this claim by showing that extraordinarily high undervote rates were also observed in the Florida Attorney General race in Charlotte and Lee Counties, places where that race appeared below the Governor race on the same screen. Using a statistical imputation model to identify and allocate excess undervotes, we find that there is a roughly 90 percent chance that the much-discussed Sarasota undervotes were pivotal in the very close 13th Congressional District race.

The same count revealed that 21,303 ballots, approximately 8.2 percent of those cast, in CD 13 recorded no candidate choice in the House race (This count includes a handful of so-called overvotes in which a voter voted for more than one CD 13 candidate. Overvotes, which are invalid of course. can be cast by voters who use optical scan ballots but not by those using elec-tronic touchscreen machines. The latter are often called DRE (for Direct Recording Electronic) machines. Initial CD 13 recount results were downloaded from http://election.dos.state.fl.us/pdf/certCanvasCom.pdf).

The pre-recount canvass from DeSoto County was faxed to the authors and is available from them.
they do not include thousands of legal votes that were cast in Sarasota County but not counted due to the pervasive malfunctioning of electronic voting machines( http://www.heraldtribune.com/assets/pdf/SH81371120.PDF).

....It remains possible, of course, that a programming or design flaw in Charlotte County’s, Lee County’s, and Sarasota County’s touchscreen machines caused low vote counts when a race (either the CD 13 rate or the race for Florida Attorney General) was placed on the same page as the Florida Governor’s race. However, it is our belief that the CD 13 undervote problem is most likely is related to the combination of a high-profile contest with a large number of alternatives being placed on same screen as a race with only two alternatives.


....The problematic grouping method that Sarasota employed—having a two-candidate United States House race on the same touchscreen page as the Florida gubernatorial race—was employed by other counties, albeit with different races, and in these other counties and associated races we generally find high undervote rates.

...We estimate that, had Sarasota used a ballot format akin to those in neighboring counties, with probability 0.9 Jennings would have beaten Buchanan.


...There remain two key issues over which we do not have leverage. The first is the precise reason as to why grouping races on touchscreens is a problem. Is the issue asymmetry between races, i.e., a race with two candidate grouped with a race with multiple candidates? Or, is it total number of candidates on a page? Or, is the issue vertical stacking of races versus horizontal grouping? Or, is there a tradeoff between grouping races and the number of pages in a ballot wherein greater number of pages itself leads to undervotes? We cannot address these questions at this time, but work in the vein of Herrnson, Abbe, Francia, Bederson, Lee, Sherman, Conrad, Niemi & Traugott (2005), Herron & Lewis (2006), and Herrnson, Bederson, Niemi, Conrad, Hanmer & Traugott (2006) may be applicable here.

Second, and as mentioned previously on several occasions, because this paper presents a sta-tistical analysis of vote patterns and not a physical examination of voting machines, we cannot completely rule out voting machine malfunction as a source of the Sarasota undervote. Is it techni-cally possible that software or hardware glitches are responsible for the Sarasota undervote and the Charlotte and Lee undervotes? Yes, this is possible. However, it seems to be a significant stretch of the imagination: if the issue is a software or hardware glitch, the glitch would have had to have manifested itself among almost all voting machines in Sarasota County but affected only one race and then have manifested itself in almost all machines in Charlotte and Lee Counties yet affected a different race.

We conclude with what we believe is a simple and conservative implication of our main finding: : iVotronic touchscreen voting systems should not combine important races on the same voting page. Regardless of why exactly combining races is a problem, this proposal seems likely to avoid it.

<snip>

[aquart]

1. "Innocent"? In Sarasota? Suuurrrrre.

I think the word they're looking for is "traditional."

[jumpoffdaplanet]

2. They are really pushing this lie

Sorry, but I don't buy it.

Too many people complained that day they weren't able to get the machine to accept their vote for the Democratic in that race.

It's a lie designed to stop the investigation into the rigged machines.

And only people who want rigged election are trying to sell this lie.

[bullimiami]

3. heres my take. the house and the courts, just this last year, affirmed that

it is the house itself who has the final say who is sworn in and who sits. (busby/bilbray)

the new democratic house will have the say on this race. they should tell FL CD-13 that the seat will remain empty until they have a fair election for the seat. Either by an acceptable recount which takes care of the undervotes (which I dont think is possible) or special election.

what the republicans have done in the past is just seat their member regardless of the problems with the voting. ie busby/bilbray again. but i reject this option and prefer the race actually go to who the people vote for.
Actual democracy.

[papau]

4. The Dartmouth folks agree the Dem won - so we should give her the seat. n/t

n/t

[bullimiami]

5. but the official count says otherwise. thats why, instead of forcing a bitter pill

down the throats of the republicans,
i would take a stand that only full and fair elections will be accepted by the democratic house.

[papau]

11. They did it to the Dems in the last Congress - so why do we not do the same :-) I agree n/t

n/t

[philb]

8. High undervotes on ES&S touch screens with only one issue on the ballot, & etc.

Florida: Voting Glitches Found in 6 Recent Elections

A total of 1,246 undervotes were recorded on the iVotronics for the March 8 slots referendum, versus 61 on the absentee ballot.
''Undervoting'' means a ballot was cast but no choice was made.
An alarmingly high number of so-called ''undervotes'' in the March 8 election using ES&S touch screens -- which only had one item on the ballot.
http://www.verifiedvotingfoundation.org/article.php?id=5687

[philb]

6. This may be mostly spin, he doesn't deal with all of the issues reported by the voters & poll worker

even though he concludes that Jennings almost surely would have won in a fair count.

See the statements of voters, poll workers, etc.
http://www.democraticunderground.com/discuss/duboard.php?az=show_mesg&forum=203&topic_id=457470&mesg_id=457470


There was more than one ballot type used in Sarasota, and some were more problematic than is described in this analysis.
Also, there were disappearing votes-
done most likely by default to blank programming as has been commonly seen in other races in 2004 and 2006,
or through programming or glitch in ballot design programming

It is important that the machines be isolated and all looked at to determine the various type of ballot designs used and the programming of the program compiling the votes from each.
It appears a major effort to prevent looking at the machines and programs is in process.
If the machines have remained isolated and a thorough audit done, the answere will be clear.

[OnTheOtherHand]

10. no! yes! hang on!

First, a point of protocol: the paper is Frisina et al. 2006, and Frisina's first name is Laurin, so I doubt the "he" is appropriate.

I'm not sure what "this" you were referring to as maybe "mostly spin," but it kind of stings to have a serious paper described in those terms. I do agree that the paper doesn't account for all the issues reported by voters and poll workers -- and I, too, want to see a close and unbiased audit of the machines.

[philb]

13. The study seems shallow, not consistent with what was described

There were 2 common types of problems described by voters and poll watchers.

1. disappearing votes

2. apparent biased ballot design on some machines (not all)

1. To deal with the first problem, enough machines need to be looked at regarding setup and programming to determine why the votes were disappearing. Likely causes based on other races that have had similar problem described by voters include:
programmed with "default to blank" that can be hard to override
programmed to switch to blank for specific candidate on time delay, while voter is going on to another race
deliberately miscalibrated for one candidate, so hard for voters to correctly choose that candidate
(since the votes were only disappearing for Jennings in this race and for certain specific candidates in other races in 2004 and 2006, its clear that the problem was not caused by random miscalibration)

Experts should be able to determine which of these types of problems caused the problems observed, if the machines have been maintained in the same state as when the election ended; but based on the vote history, some may have been "repaired or "recalibrated" during the election. (was there a record maintained of which machines were worked on during election process?)

2. Voters described at least 3 clearly biased ballot designs(the Dartmouth study did not consider them) :
(a) "Butterfly type design" with the Congress race header and Buchanan on page 1 and Jennings alone at top of page 2 with no race context.
(b) 2 races on one page with the top race (Governor or U.S. Senate) highlighted but Jennings race below not highlighted so the Cong. candidates look like candidates at the bottom of the other race, with a well known candidate that would be chosen at the top
(c) Jennings/Buchanan race not on any page, but shows up on review screen as blank votes
(d) design with 2 races on one page, Jennings race below another, but both races highlighted.
(e) someone has suggested there were 9 separate ballot designs used in Sarasota

To determine the extent to which any of the biased designs described by voters(a-c) (or (d) as per Dartmouth study author)
might have caused the undervotes, all of the TS machines would need to be looked at to determine and compile
how many of the screens had ballot design (a,b,c,d,e,f,...) Only by looking at all machines layout could this be determined, and it would be a simple and relatively quick process, nothing complicated).
Looking at a few machines as per the planned "audit" does nothing to address the ballot design bias issue, only by looking at all can it be resolved.
To do anything is leaves the issue unresolved, perhaps intentionally.

[OnTheOtherHand]

15. I think "narrow" is fairer than "shallow"

Writing such papers is not as easy as it looks on television. The authors are mostly appropriately careful in delineating the limits of their analysis.

It's very hard to assess the voter reports. I believe the Herald Tribune has reported that both Jennings and Buchanan voters (but more Jennings voters) have asserted both (1) that they cast votes that disappeared, and (2) that the race didn't appear at all. (There are other problem reports as well, but those seem to be most common.) I think at least some of those accounts are probably inaccurate, and I am very skeptical that there were actually 9 separate ballot designs used in Sarasota. That seems to assume that eyewitness accounts are infallible, not a reasonable assumption. (If someone was using inside access to steal votes in Sarasota, would s/he really do it by deploying 9 separate ballot designs? It doesn't seem parsimonious to me.) But if we can find out, we should.

[Sancho]

16. Yes, there likely were a variety of ballots...

We see this in Pinellas, and it shows in the precinct level data. Some precincts voted for a "hospital board" or "charter board" that others did not depending on their location. You would think that these extra votes would have a common page or be tacked on at the end, but if they were inserted between other general races, it would create different paging. There is sometimes state statute that dictates the order of presentation of candidates and races, but I don't know the details.

In Pinellas, we get different versions of the on-screen ballots depending on the eligible races or amendments.

[OnTheOtherHand]

18. Venice and North Port had their own races

but since there are high undervotes across the board, that isn't crucial in itself. (Of course there's no reason why the North Port ballot style couldn't have been messed up even worse.) I'm pretty sure that the federal races appeared on pages 1 and 2 countywide.

[Sancho]

32. The ballot screenshots have the major races on pages 1 and 2...

but I wonder if some techie inserted a local amendment or something, and altered the touch points or split the screen. Some local papers have published exact replications of the screens to practice with...and practice machines have been set up for the last couple of elections. I'm not really convinced that the Dartmouth hypothesis of ballot design would be the cause of this much undervote. The test run hasn't confused the volunteers yet.

The ballot theory may be possible, but it is more of a stretch than machine error, no matter if it was bad programming or purposeful hacking.

[OnTheOtherHand]

33. I'm just reserving judgment on that

There are actually people who study DRE usability, and I am not among them. You say, "I'm not really convinced that the Dartmouth hypothesis of ballot design would be the cause of this much undervote," and I'm not either. As to what is more of a stretch, I really can't tell yet.

I don't think the test run sheds much light on how confusing the ballot may be, because it seems inconceivable that the participants don't know where to look for the race. (And I think they have voting scripts, so in fact they are directly cued that Jennings/Buchanan is the second race, is that right?) To get a meaningful usability result, one might test a similar ballot format elsewhere in the country, inviting people to "vote for all the Democrats" or "vote for all the Republicans," and see what happens. Not perfect, but it might be revealing.

And of course the machines ought to be tested -- not necessarily every damn one (since the undervotes were so widespread), but more than a handful.

If this problem can't be replicated in testing, the question arises: is it a disappearing hack? and if so, why? If someone had the requisite access to implement such a hack, would s/he also have the requisite access to implement a hack with a less detectable result? So far I haven't seen a compelling explanation of why this would be a good way to hack an iVotronic, but I know little about iVotronics myself.

[Pokey Anderson]

7. Nope. Doesn't work with the facts.

The Orlando Sentinel studied the undervote ballots in Sarasota. Found that they skewed Democratic -- the ballots with the undervotes had voted substantially Democratic on other races, more than one would expect by chance.

http://www.truthout.org/docs_2006/112206S.shtml

[OnTheOtherHand]

9. I think they'll be updating their estimates

The article concludes by saying that they are moving to the individual level. Their model estimates are very conservative even for the precinct level.

[read the law first]

12. How would they know?

How were they able to separate out the 18,000 undervotes to look at how the undervoters voted? Isn't the problem that you CAN'T tell how they voted? How did they manage to separate out only the undervoters and examine only their ballots?

[philb]

14. the touch screens have a memory. You can determine how each voter voted if you can believe the memor

y (you just don't know the individuals that the votes go with, so don't know if thats the way they really voted or whether its a manipulated record) (in this case they know for the records with a missing Jennings race vote, the other race results)
(known to be mostly Dems per audit)

of course its easy to have programmed the ballot design file or the compilation to give a certain machine whatever records that was desired.

(and the ES&S programmers who program the machines apparently have unlimited access to program the machines however they like- with no checks or balances)

[eomer]

21. I was surprised that the Orlando Sentinel got access to that data so quickly.

Sure would be nice if they would publish the data. Apparently it is a public record, or else how do they have it?

[read the law first]

24. touch screens don't have a memory. it's the pcmcia card

the touch screen machines, per se, don't have a memory. it's the pcmcia card that stores the memory. presumably, the pcmcia card can recreate each screen that was displayed on the dre. how did the paper isolate the ballots? I would assume that the 18,000 came from different dre's scattered all throughout the county. how did the paper isolate the particular ballots on particular machines only locating the 18,000 undervotes? especially so fast. that's the question.

[eomer]

26. The data must have already been uploaded from the cards into some database.

The candidates' experts were allowed to review the undervote numbers per machine and then make recommendations of which machines to test based on that info. So apparently the County elections office has the ballot data from the entire county assembled in some centalized repository and can run queries against it.

Probably the Sentinel just made a public records request and then the County delivered the requested data to them. In my county (Miami-Dade) you can customize a records request basically down to whatever SQL query (or more complex report layout) you want run but you do have to pay labor plus 15 cents per page for the report.

I was surprised that this level of detail was available but maybe that would be the case in any county as long as you are willing to pay whatever the cost comes to.

_{Edit: typos}

[read the law first]

27. a question

what's an sql query?
so....you can download the entire individual ballots to a database?

[eomer]

28. In my county (which also uses ES&S iVotronics DREs)

they take the memory cards out of the DREs and transport them to tabulation centers where they read them and tabulate the results. After this step they have all the data from the cards assembled somewhere centrally -- most likely in a database since it seems to be easy for them to produce reports from it.

SQL = Structured Query Language.
SQL Query = a statement in SQL that will retrieve records from the database using whatever criteria you specify.

[read the law first]

29. right.

I'm familiar with the accumulators and the reports they generate and I knew that they could recreate screens. I just didn't know that you could recreate an entire ballot but I guess that would be the case in that they just recreate the summary screen for each voter. I guess that's what they do. I still think that the paper is looking at raw numbers and extrapolating rather than looking at recreated ballots.

[eomer]

31. What do you mean by raw numbers?

I think the data on the memory cards is just the ballot data. IOW, all the votes of one voter on one ballot, repeat for all the voters. I don't think it stores, over and over for each voter, the screens. The screen definitions would be separate, would not be stored repeatedly for each voter and would, I think, be stored somewhere other than on the memory cards that collect the votes.

The vote data on the memory card would be exactly what the Sentinel needed if the design of the system is at all reasonable. Of course, maybe that's not a safe assumption.

It sounds like you have seen a system recreate screens but I imagine that would be done by taking the ballot data that is stored in the memory cards and then merging it with system-level (as opposed to voter-level) screen definitions in order to recreate what the screens should have looked like but not necessarily what they really did look like.

And, to be clear, I'm just making educated guesses. I've heard a couple of bits and pieces but I don't know all the actual details.

[eomer]

17. Cherry picking, unsafe assumptions, rationalization.

The Attorney General, CFO and Ag Commissioner races in Charlotte and Lee Counties buttress the case for ballot design as the cause of the undervotes (as opposed to machine malfunction as a cause). The House races in Miami-Dade County (all of them except CD 17, which was unopposed) infirm their case because the undervote rates for the Miami-Dade House races are in the 5% to 8% range and are not as drastically out of line with absentee undervote rates. The Miami-Dade House races had a ballot design problem that is very similar to those in Charlotte and Lee. Did they pick the data that buttressed their case and exclude the data that infirmed it?

I'm otherwise unconvinced by the logic in this paper. There are just too many unknown forces that may or may not be present for the assumed relationships on which the case is based to be safe assumptions. For example, the logic uses an assumption that undervote proclivity will be somewhat consistent across the various races within a geographic area. There may in fact be no such tendency. The proclivity to undervote for Governor can possibly be totally foreign and unrelated to the proclivity to undervote for Agriculture Commisioner. The paper also tends to think of each county as a homogenous entity and then treats the collection of counties as heterogenous. In other words, crossing a county line magically makes voters different on one side of the line from those on the other. This assumption is clearly not true in many cases. You may have cases of smaller communities that span across county lines and whatever proclivities there are may easily be related to those communities and therefore be orthogonal to the county of residence characteristic of a voter. Similarly, partisanship tendencies that are assumed to span across races may be a total fallacy. There may be issues of a particular race that make it totally unrelated to some other race in terms of how partisanship affects it.

All in all this paper is loaded with unsafe assumptions and is quite a stretch in its conclusion that the undervotes were most likely caused by ballot design and not by machine malfunction. And, what's more, the facts basically prove that the conclusion is false. There are just too many eye witness reports of machine malfunction that we would have to ignore in order to believe the far-fetched chain of logic presented in this paper. So the interesting exercise is to figure out where this paper went wrong. It cannot possibly be right.

[OnTheOtherHand]

19. two issues

(1) I'm still not sure the paper is fundamentally wrong about ballot design -- keeping my powder dry on that one. Do you have a picture of ballot layout in Miami-Dade? Of course I agree that ballot design can't explain all the reports.

(2) I'm sort of curious myself about how they ended up with such a conservative allocation of undervotes; I think I know, but haven't looked closely. It really is an intellectual exercise at this point.

[eomer]

20. Re: two issues

(1)(a) There was a sample ballot on the Elections Dept website but they've removed it now. I paid attention when I voted (because someone I know had already told me that the House race was missing for her) and my screens had the Senate race, with 6 candidates, at the top of the first screen and the House race, with 2 candidates, at the bottom. The sample ballot had the same configuration before it was taken off. I believe the ballot design would have been like this everywhere since the top races were at the top of the screens and wouldn't have been affected by variations in down-ticket races. But that is admittedly somewhat an assumption.

(1)(b) I'm not sure either. But my position is: neither are they.

(2) Hopefully it is an intellectual exercise but maybe not. If the parallel testing finds machine malfunction either in the first batch planned for today and Friday or in some subsequent testing that they do because they are determined to find the explanation for the voter reports then it is intellectual. If, OTOH, they don't find any malfunctions in these first tests of just a few machines and decide to stop looking then we will be back to reading tea leaves to figure out who won the frigging election.

[OnTheOtherHand]

23. one more quickly

(1)(a) My gut tells me that's a less confusing design than the one in Sarasota -- what struck me (and some others) about Sarasota is how our eyes were drawn to the header below the House race. But that's just a speculation FWIW.

(1)(b) We agree.

(2) Well, I think that this particular article's estimates are superseded by other data (I think they were awfully conservative even at the precinct level, although that's sort of refreshing after a few weeks of TIA). I agree that we may nonetheless go back to "reading tea leaves." Sigh.

[eomer]

22. Just noticed that the Lee County CD 14 results also infirm their case.

They explain it away: "However, the CD 14 effect, we suspect, reflects partisanship
undervoting in a rather lopsided race." (top of page 22).

That makes four House races (three in Miami-Dade and one in Lee) that go against their case. They ignore those and instead go with the down-ticket races that support their case. But maybe down-ticket races will behave differently than House races. Maybe the four House races they ignore are really the ones they should have paid attention to and the down ticket races are the ones they should have been explained away.

[read the law first]

25. looks like this is one Associate Professor from Dartmouth and not Darmouth, right?

This looks like it's just one Associate Professor at Dartmouth and not an official publication of Dartmouth, right?

[OnTheOtherHand]

30. yes, the OP title is a bit misleading

No big deal as long as people don't get confused and start saying, "Well, Dartmouth says...." It doesn't work that way.