Towards Work-Practice Modeling of Elections and theElection Incident Reporting System (EIRS)

Joseph Lorenzo HallSchool of Information Management and Systems

University of California at Berkeley

joehall@berkeley.edu

ABSTRACTElection incidents – where the elections system fails voters– are of increasing concern with the dramatic rise in theuse of computerized electronic voting machines that storevote records in computer memory. The Election ProtectionCoalition (EPC) used a web-based election incident record-ing and reporting tool called the Election Incident ReportingSystem (EIRS) developed by the Verified Voting Foundation(VVF) and Computer Professionals for Social Responsibil-ity (CPSR) for the presidential election in 2004. In decadesof election protection, this was the first time EPC had usednetworked information technology to support their effortsand provide for a publicly-accessible database of incident re-ports. In this paper, I describe the first phase of an ongoingproject to model and simulate elections, election technologyand election incident reporting. I use the context models ofthe CommonKADS modeling methodology to examine howthe process of election incident reporting with EIRS withinEPC. Using the Brahms suite of multi-agent work practicesimulation tools, I then develop and simulate a model of anominal voter voting on an electronic voting machine as afoundational step towards simulating the process of electionincident reporting.

1. INTRODUCTIONElection protection – the process of providing legal and logis-tical assistance to increase electoral enfranchisement – hasits roots in the civil rights movement of the early 1960s fromthe creation of the Lawyer’s Committee for Civil Rights Un-der Law at the request of John F. Kennedy in 1963 to thepassing of the Voting Rights Act of 19651. Modern electionprotection involves litigation, real-time election incident as-sistance, voter education and election reform advocacy.

A significant part of election protection activities involve re-sponding to election incidents during an election cycle. Anelection incident, broadly, is any difficulty that an eligiblevoter has in the process of casting their vote. Tracking elec-tion incidents on election day allows election protection or-ganizations to keep abreast of trends in the failure of ourelectoral system. In turn, these organizations can use elec-tion incident reports as data for litigation or to persuadeelection officials to take needed actions to reduce the prob-lems people have in casting their vote.

1Voting Rights Act of 1965, Pub. L. No. 89-110, 79 Stat. 445(current version at 42 USC §§1971, 1973 to 1973gg-8 (2003)).

With the 2004 election cycle, election protection organiza-tions realized that having computerized databases of elec-tion incidents would provide two other strategic advantages.First, having information technology mediate the storageand communication of election incident data in real-time onelection day would allow more efficient resource allocationand case management. Second, the resulting database ofinformation on election incidents could serve as a valuableresource for academic analysis and data for prescriptive elec-tion policy and reform work2.

There were three major efforts to track election incidents inthe November 2004 general election. In a pilot study of exitpoll-based incident tracking, VoteWatch3 deployed teams ofpollsters to a random sample of polling places in BernalilloCounty, New Mexico and Cuyahoga County, Ohio [4].

In an effort sponsored by NBC News and MSNBC, theCommon Cause Education Fund together with a numberof academic partners4 encouraged voters to call a hotline(1-866-MYVOTE1) where they could find polling place in-formation, connect to their local election officials and makea statement about their voting experience which was thensaved as a 60-second digital audio file [2]. The MYVOTE1hotline received 210,000 calls of which 55,770 included state-ments. At the time of writing this article, approximatelyone-third of these statements have been transcribed andcoded.

Finally, The Election Protection Coalition (EPC) – a coali-tion of sixty non-profit organizations lead by The Lawyer’sCommittee for Civil Rights Under Law (LCCRUL) and ThePeople for the American Way Foundation (PFAWF) – mo-bilized thousands of volunteers to record and respond inreal-time to election incidents. Using a hotline (1-866-OUR-VOTE), an incident reporting system and a vast infrastruc-ture of volunteers, EPC recorded and responded to approx-imately 40,000 election incidents. Two EPC coalition mem-bers, The Verified Voting Foundation (VVF) and ComputerProfessionals for Social Responsibility (CPSR), developed

2Before the 2004 election, election incidents were largelyrecorded on paper forms and then placed in storage.3http://www.votewatch.us/ (VoteWatch is now known asthe Election Science Institute.)4These partners included The Fels Institute of Governmentat the University of Pennsylvania, The Reform Institute,The Hispanic Voter Project at Johns Hopkins Universityand The National Constitution Center.

a web-based information service and database for record-ing and tracking election incidents called the Election Inci-dent Reporting System (EIRS). The EIRS data is publicly-available and is currently being recoded by a team of VVFacademic volunteers including this author. EIRS data hasbeen the subject of a number of preliminary analyses [3, 6,8].

All these efforts recognize the power of data in shaping elec-tion policy at the local, state and federal levels. Electionadministration has become a complex activity with increas-ing points of failure at various levels in the system. With aclear picture of the functional and dysfunctional aspects ofthe system, we can better direct regulatory efforts, resourceallocation and future research. These points are especiallyimportant as we computerize our elections, vastly increas-ing the complexity and and functionality of the instrumentswith which we vote. Vendors and election officials both havean interest in tracking incidents concerning human factorsissues, malfunctions, bugs, discrepancies and other problemsthat arise with electronic voting technology in order to max-imize voter enfranchisement. In fact, one piece of legislationin the current Congress would require tracking of election ir-regularities in the Department of Justice’s Interactive CaseManagement System5.

In this paper, I describe an exploratory project I undertookto critically evaluate the use of EIRS within EPC. I firstbriefly discuss the context and motivation of this projectas well as give some background on the methodologies andtools that I use (§ 2). Given the complexity of the EPC’s ac-tivities, I chose first to build a theoretical model of electionincident response and recording using the CommonKADS [9]modeling methodology (§ 3). I then formalized these theo-retical models by porting them into a multi-agent computermodeling and simulation language, Brahms [11], designed toanalyze human and machine interaction (§ 4). I model andsimulate a typical voter using an electronic voting machinewith the voter having no problem. Finally, I discuss theimplications of this modeling and simulation exercise (§ 5),discuss the future of this work and then offer recommenda-tions for future election incident reporting (§ 6).

2. BACKGROUND & MOTIVATIONTo give the reader some needed context, this section ex-plains a bit about EIRS, CommonKADS modeling, Brahmsmodeling and simulation and how these later two can becombined into work-practice modeling and simulation.

2.1 EIRSIn early 2004, a group of organizations involved with electionprotection activities came together in Chicago at a meetingsponsored by the Quixote Foundation. Will Doherty, theexecutive director of VVF, presented the idea of creatinga tool for tracking election incidents. Other organizationsrecognized that they too could use such a tool. Months later,VVF and its plan to develop EIRS became a large part ofEPC’s plan for the November 2004 election.

5See §3(c) of the Voting Opportunity and Technology En-hancement Rights Act of 2005, H.R. 533, 109th Cong., 1stSess., (2005).

EIRS has three main features [13]. First, EIRS provides asuite of form-based interfaces for inputing information into aback-end database. EIRS data-entry forms exist for pollingelection officials about their local technology and proce-dures, for inputting information about election incidents andfor observing logic and accuracy testing of electronic votingsystems.

Second, EIRS has a map-based interface for retrieval andanalysis of election data: polling place and precinct loca-tions, voting technologies, voting regulations and proceduresand election incidents. This interface is publicly-availableon the web6 and also allows downloading incident data inspreadsheet form.

Third, EIRS incorporates volunteer management capabili-ties. Volunteers can sign up using EIRS and express theirinterests and skills and then volunteer managers from elec-tion protection organizations can assign tasks to volunteersand see to what extent tasks have been completed.

It is important to note that all of EIRS runs on open sourcesoftware and that this was a major factor in being able todevelop, deploy and sustain such a complicated informationservice.

2.2 CommonKADSCommonKADS is a knowledge engineering modeling method-ology, developed by a number of industry-university consor-tiums in Europe, for the design and analysis of knowledge-based systems and knowledge-intensive activities [9]. TheCommonKADS methodology consists of developing a num-ber of paper-based models; that is, models of the organi-zation in question, tasks that the organization undertakes,agents and systems that perform these tasks, the communi-cation necessary to make everything happen and the knowl-edge distribution within the organization and its systems.CommonKADS is an outgrowth of KADS [14] and Compo-nents of Expertise [12].

Knowledge engineering models can take a variety of differ-ent perspectives that each focus on different aspects of themodeling subject. CommonKADS grew out of the observa-tion that it would be desirable to have a critically chosensuite of models that emphasize certain aspects of knowledgeengineering and abstract others away. This is especially de-sirable with highly complex processes and organizations asthe modeler can temporarily ignore aspects not relevant tothe model at hand.

The core of the CommonKADS methodology (CommonKADS)is a series of modular worksheets for each of six models:the organization model, task model, agent model, knowl-edge model, communication model and the design model.In my modeling I focus on the first three of these, called theCommonKADS Context Models and will save detailed ex-ploration of the Communication and Knowledge Models forafter I have gathered more contextual data7. Through mod-eling various axes of knowledge-intensive applications, we

6http://maps.voteprotect.org/7As I am dealing with an existing system, I do not considerthe design model.

examine the system from a variety of different perspectiveswith different central concerns.

The CommonKADS modeler conducts an iterative processwhich involves multiple iterations on the content in the var-ious worksheets for each model. The CommonKADS work-sheets were designed to aggregate information about the or-ganization and processes in question from which the modelercan better answer questions about how they should changein the future. Because of its strengths, CommonKADS isused in Europe as a standard process re-engineering method-ology. The CommonKADS models are also well-suited asinput for Brahms modeling and simulation.

2.3 BrahmsBrahms is a multi-agent modeling and simulation program-ming language and development environment for analyzingwork practice [11]. Brahms consists of a language specifica-tion8 and an integrated development environment in whichBrahms models are developed, debugged, executed and dis-played.

Brahms was designed over the past fifteen years as part ofresearch conducted at the Institute for Research on Learn-ing (IRL), NYNEX9 Science & Technology Institute, andmost recently in the Work Systems Design and EvaluationGroup within the Computational Science Division (CodeIC) of NASA’s Ames Research Center. Modelers have usedBrahms in a number of research projects and recently fordeveloping work practice support tools for future mannedmissions to the Moon and Mars [1, 10, 11].

The discussion of “Related Work” in Sierhuis and Clancey2002 [10] gives a particularly succinct description of whya modeling paradigm such as Brahms needed to be devel-oped. Essentially, techniques such as CommonKADS andContextual Inquiry [5] for formal modeling of systems designexisted but none incorporated ideas of participatory designand modeling the work practice of individual human andtechnological agents at the activity level. Brahms modelingstarts with formal paper-based modeling and then the vali-dation of the model comes in developing the simulation anditerating on historical or prescriptive system behavior withstake-holders. In allowing the modeler to formally specifyand simulate systems design, Brahms acts as bridge betweenanalysts, administrators and workers.

2.4 Why work-practice modeling?Modeling the work practice of individuals in an organizationor process allows us to specify what actually happens duringwork. As in any modeling activity, the modeler first collectsdata and observations about the work, the organization andthe people doing the work. Then, the modeler formalizesthe model using techniques such as CommonKADS or Con-textual Inquiry. Any data that may be missing can thenbe located or noted as a possible deficiency in the model.Work-practice modeling serves as a necessary starting pointfor Brahms modeling and simulation.8http://www.agentisolutions.com/documentation/documentation.htm9NYNEX merged with Bell Atlantic in 1996 and then BellAtlantic with GTE in 1998 to form Verizon Communica-tions.

Elections and election incident reporting are very complexactivities. With over 122 million voters, 200,000 pollingplaces, 3,000 election jurisdictions and approximately onehundred organizations involved in election protection in theUnited States, there is a wide variety in how votes are cast,counted and problems reported and reconciled. To bothbe able to understand and document the changes in votingpractices due to procedures, technology or disability, it isimportant to model these practices at the activity level andthen have formal models of how things can go wrong.

Work-practice modeling also allows us to think of systems-level usability, not simple if one subsystem is usable to avoter with a set of specified characteristics. With work-practice modeling we can formally document the currentactivities involved in voting and how voters with differentcharacteristics respond to the same formal system. Withthis work practice modeled and simulated in Brahms, we canalso test alternative models and configurations and ascer-tain, before deployment, if specific policies and procedurescould lead to increased enfranchisement, resource conserva-tion and other improvements.

2.5 Data GatheringI should state up-front that I was involved with EPC activi-ties of the fall of 2004, before I had any notion of the currentproject. In the months before the election, I had developeda set of Electronic Voting Machine Information Sheets10 forthe Electronic Frontier Foundation (EFF) that gave techni-cal details on electronic voting machines and problems theyhad experienced in the past. As part of this work, I was sta-tioned at the Folsom St. Legal Command Center (LCC) inSan Francisco on Election Day, 2004. My task was to answerquestions from attorneys in the field that had been trainedby the EFF to handle voting technology issues at the locallevel. This involvement allowed me to thoroughly observeelection incident reporting in one instance on Election Dayand my larger involvement undoubtedly added to this bodyof knowledge.

To enrich my understanding EIRS and EPC, I conductedqualitative interviews with a set of six individuals at highlevels of their organizations within EPC and EIRS. I alsohad access to transcripts of eight interviews and other datacollected by two members of the EIRS user interface team,Catherine Marshall and Jeff Johnson, in their study of EIRSpresented at SIGCHI 2005 [7]. I interviewed each person forapproximately 3/4 of an hour and asked them a range ofquestions about their role in EPC and relationship to EIRSas well as questions about problems and opportunities withEIRS and EPC.

I also immersed myself into the EIRS team and began takingpart in the EIRS-Leaders group which consisted of leaders ofthe EIRS subteams (See Fig. 1). As I developed my models,I asked for feedback from the EIRS-leaders group. The EIRSteam was very gracious and allowed me access to logisticaldocumentation when necessary.

3. COMMONKADS MODELING10http://www.eff.org/Activism/E-voting/

I chose to develop the CommonKADS Context Models (Or-ganization, Task and Agent Models) for the process of votingand the process of election incident reporting. As casting avote is not as complicated as election incident reporting, Idescribe the details of what I found in the modeling processfor each model relevant to election incident reporting. Asthe Context Models are crucial to developing the Knowledgeand Communication Models, I’ll need to work more closelywith EPC member organizations that I don’t have as readyaccess to to improve these models before moving on.

3.1 Organization ModelThe Organization model involves building out various work-sheets that deal with problems and opportunities, breakingdown processes and identifying knowledge assets. Figure 1and figure 2 show the organizational structure of EIRS andthe EPC legal deployment on Election Day.

3.1.1 Problems and OpportunitiesWhile conducting interviews, a frequent concern that wasraised was that the calling infrastructure wasn’t ideal. Thatis, the infrastructure that routed hotline calls from the voterto the proper Local Call-In Center had some deficiencies in-cluding load-balancing problems which likely resulted in thehotline going down for a while during Election Day. Whena Call-In Center became busy and could handle no morecalls, all calls were routed to the main Call-In Center inWashington DC. This resulted in the DC Call-In Center re-ceiving many calls throughout the day while other Call-InCenters that could have shared some of this load were rel-atively unoccupied. Also, the majority of calls from voterswere polling place location queries, which arguably didn’tneed to be answered by humans.

Another common theme was that organizational communi-cation between EPC members was rough at times. Someorganizations seemed wary of sharing volunteer informationwith other organizations for fear of volunteer defection. Itappears that some EPC members were concerned that thequality and public availability of data in the EIRS databasecould reflect poorly upon themselves. Finally, there wasno formal internal decision-making body or process withinEPC. This seems to have resulted in some confusion aboutwhat actions could be claimed to be conducted under theauspices of EPC and which should be clearly disassociatedfrom the EPC.

The largest problem for the EIRS project within EPC seemsto have been that there were conflicting requirements forEIRS development from different EPC member organiza-tions. EIRS was developed over a very short developmentcycle and the two principle decision-making organizationswithin EPC, LCCRUL and PFAWF, did not formally spec-ify their requirements for the tool and did not seem consciousof logistical and managerial requirements for software devel-opment. One stark example of these conflicts were in EIRS’volunteer management features. PFAWF manages volun-teers using third-party software but LCCRUL did not usevolunteer management software and wanted this capabilityincorporated into EIRS. On Election Day, the staffing mod-els for call centers was very much plug-and-play with EPCmember organizations trying to effectively utilize all volun-teers that they had on hand. The EIRS volunteer man-

agement software did not allow a “lightweight” registrationand each volunteer had to answer a list of questions beforebeing able to submit incident reports. The result of this con-flict between having rich volunteer management and a highlyfluid staffing model was that supervisors used workstation-specific accounts for EIRS. This completely undermined thesecurity model of EIRS where each incident report shouldhave been attributable to an individual volunteer.

Another problem, which I’ve discussed extensively in an-other paper11, is that the map-based analysis tool in EIRS– that is used for analyzing and exploring the incident datain real-time – did not meet the needs of EPC analysts. Infact, due to their desire to have some degree of complexBoolean search, IT staffers at PFAWF even created theirown no-frills interface the night before th Election.

As mentioned above, the EIRS access-control model didnot work as planned and was found to be impractical bysome EPC members. The access-control hierarchy was de-signed to minimize the poisoning or pollution of the incidentdatabase by malicious actors, however it turned out to beoverly complex. Some Call-In Center supervisors could notauthorize some volunteers to submit incidents, and this re-sulted in these incidents not showing up in the databaseuntil an EIRS representative could authorize their accounts.

Finally, technical abilities were not distributed uniformlyamongst EPC organizations. Extensive set-up and coordi-nation was required of key technical personnel within EIRSwithout them previously knowing that they would be re-sponsible for this function. This is understandable as thiswas the first time EPC used computerized information tech-nology to coordinate and record incident reporting activities.

In the realm of opportunities, the following were frequently-stated desires:

• EIRS and EPC should learn from the first deploymentof information technology as a major part of electionprotection processes and preparation. They have theopportunity to treat election protection as an iterativedesign process where the various functions of EIRS aredesigned to their driving uses.

• With this data, EPC can leverage the data collectedin 2004 election to get a better understanding of theelection landscape and where policy-related and logis-tical resource can be better targeted. A large part ofthis is the current process of supplementary coding12

of the 2004 data.

• It is important to engage in more forward-thinkingplanning with increased communication between coali-tion members. The election incident reporting processand EPC’s preparations for key elections might bet-ter be managed using a paradigm like that of software

11I conducted a preliminary usability analysis of the EIRSmap interface for a graduate class in Needs Assessment andUsability at UC Berkeley. This report is available from theauthor upon request.

12This involves recoding the 40,000 incidents in the EIRSdatabase to get better quantitative and qualitative insightinto its contents.

development projects where various functions – devel-opment, quality assurance, marketing, logistics, etc.– have to be tightly integrated and tied to milestones.This could save all parties a lot of time and frustrationand increase communication within EPC.

• One possibility that was considered for the 2004 elec-tion was that of allowing voters to enter their own elec-tion incidents into the EIRS database via a web form.However, this was a low priority in 2004 and there aresignificant concerns with the implementation of thiskind of feature.

• There is a desire to find ways to increase the quality ofthe incident data in the EIRS and Verifier13 databaseto adequately support real-time response and detailedpost-election analysis.

• A key opportunity related to voting technology wouldbe providing detailed voting machine and election in-formation that included information suitable for re-sponding to incidents we know of from 2004 data.

3.1.2 Relevant ProcessesThere are two main processes in elections incident report-ing as undertaken by EPC that we can analyze rationally:the process of Call Center Operators (CCO) answering hot-line phone calls and the process of Mobile Field Attorneysresponding to election incidents in the field14.

First, there is the process of incident reporting in general(see figure 4). This involves a voter, before or after havingvoted (see figure 3), calling the EPC hotline and gettingassistance or reporting an irregularity. A CCO answers thephone call (see figure 5) and attempts to conduct a form oftriage to see how they can best be of assistance. Finally,MFAs may respond to certain incidents (see figure 6) in thefield at polling places or at elections headquarters. Thesetasks are discussed in more detail in § 3.2.

3.1.3 Knowledge AssetsWhat types of knowledge assets are key in the above tasks?First, there are various knowledge assets surrounding vot-ing technology. Voting technology material that agents mayneed includes how to vote with a given technology and whattechnology is being used where. I spearheaded the devel-opment of materials that detailed how to vote on electronicvoting machines, however, there are other types of votingtechnology, such as lever machines and optical scan tech-nologies, that I did not produce materials for. The Verifierdatabase was the main resource for ascertaining what tech-nology was being used across the country. Finally, as men-tioned in the discussion of opportunities above, there was aneed for a resource that could detail what kinds of voting

13The Verifier database contains information on voting tech-nology, election officials and local election procedures

14It’s important to note that there are other processes inelection incident reporting that should be analyzed. For ex-ample, the work practices that take place in the NationalLegal Command Center are essentially what drives the ac-tions of the EPC. However, these actions are largely ad hocand developed as problems unfold and litigation needs to befiled. I hope to pin this down more completely in the future.

technology problems are easy to reconcile (such as miscali-brations of the touchscreens on DREs), and which are morecomplex (such as rebooting of machines).

Second, as so many of the calls into the hotline were vot-ers that wanted assistance finding their polling place, manyCCOs used tools such as http://mypollingplace.com andweb-based tools that mapped area codes and addresses tozip codes to find polling place information.

Third, there is a larger diffuse question of the appropriateresponse based on an incident. While some problems hadclearly prescribed responses, others did not. Frequently, theCCO would remember how they or other CCOs near themhad responded to a similar incident earlier in the day andrespond in kind. However, with more complicated problemsit was frequently the job of a legal commander or a technicalspecialist to decide what should be done and what the votershould be told to do.

Fourth, an even more abstract knowledge-related resourcewas the decision of if and when legal complaints should befiled. These decisions are not to be taken lightly and hingeon a number of factors including data on the problem athand, the availability of witnesses to give depositions to sup-port the complaint and strategic issues of whether or not acomplaint will succeed and what possible impact a successfulor failed complaint could mean. This is further complicatedby the fact that individual EPC organizations have differentsponsor bases and focuses and that there may be a disagree-ment among EPC members as to how important a particularproblem is15.

Finally, if a complaint does need to be filed or if an incidentreport needs follow-up this inevitably requires access to per-sonal information. Incident recorders have access to the fullincident reports that they have entered, but no others andeven if an EPC volunteer has access to the personal informa-tion of voters, there was a mysterious protocol for clearingthe use of this personal information with other EPC organi-zations. If anyone wanted to use personal information, theywere to send a request to an EPC email list that consisted ofa number of high-level EPC organizers. If there was no ob-jection within 24 hours, the use of the personal informationwould be approved.

3.2 Task ModelIn this section I discuss the tasks I have modeled that I hopeto translate into Brahms code.

3.2.1 VotingFigure 3 shows an activity model of the task of a typicalvoter voting on a Sequoia Voting Systems AVC Edge16 vot-ing system. As voting is the fundamental activity around

15For example, the EFF almost filed a legal complaint inBroward County, FL due to reports that some had to waitin line for hours on end to cast their votes. There is someindication that other EPC organizations did not want a com-plaint to be filed on this issue and EFF staff spent valuabletime trying to ascertain if a complaint would be proper andsuccessful.

16http://www.eff.org/Activism/E-voting/20040818_sequoia_avc_edge_v0.8.pdf

which all of election incident reporting is centered, I havestarted the Brahms modeling of election incident report-ing with modeling this activity. This activity has two mainagents, the pollworker and voter, and two objects, the votingmachine and the smartcard token17.

The voter begins the process by approaching the registrationdesk. The pollworker asks for the voter’s name and thenchecks a registration book to see that the voter is registeredto vote in the polling place and that the voter hasn’t alreadyvoted in the election or received an absentee ballot in themail. If the voter is registered and hasn’t voted, the pollworker issues a smartcard token that authorizes the voter tovote once. The voter takes the smartcard to an unoccupiedvoting machine and inserts the smartcard into the machine(much like an ATM card is inserted into an ATM). Themachine then asks the voter to choose a ballot language andthen displays voting instructions for the voting machine inthe chosen language. The voter then votes in each conteston the ballot and advances the screen to the next contestas necessary. After the last contest, the machine displays ascreen that shows all the choices that the voter has made forthis election. If the choices are incorrect, the machine allowsthe voter to advance to the errant contest(s) and change theincorrect vote(s). If the choices are correct, the machinerecords the ballot in multiple memory locations internal tothe machine and then ejects the smartcard (which has beendeauthorized and cannot be used to vote again until reset).The voter returns the smartcard to the poll worker and thepoll worker gives an “I Voted!” sticker to the voter.

This process is more complicated than for paper-based vot-ing where a voter is interacting with the ballot in a formthat they are familiar with and then deposits the ballot intoa ballot box. Also, this model is for a non-disabled voterwho has no problem casting their vote18.

3.2.2 Incident ReportingI have one activity model for the process of incident report-ing in general (see figure 4). From this high-level view, Iwas able to detail the process of a CCO answering calls andengaging in incident triage (see figure 5) as well as a MFAhandling an election incident at the local level (see figure 6).

Figure 4 illustrates that the high-level process of incidentreporting is relatively straightforward. Voters first cast theirvote and either have a problem voting or have no problem.If a voter has a problem they will call the EPC hotline toreport the problem. At the Call-In Center a CCO answersthe call and tries to determine if it is a simple problem (suchas polling place location query) or a more complex problemthat they need assistance to reconcile. If the problem issimple and the CCO has the resources to help the voteror give the voter adequate information about the problem,the CCO can simply explain the problem, EPC’s current

17A smartcard is a credit card-sized object that allows thevoter to vote once.

18It is also important to note that there are many ways inwhich voters can have trouble casting their vote both beforethe enter the polling place (such as with registration-relatedproblems and intimidation) or after (such as machine mal-functions or mishandling of voting equipment that causesloss of votes).

action with respect to the problem and what the voter cando if the problem needs correction. If the problem is morecomplicated, the CCO will have to ask for the assistanceof a Legal Commander (LC) or a Technical Specialist (TS)from the Legal Command Center (which may or may notbe co-located). The LC may dispatch a MFA to the pollingplace in question (see below). If the problem is novel or ofspecific concern to the LC or TS, they may have to contactsomeone at the national level to report the problem and seekadvice for subsequent action.

Figure 5 details the activity model for EPC Call CenterOperators (CCO) answering and responding to calls with afocus on how voting technology incidents are handled. TheCCO answers the call and first sees if the voter simply wantsto know their polling place. Then the CCO determines if thecall only needs documentation – for example, if the problemis known and is being handled. If the problem is more com-plicated. the CCO determines if it is an issue with votingtechnology or not. If it is not technology-related the CCOholds up a red card that signals the CCO needs help from aLegal Commander. If the problem is technology-related theCCO tries to determine if it the voter simply needs instruc-tions on how to vote or if the problem is more complicated.If the voter needs voting instructions for a specific votingtechnology, the CCO asks what kind of voting technologythe voter is using. If the voter doesn’t know what type ofvoting technology they will be voting on the CCO can find itusing EIRS’ Verifier database that lists types of voting tech-nology for all counties in the US. The CCO then finds theinstructions appropriate for the voting technology in ques-tion and describes how to vote to the voter. In all cases, theCCO documents the incident in EIRS. The CCO or otherCall Center staff may mark an incident for follow-up in cer-tain cases. (Note: my data on the incident-triage process isparticularly sparse. I invite anyone with specific informationto contact me and help me to refine this model.)

Figure 6 shows the activity model of EPC mobile field at-torneys (MFA) responding to election incidents a the locallevel. A team of two MFAs is assigned to approximately sixpolling places. They typically patrol these six polling placesuntil they receive a report of an election incident. Imme-diately, they must decide if a response involving physicallygoing somewhere is necessary or if calling the appropriatecontact on the phone would be better. If a physical responseis necessary, the MFA team goes to either the polling placein question or the election headquarters where election of-ficials are stationed. In each case, the MFA team has tobe granted access to the facility; if they are unable to getaccess they will call the election official or their MFA Man-ager. When they are granted access to the polling place orelections headquarters they will do one or more of gatheringinformation, requesting action from the poll worker or elec-tion official and, at the polling place, interacting with votersthat might be affected by the problem. In both cases, theMFA team documents what actions they’ve taken by call-ing their MFA Manager or, in some cases, the EPC hotline.In all cases, the MFA team may follow-up on voting inci-dents with the voter, election official or EPC representativeworking on the problem. (Note: There are other models forfield attorneys for different EPC coalition members like theNAACP. Please get in touch with me if you know of other

models that were implemented on November 2, 2004.)

Understandably, I have been unable to detail a number ofprocesses in election incident reporting usually because theyare ad hoc and depend on the specific circumstances of thesituation at hand. For example, the coordination amongstlegal commanders at the local and national level is currentlyopaque to me. Similarly, how high-level organizational mem-bers decided when and if to litigate is not currently clear.

3.3 Agent ModelThere are a number of agents in this picture. Specifically,there are:

• Voters – Voters both cast votes and report electionincidents. Voters have the widest range of character-istics as every able-minded United States citizen hasthe right to vote and may choose to exercise this right.Voters will span the range of familiarity with the vot-ing process, with voting on certain equipment and withphysical and cognitive disability. Typical activities in-clude registering to vote, choosing a voting language(i.e., being somewhat literate), voting in a contest, re-viewing their choices and taking voting tokens to andfrom voting booths.

• Poll Workers – Poll workers are the agents which fa-cilitate the casting of votes by voters in the pollingplace. Poll workers are largely senior citizens andmany have assisted in many elections before. Typicalactivities include setting up a polling place an its vot-ing booths, checking the registration status of voters,issuing voting tokens, reseting voting tokens, assistingwith polling place problems and closing elections in apolling place.

• Election Officials – Election officials plan and ad-minister elections activities. They are as varied incharacteristics as voters are save that they have quitea bit of experience with election activities. Typical ac-tivities include preparing ballot information for pollingplaces, sending out and counting absentee ballots, reg-istering voters, hiring and training poll workers, coor-dinating election activity on Election Day, communi-cating election results to the public, media and othergovernment agencies and procuring voting equipmentand supplies.

• Call Center Operators – CCOs are EPC volunteersthat answer phone calls on and around Election Day.These people are largely attorneys and law studentsand have a more narrow set of characteristics thanthe agents above. For example, they may have spe-cific legal knowledge and past experience with electionprotection. Typical activities include answering phonecalls, incident triage, looking up polling place informa-tion and voting technology details, submitting incidentreports and working with legal commanders and tech-nical specialists to solve voters problems.

• Legal Commanders – LCs are essentially experi-enced CCOs with specific legal training and coordi-nation instructions. They act both as strategists and

CCO supervisors. Typical activities include managingCCO shifts, handling complicated election incidents,dispatching MFAs and working with LCs at the na-tional level to coordinate the aggregation of informa-tion and possible litigation.

• Mobile Field Attorney Managers – MFA Man-agers direct the action of and support teams of MFAs.An MFA Manager could also be a Legal Commanderbut typically handles five teams of two MFAs each fora total of 30 polling places. Typical activities can in-clude activities of an LC as well as direct communica-tion with their teams of MFAs in the field.

• Mobile Field Attorneys – MFAs respond to inci-dents in the field. Typical activities include respond-ing to election incidents at polling places as well as atelection headquarters. They also do a fair amount ofcoordination via the phone.

3.4 Thoughts on the Knowledge and Commu-nication Models

I have significantly developed the three Context Models ofCommonKADS and have left the Knowledge and Communi-cation Models for future work. However, I do have thoughtson these models.

Constructing the CommonKADS knowledge model involveszeroing in on knowledge assets, specifying knowledge that isreused throughout certain tasks and refining this knowledgewith an eye towards packaging this knowledge for futureuse. However, as I’ve noted above (§ 3.1.3) there are a fewareas where knowledge could arguably be better packagedand other areas where it isn’t even clear what knowledge isbeing brought to bear. I hope to expand my data collectionto include ad hoc and planned strategies for these facets ofelection incident reporting.

The Communication model involves specifying how knowl-edge is communicated throughout the organization. Quitea bit of the communication on Election Day occurred viaphone, email and, in some cases, over instant messaging ser-vices. However, my insight into the content of these commu-nications is only as deep as my own experience and the CallCenter/Legal Command Center at which I was stationed.I hope to also better specify what types of communicationhappened between high-level EIRS and EPC members inthe future.

4. BRAHMS MODELING AND SIMULATIONAs a first step towards the modeling and simulation of EPCactivities on Election Day 2004, I decided to code a Brahmsmodel for a typical voter casting a vote on a typical comput-erized voting machine19. I decided not to attempt to codeup the other two activity models that I have reasonably well-specified (Figures 5 and 6) without further refinement of theContext Models and some significant work on the Commu-nication and Knowledge Models.

19The activity models and Brahms source code for thisproject is available here: http://pobox.com/~joehall/brahms/

As a primer to some of the visualizations I use from theBrahms AgentViewer software, please take a look at fig-ure 7 which illustrates the common features of a Brahmssimulation’s output.

I modeled and simulated the process for voting detailed in§ 3.2.1. I modeled a typical voter voting on a Sequoia AVCEdge direct recording electronic voting machine and I chosea hypothetical school in hypothetical county that uses thistype of voting machine.

4.1 Checking Voter RegistrationThe simulation starts with the Voter waiting in line andnoticing that the registration desk is unoccupied (see fig-ure 8). The voter moves to the desk and broadcasts to therest of the polling place that the registration desk is nowoccupied. The poll worker waits for the voter to approachand then asks the Voter to state her name. The voter com-municates her name to the pollworker and the pollworkerthen checks to see if this name exists in the registration pollbook and if it has a signature next to it. If it does existand doesn’t have a signature next to it, the poll worker asksthe voter to sign the poll book. The voter signs the pollbook and then the pollworker takes a smartcard out of abox filled with smartcards and gives one to the voter. TheVoter then moves to the voting booth and broadcasts to thepolling place that the registration area is now unoccupied.The poll worker agent then becomes available to help othervoters.

4.2 Voting and Casting the BallotFigures 9 and 10 show the output of the Brahms simulationbetween the Voter and the Sequoia AVC Edge voting ma-chine. The Voter approaches the machine and inserts thesmartcard. This takes the machine off of its WaitForVoter

activity and then it displays a screen that asks the voter tochoose a ballot language. The voter communicates her bal-lot language to the machine and the machine then displaysvoting instructions in this language. Then, the voter readsthese instructions and advances to the next screen.

In this simulation I have chosen an arbitrary number of con-tests to vote on, four, and made each contest the same. TheVoter waits for the contest screen to display and then spendssome time making her decision on that contest. When thevoter is done, she advances to the next contest. After thelast contest, the next screen displayed shows a summary ofthe votes the machine has recorded. The voter then has theopportunity to correct contests that were recorded incor-rectly or cast her ballot. In this case, there are no problemswith the ballot and the voter simply casts her ballot. Themachine takes five seconds or so to record the ballot in mul-tiple memory locations and then ejects the smartcard.

4.3 Returning the SmartcardFigure 11 shows the last part of the simulation where theVoter returns the smartcard to the poll worker. The Votertakes the smartcard from the machine and moves back tothe registration desk. The Voter broadcasts to the rest ofthe polling place that the registration desk is now occupied.The Voter gives back the smartcard and leaves the pollingplace. The poll worker places the smartcard in the box and

then becomes available to help other voters after setting aworld fact that the registration area is unoccupied.

5. DISCUSSIONThis model and simulation reasonably represents what amajority of voters experience while casting their vote. Thetimes for each activity are realistic and this is roughly the to-tal amount of time that a voter would take to cast a vote. Ofcourse, not all voters experience such a smooth voting pro-cess and the longer-term aims of this project are to modeland simulate those as well.

5.1 Future ResearchIn the future, I see this research doing a number of things.First, I would like to model and simulate additional typesof voting technology from simple paper ballots, punchcardsand optical scan to more complicated lever machines andother types of computerized voting equipment. This workwill be in conjunction with my efforts as the chair of theExisting Systems Explication Working Group of the VotingSystem Performance Rating Organization.

The ultimate aim will be to formally define and simulatethe agents and knowledge resources that come together inthe process of election incident reporting by the EPC. Thiswill take more than just developing models and simulationsin that there is much I currently don’t know or that isn’teasily knowable. Hopefully, this paper will begin the processof formalizing the process of elections incident reporting.

6. RECOMMENDATIONSIn the course of building the Context Models for EIRS withinEPC, I heard a number of recommendations that might beconsidered for the future:

• Leverage the technical knowledge within the coalitionto make the calling infrastructure more cost effectiveand reliable as well as possibly expand automated ser-vices. For example, it would be more efficient to havetasks like polling place look-up handled by an auto-mated interface.

• Heighten communication within the coalition so thatthe proper parts of different organizations are awareof each other’s needs and abilities. This will increaseefficiency as well as allow EPC members to better planfor their roles and ultimately will help the coalition toensure that legally-cast votes are counted.

• There should be some decision-making body, like anexecutive board, for certain activities of the coalitionand possibly a lightweight coalition agreement thatmember organizations agree to that specifies what ac-tivities can be done under the auspices of EPC andwhich clearly cannot. This will help to define wheredifficult decisions should be directed as well as makethe coalition more of a coalition and less of a hierarchy.

• Coalition members, an executive board thereof or a de-sign sub-committee should agree upon functional de-sign requirements for the incident reporting system– form interface, reporting interface, telephone sys-tem and volunteer coordination tools – and allow for

enough time and some resources for user-centered de-sign (UCD) and testing.

• The Coalition should decide on an access-control modelfor EIRS that allows for different coalition staffingmodels while also conscious of the need for securityand follow-up with incident recorders as well as inci-dent reporters.

• Areas where certain coalition member activities aredependent on other member activities need special at-tention and joint-planning.

7. CONCLUSIONElection incident reporting will be an increasingly impor-tant part of our electoral system. As the process of votingbecomes more complicated and we ask more from the toolswe use to cast votes, it is inevitable that their complexityincreases. With complexity comes a higher probability forerrors and increased opportunity for malicious manipulationof vote data. The best bet for resolving these problems inreal-time and to ensure that they don’t happen again is tocollect as much information as we can and track incidentsas they unfold.

It is beyond doubt that election incident reporting as a gen-eral activity should continue and that this maybe a functionthat the federal government should undertake. However, asthere is only one serious proposal along these lines on thetable right now which is not getting much attention, non-profit election protection organizations should prepare tomake election incident reporting a significant part of whatthey do. Further, it is important that the few resourcesavailable for this activity are used to the best of their abil-ity.

Formally delineating what work people do, what knowledgeresources they use in this work and the larger systems-leveldynamics will go a long way towards re-engineering the pro-cess of election incident reporting.

The goal here is to not allow the wave of voting technologydrown public-interest election protection activities; instead,we can use information technology and eager volunteers toensure that problems introduced by complicated technolo-gies in the voting process are minimal and, most impor-tantly, fully recoverable.

8. REFERENCES[1] W. J. Clancey, M. Sierhuis, C. Kaskiris, and

R. v. Hoof. Advantages of brahms for specifying andimplementing a multiagent human-robotic explorationsystem. In Proceedings of the The 16th InternationalFLAIRS Conference, St. Augustine, Florida, 2003.

[2] Common Cause. Nearly 210,000 calls to voter alertline, final tally shows. Press release, November 2004.http://www.commoncause.org/site/apps/nl/

content2.asp?c=dkLNK1MQIwG&b=194883&ct=293705.

[3] D. Dill and W. Doherty. Electronic voting systems: Areport for the national research council. Whitepapersubmission to the National Research Council’sCommittee on Electronic Voting, Dec. 2005.

http://www7.nationalacademies.org/cstb/

project_evoting_vvf.pdf.

[4] S. Hertzberg, S. Gabbard, N. Vicinanza, andF. Scheuren. Votewatch new mexico exit poll report:Presidential election 2004. Technical report, ElectionScience Institute, 2004.http://www.votewatch.us/Members/

StevenHertzberg/report.2004-12-13.1986469362/.

[5] K. Holtzblatt and H. Beyer. Contextual Design: ACustomer- Centered Approach to Systems Design.Morgan Kaufmann, San Francisco, 1998.

[6] P. N. Howard. In the margins: Political victory in thecontext of technology error, residual votes, andincident reports in 2004. Center for Communicationand Civic Engagement Working Paper # 2005-1,January 2005. http://depts.washington.edu/ccce/assets/documents/withinthemargins.pdf.

[7] J. Johnson and C. Marshall. Convergent usabilityevaluation: A case study from the eirs project. InProceedings of CHI. ACM, 2005.

[8] D. K. Mulligan and J. L. Hall. Preliminary analysis ofe-voting problems highlights need for heightenedstandards and testing. Whitepaper submission to theNational Research Council’s Committee on ElectronicVoting, Dec. 2005. http://www7.nationalacademies.org/cstb/project_evoting_mulligan.pdf.

[9] G. Schreiber, H. Akkermans, A. Anjewierden,R. de Hoog, N. Shadbolt, W. V. de Velde, andB. Wielinga. Knowledge Engineering andManagement: The CommonKADS Methodology. MITPress, 2002.

[10] M. Sierhuis and W. J. Clancey. Modeling andsimulating work practice: A human-centered methodfor work systems design. IEEE Intelligent Systems,17(5), 2002.

[11] M. Sierhuis, W. J. Clancey, and R. v. Hoof. Brahms: amultiagent modeling environment for simulating socialphenomena. In Proceedings of the First conference ofthe European Social Simulation Association (SIMSOCVI), Groningen, The Netherlands, 2003.

[12] L. Steels. Components of expertise. AI Magazine,1990.

[13] Verified Voting Foundation. Election incidentreporting system. Project description web page, 2004.http://www.verifiedvotingfoundation.org/eirs.

[14] B. J. Wielinga, A. T. Schreiber, and J. A. Breuker.KADS: A modelling approach to knowledgeengineering. Knowledge Acquisition, 4(1):5–53, 1992.

Figure 1: The organizational hierarchy of the Election Incident Reporting System (EIRS) within the ElectionProtection Coalition (EPC). The EPC is the umbrella organization under which coalition members operatedand cooperated. The decision-making organizations within EPC were the Lawyer’s Committee for CivilRights Under Law (LCCRUL) and the People For the American Way Foundation (PFAWF). Other coalitionmembers, such as EIRS developers the Verified Voting Foundation (VVF) and Computer Professionals forSocial Responsibility (CPSR), were one step down on the organizational hierarchy and at times played the roleof contractor. Within EIRS there were a number of teams supervised by a management team that consistedof the two executive directors of VVF and CPSR and the EIRS project manager, John McCarthy. The otherEIRS teams in the figure had varying membership (dotted borders), roughly the same organizational cloutand are described in § 3.1.

Figure 2: This figure depicts the legal deployment model of EPC as organized by the Lawyer’s Committee.There was one National Legal Command Center and three National Call-In Centers which handled legalissues and calls from across the nation. At the local level there were about 30 Legal Command Centers(LCC) organized by Local Legal Coordinating Committees each LCC had a number of Mobile Field Attorney(MFA) Managers. There were about eight Local Call-In Centers that took calls from adjacent areas and eachCall-In Center had a battery of Call Center Operators. Each MFA Manager directed the actions and assistedfive teams of Mobile Field Attorneys made up of two attorneys each that covered six polling places at thedistrict level. Finally, some individual attorneys and law students served as Stationary Legal Volunteers attargeted polling places.

Figure 3: The activity model for a typical voter (non-disabled) voting on a Sequoia Voting Systems modelAVC Edge Direct Recording Electronic (DRE) Voting Machine. A pollworker first has to check that thevoter is registered to vote and that they haven’t voted already. If so, the pollworker issues a credit card-likedevice called a smart card that authorizes the voter to vote once on the voting machine. The voter then goesto the voting machine and inserts the smartcard into the machine and then selects the language they wouldlike to vote in. After reading the voting instructions, the voter votes for each contest and then advances tothe next contest. When all contests have been voted on, the voter reviews his or her choices on a reviewscreen that displays the votes the machine is recorded. If any choices are incorrect, the voter can advanceto each contest where their choice was incorrectly recorded and correct their vote. If all the choices arecorrect, the voter can then cast their ballot which is recorded in multiple electronic memory locations withinthe machine. The machine then resets the smartcard and ejects it. The voter returns the smartcard to thepollworker and is given an “I Voted!” sticker.

Figure 4: The high-level activity model of the process of dealing with election incidents. Voters first casttheir vote and either have a problem voting or have no problem. If a voter has a problem they will call theEPC hotline to report the problem. At the Call-In Center a Call Center Operator (CCO) answers the calland tries to determine if it is a simple problem (such as polling place location) or a more complex problemthat they need assistance to reconcile. If the problem is simple and the CCO has the resources to help thevoter or give the voter adequate information about the problem, the CCO can simply explain the problem,EPC’s current action with respect to the problem and what the voter can do if the problem needs correction.If the problem is more complicated, the CCO will have to ask for the assistance of a Legal Commander (LC)or a Technical Specialist (TS) from the Legal Command Center (which may or may not be co-located). If theproblem is novel or of specific concern to the LC or TS, they may have to contact someone at the nationallevel to report the problem and seek advice for subsequent actions.

Figure 5: The detailed activity model for EPC Call Center Operators (CCO) answering and respondingto calls with a focus on how voting technology incidents are handled. The CCO answers the call and firstsees if the voter simply wants to know their polling place. Then the CCO determines if the call only needsdocumentation – for example, if the problem is known and is being handled. If the problem is even morecomplicated the CCO determines if it is an issue with voting technology or with other legal matters. Ifit is not technology-related the CCO holds up a red card that signals the CCO needs help from a LegalCommander. If the problem is technology-related the CCO tries to determine if it the voter simply needsinstructions on how to vote or if the problem is more complicated. If the voter needs voting instructionsfor a specific voting technology, the CCO asks what kind of voting technology the voter is using. If thevoter doesn’t know what type of voting technology they will be voting on the CCO can find it using EIRS’Verifier database that lists types of voting technology for all counties in the US. The CCO then finds theinstructions appropriate for the voting technology in question and describes how to vote to the voter. In allcases, the CCO documents the incident in EIRS. The CCO or other Call Center staff may mark an incidentfor follow-up in certain cases. (Note: my data on the incident-triage process is particularly sparse. I inviteanyone with specific information to contact me and help me to refine this model.)

Figure 6: One activity model of EPC mobile field attorneys (MFA) responding to election incidents a the locallevel. A team of two MFAs is assigned to approximately six polling places. They typically patrol these sixpolling places until they receive a report of an election incident. Immediately, they must decide if a responseinvolving physically going somewhere is necessary or if calling the appropriate contact on the phone wouldbe better. If a physical response is necessary, the MFA team goes to either the polling place in questionor the election headquarters where election officials are stationed. In each case, the MFA team has to begranted access to the facility; if they are unable to get access they will call the election official or their MFAManager. When they are granted access to the polling place or elections headquarters they will do one ormore of gathering information, requesting action from the poll worker or election official and, at the pollingplace, interact with voters that might be affected by the problem. In both cases, the MFA team documentswhat actions they’ve taken by calling their MFA Manager or, in some cases, the EPC hotline. In all cases, theMFA team may follow-up on voting incidents with the voter, election official or EPC representative workingon the problem. (Note: There are other models for field attorneys for different EPC coalition members likethe NAACP. Please get in touch with me if you know of other models that were implemented on November 2,2004.)

Figure 7: This figure is a screenshot from the Brahms AgentViewer software that displays the results ofsimulations. Here are some of the common components of Brahms AgentViewer output: 1) Light bulb iconsmark where thoughtframes are triggered; 2) Workframes are noted by blue boxes with the workframe’s namedisplayed if space permits; 3) Activities are noted by salmon-colored boxes under each workframe; 4) Directcommunication between agents and or objects is noted by a blue arrow with its anchor at the communicatorand its head at the agent/object receiving the communication; 5) Communications with agents or objectsnot displayed in the AgentViewer are marked with the communication going to or coming from a hand orbox icon respectively, and; 6) Broadcast communications – from one agent/object to all agent/objects in thesame Area – are noted with a broadcast icon at the head of their arrows (in this case, this is the SchoolClockbroadcasting the time every 120 seconds).

Figure 8: The Brahms AgentViewer output for the interaction between the voter and poll worker as thepollworker checks the registration status of the voter and issues a smartcard to the voter.

Figure 9: The Brahms AgentViewer output for the first part of the interaction between the voter and theSequoia AVC Edge voting machine. The Voter inserts the smartcard, chooses a language and then reads thevoting instructions displayed. The voter then casts votes for the first two contests on this ballot (you can seea bit of the third contest at the far edge).

Figure 10: The Brahms AgentViewer output for the second part of the interaction between the voter andthe voting machine. The voter votes in the last two contests and then examines the review screen. The voterfinds no problem and then casts his or her ballot. Finally, the voting machine ejects the smartcard.

Figure 11: The Brahms AgentViewer output for the interaction between the voter returning the smartcardto the poll worker. The poll worker takes the smart card and places it in a box.