Automated Election System: AES Watch STAR Card

Updated as of Feb 28, 2010

AES Watch ConvenersUniversity of the Philippines Alumni Association (UPAA); Center for People

Empowerment in Governance (CenPEG); National Secretariat for Social Action (NASSA-CBCP), Bishop Broderick Pabillo (CBCP) and Bishop Deogracias Yniguez (CBCP and Ecumenical Bishops Forum / EBF); National Council of

Churches in the Philippines (NCCP); Dr. Reena Estuar, chair of AdMU Dept. of Information Communications System; Dr. Jaime Caro, U.P. computer science &

President, Computing Society of the Philippines (CSP); Dr. Rachel Roxas, DLSU College of Computer Studies; Computer Professionals Union (CPU);

Association of Major Religious Superiors in the Philippines (AMRSP); Association of Schools of Public Administration of the Philippines (ASPAP); Philippine Computer Emergency Response Team (PhCERT); Transparency

International (TI-Philippines); National Union of Students of the Philippines (NUSP); Engr. Rodolfo “Jun” Lozada; Health Alliance for Democracy (HEAD);

Senior Catholic Citizens’ Organization; Transparentelections.org; CCM; Coordinating Council for People’s Development (CPDG); Solidarity Philippines; Pagbabago (Movement for Social Change); Council for Health & Development

(CHD); Movement for Good Governance (MGG); and others.

AES Watch STAR CardAES Watch is a network of concerned groups and

individuals monitoring the poll automation preparations.AES Watch uses the System Trustworthiness,

Accountability, and Readiness Scorecard (STAR Card) to assess the readiness and trustworthiness of the AES.

The STAR Card assesses the AES on 20 items of concern based on available facts.

Each item is rated: PASS WARNING DANGER FAIL

Categories of Concern The 20 items of concern in the STAR Card are grouped into 4

categories:A. Setting up the System

Will the AES be ready for full implementation by May 10?

B. Internal Security and Trustworthiness Will the AES have the necessary safeguards to prevent fraud?

C. Personnel Training and Voters’ Education Will the teachers and the voters know exactly what to do on May 10?

D. Contingencies Will all concerned know what to do when things go wrong?

We want to ensure the holding of peaceful, clean, and credible elections on May 10, 2010.

A. Setting Up the System1. Timely delivery of hardware and software components to

Comelec2. Quality of machines3. Technology certification4. Availability of transmission facilities5. Deployment of machines6. Physical security of machines7. Precinct-specific ballots8. Resource inventory at the voting centers9. Adequate General Instructions

Will the AES be ready for full implementation by May 10?

A. Setting up

1. Timely Delivery of AES Components Will all components of the AES (including ballot boxes)

be delivered on time so as not to compromise quality assurance?

All 82,200 PCOS are in the country (Feb 27) – 68,620 in Cabuyao warehouse, 13,598 with Customs

77,000 ballot boxes supply contracted only on Feb 1 with Smartmatic (made in Taiwan, full delivery in Apr)

Rating: WARNING

A. Setting up

2. Quality of MachinesWill quality assurance tests be completed in time to meet

other requirements dependent on such tests? About 60,000 reported on Feb 25 to have passed

operational tests at Cabuyao warehouse.Machines are still to be configured.Actual case as reported in the papers: Out of 600 ballots

fed in a test, only 30 were read properly.Field test on Jan 20: only 40% of ballot read in AguhoMock election on Feb 6: problems with PCOS

Ricardo Papa: ballot feed problem due to tape on scanner; Maharlika: ballot feed jammed twice New Era: 5 ballots rejected with no clear reason for some.

Comelec admitted lack of manpower for testing. Rating: DANGER

A. Setting up

3. Technology CertificationWill the technology certification be completed by Feb 10, the

deadline set by AES Law?AES Law provides: the Technical Evaluation Committee (TEC)

shall certify, through an established international certification entity (SysTest Labs of Colorado contracted in Oct 2009), that the AES, including its hardward and software components, is operating properly, securely, and accurately.

TEC Resolution dated Feb 10 stated that 3 requirements are yet to be successfully completed: Audit on the accuracy, functionality & security controls on the AES

software based on 5 tests Certification: source code reviewed is the same as that used by equipment

(can be done only after configuration of PCOS & CCS in Apr) Operationalization of a continuity plan

Rating: FAIL (downgraded from Danger)

A. Setting up

4. Availability of Transmission FacilitiesAre reliable electronic transmission facilities available at

all clustered precincts? Has contracts been signed with the telcos? Has provisions been made for transmission in areas not covered by the telcos?

Site survey by Smartmatic-TIM 93% done: 64% of country has connectivity and 32% none.

Transmission glitches experienced during field tests requiring use of BGAN (satellite) even in Pateros, which is within Metro Manila.

5,000 BGAN devices acquired. Is the number sufficient?Rating: DANGER

A. Setting up

5. Deployment of MachinesHas Comelec prepared a plan that ensures problem-free

deployment of machines? Has a contract been signed with an established logistics company?

PCOS machines are precinct specific. Errors in delivery can upset the voting process on election day.

Ballot boxes are quite bulky.Comelec reported that contracts have been signed with certain

provincial logistics companies. Comelec must disclose its logistics plan, contracts with logistics

companies and the capability and track record of the contractors.Rating: WARNING

A. Setting up

5. Deployment of Machines

A. Setting up

6. Physical Security of MachinesAre there adequate plans for ensuring all machines, including

the CF cards, are protected from theft or tampering in their assigned locations?

The General Instructions specifies that political parties are free to send their poll watchers to secure the machines themselves but there is no clear agreement and explicit plan yet as to how this will be done.

The CF card will contains the record of votes. It is sealed in the PCOS but may be removed in case of PCOS malfunction to be transferred or moved.

How will the CF cards be protected from snatching, switching, and other risks?

Rating: WARNING

A. Setting up

7. Precinct-Specific BallotsWill the ballots with pre-printed names of candidates for

specific localities be ready in time? What are the logistics arrangements for deployment of ballots?

Printing of 50.7 million ballots started only on Feb 7 (delayed from original Jan 7) and to finish on Apr 25 (later than original Apr 18). Should print about 660K per day.

Actual printing: 6 mil in 11 days (Feb 7-25) or ave 330K.Ballot packs are precinct specific:

1,631 unique ballot faces 76,340 clustered precinct, each to be given a fixed number of

ballots (based on number of registered voters) with ID.Bidding for services to deploy ballots announced on Feb

6 and scheduled for Mar 1. Start of delivery on Mar 10.Rating: WARNING

A. Setting up

8. Resource Inventory at Voting CentersAre all the necessary resources available at each of the voting

centers, including: Power supply and periods of operation?

IT-capable personnel for the BEIs and BOCs?

Transmission facilities? The site survey being conducted covers only transmission

facilities; no mention of other critical facilities such as power and DOST-certified IT-capable persons to support BEI/BOC.

Transmission failures during field testing indicate inadequate surveys.

Rating: DANGER

A. Setting up

9. Adequate General InstructionsAre adequate General Instructions (GI) for the conduct of

automated elections ready?GI for BEIs on the Voting, Counting, and Transmission of

election results issued on Dec 29, 2009.GI for the Consolidation, Transmission, and Canvassing of

Votes not finalized. When will this be issued?No response from Comelec about refinements of the GI as

suggested by stakeholders, e.g., security of private keys, disposition of ballots rejected by PCOS machines, etc.

Mock elections: 50 voters took one hour (projected to 550 voters over 11 hours of election day). Clustered precincts will have up to 1,000 voters.

Rating: DANGER

B. Internal Security & Trustworthiness10. Source code11. Verifiability of voting and results at

all levels of canvassing12. Secured transmission of election

results13. Initialization of machines14. Manual audit of vote counts

Will the AES have the necessary safeguards to prevent fraud?

B. Internal Secutrity

10. Source Code Was the source code made available to interested political

parties and groups for review as mandated by the AES Law?No, not promptly as provided for in RA 9369, Sec 12 when an

AES technology was selected in July 2009.Source code “review” is being scheduled by Comelec after

certification by SysTest, with limited time left before election day and under restrictive rules – mere “walkthrough”!

Reliability and trustworthiness of the system put in doubt.Rating: FAILComelec claims there is a mechanism by which interested

parties can validate that the source code certified by SysTest is the same as that used by each of the PCOS and CCS machines.

Will Comelec enable such mechanism so validation can be done just before the start and right after voting on election day?

B. Internal Security

11. Verifiability of Voting and ResultsWill a voter be able to verify if machine has registered his

chosen candidates as required by the AES Law?No, function is available in machine but disabled.Will watchers be able to easily verify the election results

received and canvassed at the BOCs against the election results printed out at the precincts?

No arrangements have been made to facilitate verification, such as by projecting on big screens the statements of votes by precincts (as suggested by Cong. Rufus Rodriguez) .

Rating: DANGER

B. Internal Security

12. Secured Transmission of ResultsWill the BEI teachers themselves individually generate

their PIN codes, hold it for safekeeping, and use it for digitally signing the transmission of election results?

No, Comelc/Smartmatic will generate and provide the PIN codes to be used by the teachers.

GI (draft or even Comelec Resolution 8739) is silent on procedures and/or mechanics for making transmissions secure.

Rating: WARNING

B. Internal Security

13. Initialization of MachinesWill procedures be implemented to assure stakeholders that

there are no votes or ballot images pre-stored in the machine memory (CF card) prior to voting on election day?

COMELEC requires all AES machines to be cleared or “zeroed out” to show that there are no entries/votes in the PCOS memory.

Election results showing zero votes will be printed out just before start of voting on election day.

But what are the safeguards to ensure that there are no ballot images pre-stored in the machine memory prior to voting?

Zeroing demo done with PCOS, but not CCS.Rating: WARNING

B. Internal Security

14. Manual Audit of Vote CountsWill the random manual audit be done by independent

auditors, for a sufficient number of precincts, and before proclamation?

Technical Working Group on Random Manual Audit (TWG-RMA) created in Nov 2009 with Amb De Villa of PPCRV as Chair and OIC of Comelec Internal Audit Office and COA resident representative as members. No decision yet on who will do the audit and how. Independent auditors key.

Sampling methodology has yet to be establishedComelec has confirmed that the manual audit will be done

after proclamation.The random manual audit is the last possible check on the

trustworthiness of the AES. Rating: WARNING

C. Personnel Training & Voters’ Education

15. Training of technical personnel and members of the BEI and BOC

16. Stakeholder education and training

17. Precinct assignment of voters

Will the teachers and the voters know exactly what to do on election day?

C. Training & Education

15. Training of Technical PersonnelWill Comelec be able to train the teachers and other personnel in

time on both normal and contingency operations of the AES? AES Law requires BEIs/BOCs to each have an IT-capable person. Training for 230,000 teachers to start Mar 1 (2-1/2 months later

than original Dec 15). 76,340 teachers must be DOST certified.Ballot with old format to be used for training: confusing.Reports show growing fear among teachers that this delay will

leave them inadequately trained in operating the machine.Will Smartmatic-TIM be able to recruit 45,000 competent

technicians needed on election day? From where and how?No report on status / progress of training.Rating: DANGER

C. Training & Education

16. Stakeholder Education & TrainingWill there be an extensive and sufficient voter education on the

AES before the elections?RA 9369, Sec 26 requires Comelec, not later than 6 months before

elections, to undertake widespread stakeholder education. Mock election: ballots rejected by PCOS due to ambiguous marks

(indication of need for effective voter education).Pulse Asia's Jan 2010 Survey: 71% of Filipinos have little or

almost no knowledge at all of the AES (79% among class E).No disclosure of detailed plans for training, progress (number of

voters trained, particularly for voters in rural and other remote areas), assessment of effectiveness of training programs

Rating: DANGER

C. Training & Education

17. Precinct Assignment of VotersWill voters know ahead of time which clustered precinct to

go to so as to avoid confusion and delays on election day ? The AES involves clustering of precincts. Majority of

voters will likely have to go to new polling places.No update reported on the preparation of voters list for each

of the clustered precincts and on the plan for informing voters of their new precinct assignments.

Rating: DANGER

D. Contingencies18. Continuity plan19. Electoral protest mechanism20. Alternative election system in

some parts of the country

Will Comelec and all concerned know what to do when things go wrong?

D. Contingencies

18. Continuity PlanIs Comelec ready with an adequate continuity plan that covers all

foreseeable eventualities? RA9369 requires the development, provisioning, and

operationalization of a continuity plan to cover risks to the AES at all points in the process such that a failure of elections, whether at voting, counting or consolidation, may be avoided.

Per TEC Resolution of Feb 10, this is not yet done.The draft continuity plan shown by Comelec appears inadequate:

replace machine or go manual in case of PCOS/CCS malfunction.Other eventualities not covered: e.g., natural disasters, civil

disturbance & other disruptions; voting after 6 pm; manual counting, canvassing & consolidation; jamming of transmission; standby resources; training of BEI/BOC personnel, drill exercises.

Continuity plan is key to avoiding failure of election. Rating: DANGER

D. Contingencies

19. Electoral Protest MechanismHas Comelec identified the grounds for electoral protests

under the AES and defined the appropriate process for addressing and resolving such protests?

No specific legal provisions available for AES-based protests (e.g., differences between machine and visual appreciation of ballot shading, discrepancies in printed and transmitted results, appeal for recount, etc.)

No progress report from Comelec.Rating: WARNING

D. Contingencies

20. Alternative Election SystemAre preparations being undertaken to implement a manual

election system if full automation is no longer realistic? How will this be done: manual counting, transmission, and canvassing? With traditional ballots or pre-printed ballots? How will results of manual and automated systmems be consolidated?

No guidelines or procedures have been announced yet for the conduct of manual elections where needed.

No clear criteria/ trigger points have been defined nor deadline set yet for determining whether to go manual and in which localities.

Rating: DANGER

Summary of Ratings (Feb 28, 2010)1. Timely Delivery of Machines

– Warning2. Quality of Machines

– Danger3. Technology Certification –

Fail4. Availability of Transmission Facilities – Danger5. Deployment of Machines –

Warning6. Physical

– Danger7. Security of Machines

– Warning8. Precinct Specific Ballots

– Warning9. Resource Inventory at Voting Centers – Danger10. Adequate General Instructions –

Danger11. Source Code

– Fail12. Verifiability of Voting and Results –

Danger13. Secured Transmission of Results –

Warning14. Initialization of Machines –

Warning15. Manual Audit of Vote Counts

– Warning16. Training of Technical Personnel –

Danger17. Precinct Assignment of Voters –

Danger18. Continuity Plan

– Danger19. Electoral Protest Mechanism –

Warning20. Alternative Election System –

Danger

STAR Card Assessment of the AES

Overall the AES is in the DANGER zone.Readiness of the AES is at risk due to the pattern of

delays and changes in the calendar.Trustworthiness of the AES is in question as needed

system safeguards are not yet in place.

As of Jan 26 As of Feb 11 As of Feb 28

PASS None None None

WARNING 9 8 8

DANGER 10 11 10

FAIL 1 1 2