CRT-DWF Draft 20060317
Total Page:16
File Type:pdf, Size:1020Kb
CRT-DWF Draft 20060317 This document contains draft text for Core Requirements and Testing (CRT) for the following parts of the next iteration of the VVSG: Overview: V olume I • Description and rationale of significant changes: V olume I Chapter 1 • Options not standardized: V olume I Chapter 2 • List of sources reviewed: V olume I Chapter 3 Terminology standard: V olume I I Product standard: V olume I II • Introduction: V olume I II Chapter 1 • Conformance clause: V olume I II Chapter 2 • General requirements: V olume I II Chapter 3 • General design requirements: V olume I II Section 3.1 • Coding standards: V olume I II Section 3.2.1 , V olume I II Section 3.5.1 • A rchivalness : V olume I II Section 3.6 • Interoperability: V olume I II Section 3.7 • Requirements by voting activity: V olume I II Chapter 4 • Election programming: V olume I II Section 4.1 • Ballot preparation, formatting, and production: V olume I II Section 4.2 • Equipment preparation: V olume I II Section 4.3 • L&A testing: V olume I II Section 4.4.1 • Opening polls: V olume I II Section 4.5 • Casting: V olume I II Section 4.6 • Ballot activation: V olume I II Section 4.6.1 • General voting functionality: V olume I II Section 4.6.2 • Voting variations: V olume I II Section 4.6.3 • Recording votes: V olume I II Section 4.6.4 • Redundant records: V olume I II Section 4.6.5 • Respecting limits: V olume I II Section 4.6.6 • Closing polls: V olume I II Section 4.7 • Counting: V olume I II Section 4.8 • Voting variations: V olume I II Section 4.8.1 • Ballot separation and rejection: V olume I II Section 4.8.2 • Paper jams: V olume I II Section 4.8.3 • Accuracy: V olume I II Section 4.8.4 • Consolidation: V olume I II Section 4.8.5 • Reporting: V olume I II Section 4.9 • General reporting functionality: V olume I II Section 4.9.1 • Audit, status, and readiness reports: V olume I II Section 4.9.2 • Vote data reports: V olume I II Section 4.9.3 • General functionality: V olume I II Section 4.9.3 .1 • Ballot counts: V olume I II Section 4.9.3 .2 • Vote totals: V olume I II Section 4.9.3 .3 • Transmitting results: V olume I II Section 4.10 • Auditing and independent verification: V olume I II Section 4.11 • Reference models: V olume I II Chapter 5 • Process model: V olume I II Section 5.1 • Vote-capture device state model: V olume I II Section 5.2 • Logic model: V olume I II Section 5.3 Standards on data to be provided: V olume I V • Technical Data Package: V olume I V Chapter 1 • Scope: V olume I V Section 1.1 • Implementation statement: V olume I V Section 1.2 • System hardware specification: V olume I V Section 1.3 • Software design and specification: V olume I V Section 1.4 • System test and verification specification: V olume I V Section 1.5 • Configuration management plan: V olume I V Section 1.6 • Quality assurance program: V olume I V Section 1.7 • System change notes: V olume I V Section 1.8 • Configuration for testing: V olume I V Section 1.9 • Voting Equipment User Documentation: V olume I V Chapter 2 • System overview: V olume I V Section 2.1 • System functionality description: V olume I V Section 2.2 • System security specification: V olume I V Section 2.3 • System operations manual: V olume I V Section 2.4 • System maintenance manual: V olume I V Section 2.5 • Personnel deployment and training requirements: V olume I V Section 2.6 • Test Plan: V olume I V Chapter 3 • Test Report: V olume I V Chapter 4 • Public Information Package: V olume I V Chapter 5 Testing standard: V olume V • Overview: V olume V Chapter 1 • Introduction to test methods: V olume V Chapter 2 • Documentation and design reviews (inspections): V olume V Chapter 3 • Test protocols: V olume V Chapter 4 Supplemental guidance (not part of the VVSG): V olume V I • Procedures required for correct system functioning: V olume V I Chapter 1 Text for security requirements (STS), human factors and privacy requirements (HFP), hardware and software performance requirements and hardware workmanship requirements (AG) are distributed in separate documents. Where such requirements appear in this document, they are only notes on things that need to be coordinated. The draft text for different sections is at differing levels of maturity. Notes on work that is yet to be done and problems that need to be fixed are shown highlighted like this. These notes are for our own use and will not appear in the final draft. Similarly, the Impact field of requirements is for our own use and will not appear in the final draft. Notes applying to the entirety of the document follow. Scoping of reviews (source code, design, what have you) and related requirements (coding conventions, QA & CM) with respect to COTS and non-COTS, previously reviewed vs. not. The important thing is not whether something is hardware, firmware, software, or even whether it is "commercial" or "off the shelf," but whether the objectives of the review that the test lab would otherwise be doing are sufficiently satisfied by scrutiny that the item has previously received. If something is WUUP, then it has received some testing through widespread use. If something is non- WUUP but has been certified under some other program and/or previously shown to be correct (e.g., an ASIC with a formally verified design), this too may suffice. Whether either of these cases do in fact suffice depends on the specifics. This needs to be discussed with STS before we come to a consensus on how to scope the various reviews. The current standard seems inconsistent in what is excluded from testing in different places, and borderline cases are not handled clearly. [2] refs: I.4.1.1, I.4.1.2, I.4.2.3, II.4.2.1, II.4.6.1, II.5.2. [5] refs: I.5.1.1, I.5.1.2, I.5.2.3, I.7.1, II.4.2.1, II.4.6.1, II.5.2. Issue #2685, need to verify that what vendor asserts is WUUP is actually WUUP. Affected sections in this draft: V olume I II Section 3.2.1 .1 , V olume I II Section 3.5.1 .1 , V olume I V Section 1.4 , R equirement I V.1.4-23 , V olume V Chapter 3 . Audit log and logging requirements need synchronization with STS. The following sections of [2] have not yet been incorporated: I.2.2.5, I.4.4.1, I.4.4.2, I.4.4.3. See also V olume I II Section 3.2.2 and V olume I II Section 4.11 . It is unclear to what extent ballot accounting and retention of CVRs should be included under auditing or divided between STS and CRT. All things "remote" (configuration using a network, remote voting, transmitting results—V olume I II S ection 4.10 ) need synchronization with STS as most of the issues there are security-related. [2] I.1.5.4 (Public Network DRE) might need incorporating somewhere; I.5 and I.6.5 are already under revision. All things "integrity." Requirements for maintaining data integrity and detecting and reporting loss of integrity are scattered through the VVSG. Not much distinction was/is made between defending from attacks versus resistance to internally or externally induced data errors. The choices of which requirements to carry over here versus what I assumed STS would take over need to be reviewed and adjusted as needed to cover all bases. All requirements get descriptive text IDs. Delete all punchcard? (Never got definitive direction.) General: after STS, HFP, AG text have been integrated, go through [5], ensure that all useful requirements and informative text are retained somewhere. Global change: terminology to be resolved: • Change qualification to certification, testing, or conformity assessment, as appropriate • ITA versus VSTL versus test lab versus testing authority • Ballot preparation, formatting, and production: poorly distinguished, may suffice to lump them all together without discriminating • Ballot configuration versus ballot format versus ballot style: In 2002 VSS, ballot configuration is the abstract, the other two synonymously refer to the concrete. In VVSG'05, ballot configuration and ballot style synonymously refer to the abstract, and ballot format refers to the concrete. Per Murphy's Law, the term most used in practice is the ambiguous one (ballot style). Harmonize stuff carried over from [2] with any changes made in [5]. Volume I Guidelines overview Chapter 1 Description and rationale of significant changes vs. [2] or [5] 1.1 Document structure The VVSG have been restructured to reduce redundancy in the requirements and to reduce fragmentation of requirements. F igure 1 shows a conceptual model of requirements, including their relationships to activities in the voting process. To structure the VVSG entirely according to the voting process would make it more usable by some readers. Unfortunately, as shown in F igure 1 , a given requirement may relate to many activities. To structure the VVSG entirely according to the voting process would create more redundancy rather than less. The new structure compromises by handling the two most common cases in two separate sections.