Administrative Procedures for Creating a IEEE Standard

Administrative Procedures for Creating an IEEE Standard A Guide to SESC Working Groups

Prepared by the IEEE Software Engineering Standards Committee Dedicated to the memory of Fletcher J. Buckley Release 2.0, July 14, 2003 Table of Contents

1. OVERVIEW...... 2

1.1. SCOPE...... 2 1.2. PURPOSE...... 2 1.3. OUTLINE OF GUIDE...... 2 2. REFERENCES...... 3

2.1. IEEE OFFICIAL DOCUMENTS...... 3 2.2. SESC DOCUMENTS GOVERNING STANDARDS PRODUCTION...... 4 2.2.1. SESC Policies...... 4 2.2.2. SESC Procedures...... 4 2.3. EXISTING STANDARDS...... 4 2.4. SESC MANAGEMENT BOARD...... 5 3. DEFINITIONS...... 5

3.1. ACRONYMS...... 5 4. FUNDAMENTAL PRINCIPLES OF STANDARDS DEVELOPMENT...... 5

5. SESC POLICIES AND PROCEDURES...... 6

6. STANDARDS DEVELOPMENT PROCESS...... 7

6.1. CREATION OF A WORKING GROUP...... 7 6.2. PREPARATION OF THE PAR...... 9 6.3. PROJECT PLAN...... 9 6.4. CREATION OF A DESIGN SPECIFICATION...... 10 6.5. WORKING GROUP / MANAGEMENT BOARD LIAISON...... 10 6.6. SEMI-ANNUAL REPORTS...... 11 6.7. READINESS REVIEW...... 11 6.8. BALLOTING...... 12 6.9. BALLOT COMMENT RESOLUTION...... 13 6.10. PROCESS COMPLETION...... 14 ANNEX A. LESSONS LEARNED...... 15

ANNEX B. A PRACTICAL GUIDE FOR DEVELOPING IEEE STANDARDS...... 18

B.1. DOCUMENTATION...... 18 B.2. BALLOTING GROUPS...... 18 B.2.1 Negative Votes on Informative Clauses...... 19 B.3. OPENNESS IN THE STANDARDS DEVELOPMENT PROCESS...... 19 B.3.1 Proactive Contributors and People Who Prefer the Sidelines...... 20 B.3.2 Electronic Communication versus Face-to-Face Meetings...... 20

1 B.4. GUIDANCE FOR EDITING STANDARDS DOCUMENTS FOR BALLOTING...... 21 B.4.1 Introduction...... 21 B.4.2 Process Factors...... 21 B.4.3 Header (on each page)...... 21 B.4.4 Footer (on each page)...... 22 B.4.5 Statement to Appear Just before the Abstract...... 22 B.4.6 Technical Factors...... 22 B.4.7 Non-Textual Information in the Ballot Draft...... 23 B.4.8 Converting from Word to PDF...... 23 ANNEX C. GUIDANCE FOR PREPARATION OF IEEE STANDARDS...... 24 C.2.1 Invitation Stage...... 24 C.2.2 Sponsor Ballot Stage...... 25 C.2.3 Ballot Resolution...... 25 C.2.4 Recirculation Ballot...... 26 C.2.5 Other Useful Links...... 26 ANNEX D. LIST OF CHANGES FROM VERSION 1.0...... 27

1. Overview

1.1. Scope This Guide applies to all Working Groups operating under the sponsorship of the Software Engineering Standards Committee (SESC). It does not apply to SESC Planning Groups or Study Groups, since the latter do not create IEEE standards.

1.2. Purpose This Guide is intended to assist Working Groups in the conduct of business, in order to reduce the administrative problems that accompany the creation of a standard. The rules imposed by this Guide are derived from IEEE- SA Standards Board policies and procedures (P&P), and are meant to be compatible with them. SESC has adopted a set of policies and procedures that are consistent with the Standards Board policies and procedures. Both the IEEE P&P and the SESC P&P apply to all new and revision standards projects that SESC sponsors. All SESC Working Groups are expected to follow these policies and procedures. In the event of a conflict between the SESC P&P and the IEEE P&P, the IEEE P&P shall prevail.

1.3. Outline of Guide This Guide consists of five main sections and three annexes. These are as follows:  Section 2 lists various references that should be used by the Working Group while preparing the standard. There are a number of official IEEE documents that govern the preparation and balloting of standards; these are available on an IEEE Web site. SESC has prepared a number of additional policy and

2 procedure documents that govern the software engineering standards development process; these are available on the SESC Web site. Some additional documents are also listed for guidance.  Section 3 lists the acronyms used in the Guide.  Section 4 describes several fundamental principles that govern the production of IEEE standards. The two most important principles are openness and consensus; both are discussed in this section.  Section 5 describes the purpose of the SESC policies and procedures, and introduces the SESC Management Board, which oversees the actual production of software engineering standards.  Section 6 describes in some detail the steps that must be used to create a new or revised standard. These steps are intended to ensure that both IEEE and SESC policies and procedures are followed, and to ensure the highest possible quality in software engineering standards.  Annex A lists some “lessons learned” over the years that can help reduce the number of negative votes on ballots. If these are followed, there should be few (if any) negative ballots for non-technical reasons. The purpose is to encourage balloters to concentrate on the technical substance of the new or revised standard, not on stylistic problems.  Annex B provides a “Practical Guide for Developing IEEE Standards,” prepared by Norman F. Schneidewind based on his 20 years’ experience developing IEEE standards. Dr. Schneidewind’s suggestions, if properly followed, will greatly simplify the standards preparation and balloting process  Annex D lists the changes that have made to this Guide since the previous version, Release 1.0, published April 7, 2000.

2. References

2.1. IEEE Official Documents The following documents are included in this Guide by reference. Working Groups must adhere to these documents. Dates are for the latest editions as of the date this Guide was written; if later editions exist, they should be used. These documents can be found on the IEEE Standards Association (IEEE-SA) Web Page, at http://standards.ieee.org/resources/index.html#guides. 1. IEEE Standards Style Manual, April 2002. From a practical viewpoint, this document has the most effect on the actual drafting of a standard. A Working Group must follow its prescriptions. 2. IEEE-SA Standards Association Operations Manual, July 2002. 3. IEEE-SA Standards Board Bylaws, January 2002. 4. IEEE Standards Companion, January 2003.

3 2.2. SESC Documents Governing Standards Production In addition to the official IEEE-SA documents, the following SESC documents are included in this Guide by reference. Working Groups are strongly urged to become familiar with the documents, since they must be followed. These documents may be found on the SESC Web Page, at http://standards.computer.org/sesc/.

2.2.1. SESC Policies SESC has adopted a set of policies that are used to set directions for standards development. See the SESC web page for the current set of policies.

2.2.2. SESC Procedures The following procedures are used by the SESC to manage the effort of creating a new or revised standard. SESC-02 Milestone Definition SESC-03 Working Group Progress Reports SESC-04 Project Plan Preparation SESC-05 PAR Criteria SESC-06 Project Plan Review SESC-07 Design Specification Preparation SESC-08 Design Specification Review SESC-09 Project Readiness Review SESC-15 Project Decision Log SESC-16 SESC Ballot Formation SESC-17 SESC Ballot Management SESC-19 Direct Balloting

4 2.3. Existing Standards It may happen that your standard is closely related to other existing standards. Indeed, this is likely to be the case – only a few SESC standards cover isolated topics. The Working Group should acquire any related standards; the Management Board Representative will help you identify them. Most existing SESC standards were printed as a single collection, Collection 1999, in early 1999, and this would be an effective way to learn about existing standards. (NOTE: THE 2003 Collection on CD-ROM should be available by mid-August.) In addition, SESC has sponsored a book by James Moore, Software Engineering Standards: A User's Road Map, IEEE Computer Society Press (1998). This describes the fundamental philosophy and organization that underlies the technical standards, and the working group is strongly advised to obtain a copy.

2.4. SESC Management Board Your most valuable resource may be the person on the SESC Management Board that is responsible for your project. Every standards project undertaken by SESC has a representative on the Management Board that is responsible for overseeing the successful completion of the project. You should feel free to ask this person for help and advice. Creating or revising a standard is a fairly complex process, subject to many rules, and there are many places where the project can stray. Management Board members are elected for two-year terms. Each term begins in July. Half the Board is elected in odd-numbered years, and half in even-numbered years. Current members can be found on the SESC Web Site.

3. Definitions

3.1. Acronyms  EIA – Electronics Industry Alliance  Ex Com – SESC Executive Committee  IEEE – Institute of Electrical and Electronics Engineers  MB – SESC Management Board  P&P – Policies and Procedures  PAR – Project Authorization Request  SA – IEEE Standards Association  SB – IEEE-SA Standards Board  SE – Software engineering

5  SES – Software engineering standard  SESC – Software Engineering Standards Committee  WG – Working Group

4. Fundamental Principles of Standards Development The development of a new or revised standard has a defined life cycle, analogous to the one used to develop software. A number of organizations get involved in the development process along the way: the Working Group (WG), the SESC Management Board (MB), the SESC Executive Committee (Ex Com), the IEEE Standards Association (IEEE-SA), the IEEE-SA Standards Board (SB), and (perhaps the most important group) the Ballot Group. Each of these organizations has a defined role in the development of the standard. Communication among the organizations takes place primarily through the transmission of information in the form of documents, which become official records of the development. Much of this is to ensure that IEEE P&P are followed, and to be able to demonstrate after the completion of a project that the P&P have been followed. One fundamental principle behind the administrative procedures is that of openness – it is absolutely required that all interested persons have an opportunity to participate in the standards development process. The SB Bylaws require (Clause 5.2.1.1) that “all meetings involving standards shall be open to all interested parties.” A second fundamental principle is that of consensus. This term is described in the SB Bylaws (Clause 2.1) as follows: “Consensus is established when, in the judgment of the IEEE-SA Standards Board, substantial agreement has been reached by directly and materially affected interest categories. Substantial agreement means much more than a simple majority, but not necessarily unanimity. Consensus requires that all views and objections be considered, and that a concerted effort be made toward their resolution.” Ultimately, consensus is measured by broad agreement to the standard by the Ballot Group. One intent of the IEEE administrative procedures that the Working Group is expected to follow is to enable a demonstration that these two principles have been obeyed. Note that this is a legal requirement, and is necessary (among other things) to protect the IEEE, SESC and the Working Group members from restraint of trade legal action. The SB delegates much of the responsibility for ensuring that its policies and procedures are followed to sponsor organizations. A sponsor is responsible for oversight of any of its assigned standards, including overseeing coordination and balloting (SB Bylaws Clause 5.2.2.2). SESC is a recognized sponsor of software engineering standards.

6 5. SESC Policies and Procedures The SESC has established a series of policies and procedures designed to (1) ensure that software engineering standards development processes follow the SB rules, (2) ensure that the resulting standards are of high technical quality, and (3) ensure that the entire collection of software engineering standards is made up of mutually consistent and supporting standards. Most of the actual oversight of software engineering standards has been delegated by SESC to a seven-member elected Management Board. Each development project is assigned to one MB member, who has primary responsibility for assisting the Working Group in performing its technical work in a manner that meets the rules of the SB and SESC. The assigned Management Board representative will be available to guide the working group through the entire standards development process. Since the MB is not automatically open to “all interested parties,” it does not become directly involved in the technical work; its role is one of standards management, administrative oversight and overall direction. (The Ballot Group evaluates the technical work.) In particular, the Management Board can implement the strategies of SESC by controlling the statements of scope and purpose in submitted PARs. (See Section 6.2 below.) The Management Board can then enforce the achievement of those statements in the actual standards. It can formulate mechanisms for integration and consistency and enforce those mechanisms.

6. Standards Development Process Certain administrative steps occur during the development of a new or revised standard. These are discussed in this clause, and are illustrated in Figure 1. IEEE-SA provides a graphic summary of the process, called “Standards Process at a Glance,” with links to supporting forms and a Microsoft Word template for a standard. See: http://standards.ieee.org/resources/glance.html. A complete, step-by-step explanation of the standards development process is available at: http://standards.ieee.org/resources/development/index.html. IEEE created a Standards Association in 1997, and has imposed two membership requirements that affect Working Groups. First, the Working Group “Official Reporter” must be a member of the Standards Association; in most cases this will be the Working Group Chair. Second, everyone in the ballot group for a standard must belong to the Standards Association. These two requirements are not negotiable.

6.1. Creation of a Working Group A Working Group is created by SESC when, in the judgment of SESC, a new software engineering standard would be useful to the technical community, or when an existing software engineering standard needs to be revised. The initial impetus for a Working Group may come from some individual who

7 champions the need for the new standard, or from SESC itself. In either case, the procedures are the same. A Working Group consists of some number of members, with certain designated officers. The size may vary – from a few people for a small technical revision to hundreds for an important new standard project. Each Working Group must have a Chair, appointed by the SESC Chair, who is responsible for ensuring that the standard project succeeds and follows the P&P of the IEEE SB and SESC. The Chair must be a member of the IEEE Standards Association.1 Additional officers are a secretary, responsible for correspondence, meeting minutes and similar matters, and an editor, responsible for the actual writing of the draft standard. It is permitted for one individual to serve as more than one officer – indeed, one person could serve all three roles. (This is reasonable, for example, for a small Working Group.) Note that these responsibilities are those delegated by SESC, which remains responsible to the IEEE SB for the draft standard. Two early business items for a new Working Group are to prepare a project plan, and to prepare a Project Authorization Request (PAR) for submission to the IEEE SB. The Working Group is permitted by IEEE to exist for no more than six months prior to submission of the PAR; this should be ample time. It is important to provide the SESC Chair with the names, addresses, phone numbers, email addresses and IEEE numbers of all members of the Working Group and for the SESC Chair to pass this information to the IEEE Standards Association. This ensures that all participants will be covered by IEEE’s umbrella indemnification against legal action. (See IEEE-SA Bylaws, Clause I-300.3, “Indemnification.”) The Working Group is permitted by IEEE to be active for no more than four years, though extensions are possible in some circumstances. SESC prefers to complete projects much sooner than that – no more than one year in the case of most revisions, and no more than 2-3 years for new standards. There is no set size for a Working Group, nor is there a fixed method of recruiting members. In the case of small revisions to existing standards, the Management Board may choose to have a small focused Working Group. On the other hand, the creation of a new standard that is expected to have considerable influence may cause the Management Board to actively solicit members in various media. In general, the Management Board will attempt to have experts in the subject area on the Working Group. In some cases, representatives of other relevant organizations may be invited to participate. The Working Group itself is expected to seek members as needed – in particular, to supply expertise not otherwise present in the Working Group. The Management Board Representative may be able to help in this regard.

1 Technically, there must be a Reporting Officer that is an SA member. However, SESC almost always designates the Working Group chair to be the Reporting Officer.

8 All Working Group members should be briefed on IEEE and SESC processes. In particular, all members must be briefed on the IEEE patents policy. IEEE has provided two briefing slides on its patent policy; these slides are available at http://standards.ieee.org/board/pat/pat-slideset.ppt. In all cases, the Working Group is expected to follow the Openness Principle discussed in Section 4 above, and may be required to document compliance. Finally, the Working Group is strongly encouraged to bring the draft standard to ballot within a reasonable time. While consensus within the Working Group is certainly important, the official consensus for a new or revised standard is achieved within the balloting group. The Working Group should therefore not attempt to polish the draft to perfection. They should put together a credible draft and speed its presentation to the balloting group. For guidance on the conduct of working group meetings, consult the “Guide to IEEE Standards Meeting Policies,” which is available at: http://standards.ieee.org/resources/meetingguide.html .

6.2. Preparation of the PAR The Project Authorization Request (PAR) is a formal request to the IEEE Standards Board to carry out the project. It is the mechanism used by IEEE to keep track of active projects. The form itself may be found on the IEEE Standards Web Site, and changes from time to time as the SB fine-tunes it. The SESC web site contains a link to the online PAR form. Most of the PAR consists of routine information – names, addresses, and basic information about the project. Two items do require careful consideration – the project purpose and the project scope. The statement of purpose and scope form a contract between the WG and SB for the development of a standard filling a particular niche. It is the purpose of the balloting group to ensure that the standard accurately fills that niche. In rare instances, the niche must be redefined; one needs the permission of the SB to do this. Furthermore, the SB will insist at the end of the project that the purpose and scope have been followed, or IEEE will not approve the new or revised standard for publication. The PAR is prepared by the working group, with input or guidance from the SESC Management Board. Guidance is given in SESC Procedure SESC-05. The signer of the PAR must give up copyright to the IEEE. This means that the signer – usually the Working Group Chair – is personally responsible for obtaining permission from the owner of any copyrighted material that is included in the standard. If you choose to include any diagrams taken from existing copyrighted material (such as textbooks), you must obtain a copyright release. It is important to understand that, once approved by the IEEE-SA Standards Board, the Title, Scope and Purpose of the project belong to IEEE— not to the Working Group, Balloting Group, SESC or anyone else. Only IEEE-SA

9 can change those items. It is the job of these other parties to prepare a standard that matches the title, scope and purpose approved by IEEE-SA.

6.3. Project Plan In some cases the MB will require the preparation of a Project Plan. This happens primarily for standards that are considered to map to the entire SESC collection, since the contents of such standards can affect other standards. This document is essentially a contract between the Working Group and the SESC Ex Com, and describes the purpose and content of the project. The Working Group prepares the project plan, with the guidance and oversight of the MB. It is important for the plan to accurately reflect the intent of the Working Group for the project because the SESC MB uses it to guide periodic reviews of the project. Therefore, the Working Group should take the time and spend the effort to get this plan right. The project plan requires the preparation of a schedule. This should include project milestones and projected dates. SESC Procedure SESC-02 gives guidance on the definition of project milestones. Guidance on the content of a project plan is given in SESC Procedure SESC-04. The MB formally reviews the completed plan, using the process documented in Procedure SESC-06. The Working Group Chair will be expected to give the MB a short presentation on the project during this review, either in person or by telephone conference. It frequently happens that Working Group progress deviates from the initial plan. This is acceptable, provided the MB is aware of and approves of the deviation. If this should occur, the project plan should be revised accordingly.

6.4. Creation of a Design Specification The design specification is a mechanism used in some cases to review the technical scope and general design of a proposed standard. The contents of a Design Specification are discussed in SESC Procedure SESC-07, and the process of carrying out a review is given in SESC Procedure SESC-08. This review is optional, and is generally required only when one of the following situations arises for a new standard:  The standard being written is expected to have a major impact on other existing or anticipated SESC standards  The standard being written has very high visibility to standards users  The contents of the standard are expected to be very controversial. In such cases, an early review of the design and expected content of the standard can prevent much future trouble. In particular, it is preferred to anticipate and correct major problems with a new standard before it goes to ballot.

10 6.5. Working Group / Management Board Liaison Each Working Group is assigned to one member of the SESC MB, who acts as liaison between SESC Ex Com and the Working Group. This MB member has a number of responsibilities to both Ex Com and the Working Group.  Ensure that the Working Group is making progress and meeting its schedule.  Ensure that the Working Group is writing or revising a standard in such a way that the Project Plan is being followed.  Ensure that SESC P&P are followed.  Provide a “sounding board” for the Working Group when it needs help or simply someone outside to talk with.  Help the Working Group deal with IEEE and SESC P&P problems. Different Working Groups require different amounts of oversight and assistance, so no general rule exists for this responsibility. In some cases, Working Groups are quite capable of carrying out their assigned tasks with little interaction with SESC, and in these cases, MB oversight may be reduced to obtaining periodic progress reports. In other cases, where the Working Group is having trouble – either technical, personnel, or whatever – the MB may become heavily involved. (Since MB members generally have full time jobs, it naturally prefers the former to the latter!) It is good practice for a Working Group to keep a formal log of major decisions that are made during the standard development process. This provides a record of the activity of the Working Group, and is a major resource in case the fairness of the process is ever challenged. SESC Procedure # 15 describes the content of such a log. Furthermore, the IEEE SB will require a documented demonstration that the Working Group has followed SESC P&P during the final approval process, and a log is an effective means for supplying this.

6.6. Semi-Annual Reports The Working Group is expected to provide formal progress reports twice a year. These are due fifteen days prior to the SESC semi-annual meetings. The report, prepared using SESC Procedure # 3, should be kept short – one or two pages are generally sufficient. This report is a formal submission to SESC of the Working Group’s status and needs, and is used by the SESC Ex Com for project oversight. The semi-annual report is a good opportunity to provide an updated list of working group members.

6.7. Readiness Review The SESC MB carries out a final review of a proposed new or revised standard prior to sending it to ballot. This is termed the readiness review. The review process is described in SESC Procedure SESC-09.

11 The primary purpose of this review is to ensure that IEEE and SESC P&P have been followed. The intent is to fix all non-technical problems prior to going to ballot. SESC has found over time that the ballot process is much more effective if balloters can concentrate on technical issues rather than non- technical concerns. The Management Board also checks that the scope and purpose given in the PAR has been fulfilled, that the plans previously promised by the Project Plan (if required) and the Design Plan (if required) have been carried out, and that the standard is consistent with the rest of the collection to the extent required by SESC policies. Balloting experience has led SESC to create a list of “lessons learned” regarding non-technical issues that can lead to a negative ballot. These are attached to this Guide as Annex A. If these rules are followed, the balloting process will be considerably less strenuous. Note that the Working Group ceases to be responsible for the standard once SESC approves it for balloting. In most cases, the Working Group is dissolved at this time; exceptions may occur when the Working Group has additional business (such as a follow-up project). All standards published by the IEEE are eventually edited by IEEE professional staff. Unfortunately, the staff sometimes uncovers problems that can require the balloting process to be reopened. This can delay publication by several months. Recently, the IEEE Standards Department has declared its willingness to perform early editorial reviews upon request. It is prudent to obtain such a review prior to the completion of balloting and to act upon the recommendations of the review.

6.8. Balloting Once the readiness review is completed, the draft standard must be balloted following IEEE rules. This is a formal process, and can take many months, but need take only a few months if a carefully disciplined and responsive process is followed. There are several major steps:  The SESC MB performs the ballot readiness review.  The Working Group should inform the SESC Chair of any individuals that should be in the Ballot Group who are not already in the SESC Ballot Pool. Keep in mind that all balloters must be members of the IEEE Standards Association. (See SESC Procedure SESC-16.)  A Ballot Group is formed, following IEEE procedures.  The Ballot Draft is prepared based on the results of the Readiness Review and any Editorial Review, and handed from the Working Group to the SESC Chair. (See SESC Procedure SESC-17.)  The SESC Chair delivers the draft, along with balloting instructions and other formal material, to IEEE.

12  IEEE-SA places the draft standard on a web page to which ballot group members have access. It also gives them access to a second web page where they can cast their ballots, and a third where they can submit ballot comments.  Each ballot group member is expected to vote on the draft standard.  A balloter may submit comments along with the ballot. These fall into two categories: binding and advisory. A binding comment is submitted only with a negative ballot; it designates a comment that the balloter feels must be satisfied in order for the ballot to be changed to affirmative. An advisory comment is submitted only as a suggestion to the Ballot Resolution Committee (see 6.9); the balloter is not insisting that advisory comments be accepted.  IEEE collects votes. The results of the vote, and a copy of each ballot, are delivered to the Chair of the Ballot Resolution Committee.

6.9. Ballot Comment Resolution According to IEEE rules, comment resolution is the responsibility of the sponsor, not the Working Group. The Working Group’s role is completed when the document is submitted for balloting. The purpose of the Ballot Resolution Committee is to achieve consensus among the members of the Balloting Group, not to protect the document drafted by the Working Group. The Management Board will form a Ballot Resolution Committee. In many cases, one or more members of the (former) Working Group will be selected to participate. Ballot resolution involves careful consideration of every submitted comment and revision of the document to increase the degree of consensus. The committee will pay particular attention to the binding comments submitted on negative ballots. If these comments can be resolved to the satisfaction of the balloter, then the negative ballot can be converted to an affirmative one. Certain comments require response from the Ballot Resolution Committee. Every unresolved binding comment on a negative ballot must be refuted. In general, comments from observers and coordinating groups must also be addressed. After all comments have been considered, the revised draft standard, along with all unresolved binding comments and their refutations, are recirculated to the ballot group. During the recirculation ballot, balloters may change their vote to affirmative. They may also change their vote to negative based on changes in the draft or on unresolved comments from other balloters. This cycle of comment resolution, revision and recirculation continues until consensus is achieved among the balloting group. IEEE defines consensus as 75% approval. Although IEEE rules require the careful consideration of comments, they also require that the standard should move forward promptly

13 once consensus is achieved. Therefore, Ballot Resolution Committees should refrain from recirculations intended only to make minor gains in the numerical consensus above the required level. It is important to note that a recirculation is required whenever the draft document is changed in any substantive manner or whenever a new binding comment is received on a negative ballot. (In the case of a recirculation ballot, new binding comments may only be submitted on portions of the document that have been affected by changes.) This is because it is the job of the Balloting Group to decide whether objections are merited; the Ballot Resolution Committee only acts in anticipation of their decision. Once all this is finished, the development is completed, and the balloted draft standard is submitted to the IEEE for final approval. (Last minute suggestions for editorial improvements can be recorded and sent to IEEE along with the balloted draft.) The IEEE Standards Board considers whether the required procedures were correctly followed. If so, the standard is published. IEEE-SA staff is willing to work with the Ballot Resolution Committee prior to balloting to get the standard into publishable form. With careful planning, the balloting group can consider the document in its publishable form.

6.10. Process Completion Once the recirculation is successfully completed, the standard is submitted by SESC to the IEEE SB for final approval. Supplementary documents included with the standard are used to show compliance with IEEE and SESC P&P, and to show that the new draft meets the PAR purpose and scope. Once approved by SB, the draft will be edited by IEEE; however, the staff editors will not change anything that affects the technical content of the draft. The Working Group chair is expected to review all staff edits to ensure that the technical content has not been changed by the corrections made to the document. After the Working Group chair’s review, the standard will be published. These items are not a responsibility of the Working Group, but they do take some time, and the Working Group should not expect immediate publication. The amount of time required for final publication of the standard varies with the workload within the IEEE publishing organization but typically takes at least 3 months from final proof copy.

14 Annex A. Lessons Learned The SESC Management Board has noticed that there is a set of common reasons why people vote negative on IEEE Software Engineering Standards. Some reasons, of course, are technical. Many, however, have to do with the style and format of the standard. A great deal of grief can be avoided if these problems are resolved prior to going to ballot. So, this list gives some advice based on examination of negative ballots over a period of several years. The list is divided into two sections, giving mandatory rules and advisory comments. Mandatory rules. A Working Group shall follow these rules. They are important. Violation of any of these items is sufficient justification for a balloter to vote “no.” 1. Follow the IEEE Style Guide. Yes, it’s a bit long, but you will be required to follow it sooner or later, so why not save a rewrite? Be particularly careful to include the required material on the cover page and in the page footers. 2. Standards are meant to impose requirements - by convention, the forcefulness of the requirements are indicated by the use of certain verb forms, as explained in the style guide. Use these: “Shall” indicates a mandatory requirement - the standard user is required to do it in order to conform to the standard “Should” indicates a recommendation, but is not mandatory. “May” indicates permission, and is generally used in the context of a list. (“The user may include any of the items in the following list....”) “Must” is a verb that implies an inevitable result. For example, if the current is 10 amps and the resistance is 5 ohms, then the voltage must be 50 volts. The word “must” should never be used with normative intent – for example, as a synonym for “shall.” “Is” and other indicative forms are OK when giving an explanation, but carry no compliance implications. You should review every use of “is” to make sure it’s meant. Note that indicative forms are used in defining things, particularly in the body of the document. Nominative phrases are used in the Definitions Section of the standard. 3. Be very careful on the use of “shall,” “should,” and “may.” In particular, be consistent throughout the draft - a concept that is required one place and permitted some place else demands justification. 4. Include a paragraph somewhere that explains what a reader is expected to do in order to conform to the standard. In the simplest cases, this is only one or two sentences, and says that all the “shall” items must be done. 5. Make sure your standard is consistent with IEEE 12207.0, IEEE 1074, IEEE 1490, ISO 9001 and ISO 9000-3, if your standard concerns concepts

15 covered by these standards. Caution: your first guess on this subject is probably wrong. Nearly all standards that mention life cycle processes or the data and documents produced by those processes will have to ensure consistency with IEEE 12207 and IEEE 1074. Nearly all standards that mention management will have to deal with 1490. Nearly all standards that deal with quality principles will have to ensure consistency with 9001 and 9000-3. You may find that the cited standards are not themselves mutually consistent. If this should occur, you should consult your MB representative for advice. 6. The list of references in clause 2 is limited to those that are required in order to use the standard. Do not include any references that supply background information. Any such should be placed in an informative annex instead. In particular, other IEEE standards should be listed here only if this standard cannot be used without them. See the discussion in the style guide. 7. Definitions are given in clause 3. Do not include terms already defined in the Webster’s Collegiate Dictionary; IEEE assumes all such definitions. Do not include terms already defined in IEEE 610, 610.12 or 729. SESC is committed to following the terminology in IEEE 1490, IEEE 12207.0 and ISO 8402, so do not change any definitions given in these standards. It is OK to include definitions used in other software engineering standards provided that you indicate the source (as discussed in the style guide). 8. Definitions shall not be circular. Definitions shall not include discussion or examples - these belong elsewhere in the standard. Definitions shall not state requirements; requirements belong in the body of the standard. For example, one should not define the term “review” as “a meeting of customer, management and technical representatives to approve work products;” the mention of “customer, management and technical representatives” is an implicit requirement to include all of those roles. 9. The Working Group Chair and all members of the Ballot Group must be members of the IEEE Standards Association. If Working Group members wish to vote on their work, it is advisable to join the Standards Association prior to the formation of the Ballot Group. Advisory. None of the following items taken alone is likely to result in a negative vote. They are good advice, however. 9. It is OK to include discussion and examples, but they should be labeled as such. It shall be absolutely clear to readers what paragraphs are normative (must be followed) and what paragraphs explain the normative paragraphs. Do not mix normative and explanatory sentences in the same paragraph. 10. Standards have “clauses,” not “sections,” and “annexes,” not “appendices.” 11. One “conforms” to a standard; the verb “comply” means a number of other things, and should be avoided.

16 12. If you are developing a revision of an existing standard, include (in the Introduction) a list of the major differences between the old and the revised version. 13. Avoid normative figures. It is fine to include figures for information, but be sure to label them as such. Normative information should be written in text form; it is very difficult to create a normative figure unless a formal notation is being introduced. 14. Minimize definitions. Standards with many definitions tend to create unnecessary conflicts, both within the set of definitions and with other standards. Whenever possible, incorporate the needed material directly into the body of the standard. 15. It is wise to treat a standards effort as a formal project, with defined objectives, products and deadlines. There is a time limit, and the limit will be enforced. 16. Take advantage of your Management Board Representative. 17. Limit the contents of the standard to the essentials. If explanations and commentary are required, place them in an informative annex. Keep the body small and terse. 18. Maintain momentum. If the effort starts drifting, Working Group members are likely to drift away, and the project may well never complete. 19. It is worth the effort to have a few (more than one) “sea lawyer” on the Working Group. These people are asked to read the most impossible interpretations into seemingly straightforward language. The purpose is to ensure that the final draft has a few opportunities for misinterpretation as possible. 20. Plan to create a strong final draft, but don’t try to be perfect. You can spend months on arguing small matters of phraseology with little chance of resolution. Rely on the Ballot Group for such matters. 21. There will almost certainly be negative ballots. Plan for them, especially for new standards. Two or three rounds of balloting are not unusual. 22. It takes much more time and effort to create a standard than you imagine, particularly if you’ve never done it. In particular, the Working Group Chair should expect to spend significant amounts of time on this – the job is not a sinecure. 23. Arrange for an early review by the IEEE staff editor. Don't expect to do this after balloting. Some of the changes recommended by the editor may require the approval of the balloting group.

17 Annex B. A Practical Guide for Developing IEEE Standards Prepared by Norman F. Schneidewind

Here are some practical guidelines based on experience as a Working Group Chair since 1984 on the following projects: IEEE 1061 original standard plus its revision IEEE 982.1 revision AIAA R-013A\IEEE P1633 revision of ANSI\AIAA R-013 Blue Ribbon Committee that developed R-013. These are tips you will not find in official IEEE documents, or in textbooks, but they are based on experiences that were universal across several projects, both IEEE and AIAA. Based on these experiences, I feel the lessons learned can be stated as principles that can help you immensely in your standards efforts.

B.1. Documentation It cannot be stressed too much the importance of good and complete documentation. The reason is that you may have to provide it to the balloting group -- in particular, balloters who vote negatively on your standard and to the IEEE Standards Board in order to defend the contents of your document. A case in point is that in 1992 when the original 1061 was balloted, although it passed with a 92% affirmative vote, there was a significant negative vote by an important organization, with claims about the document that did not comport with the facts about its development. I resolved this problem by sending the complete package of documentation, including drafts of the standard, minutes of working group meetings, documents about votes of the working group on key issues, and correspondence with negative balloters. This saved the day for 1061 because the Standards Board said it was the best-documented standard it had seen! Having thorough documentation could save your project!

B.2. Balloting Groups One of the interesting dynamics of the entire standards development process is that folks you have never heard of -- not in the working group, not your colleagues, not experts in the field -- will come out of the woodwork to be members of the balloting group by some mysterious process that is not totally understood by me. You become a member of a balloting group by receiving an invitation to ballot based on your membership in the Software Engineering Standards Committee list. It seems that one can get on this list by asking or by having been active in IEEE standards activities and, therefore, known to the SESC. Anyway, the point is that now you will be dealing with a largely unknown

18 audience, who could sink your ship, if you are not careful. For example, there are a lot of balloters who do not believe in Software Reliability Engineering and are itching for a chance to torpedo a software reliability standard. How can you defend against a threat like this? One way is to provide lucid informative material in the document that explains the rationale for the standard, the key issues involved, and why the working group decided these issues as it did. A very good way to help your standard be a success is to provide a detailed case study of how your standard would be applied. Many balloters are practitioners. If they can see the application, it will help secure their affirmative vote. A category of balloters who can be difficult to deal with are pontificators who want to show off their expertise by taking your standard to task for not including their pet theories and ideas. In some cases, these folks do have very good suggestions; in other cases, their comments can be largely irrelevant to the intended scope of the standard. A way to militate against this problem is to use the Scope clause to clearly set forth what is included and excluded in the standard. This can be helpful in reducing the number of negative ballots based on “error of omission”. While it can be frustrating to have people vote negatively on your prized document, in many cases the comments are very constructive and should be received as such. It is similar to receiving negative comments on a paper submitted for publication. You may feel hurt initially: how could reviewers not think my ideas are a great contribution to software engineering? In a calmer light, we realize there is much to be gained by incorporating these comments into your document, whether it is a paper or standard.

B.2.1 Negative Votes on Informative Clauses Only the normative clauses are an official part of the standard. Informative clauses are for the information of the reader, and are not an official part of the standard. However, some balloters will still vote negatively on these clauses. A statement like the following can help clarify the situation for the balloter and protect you from inappropriate negative ballots: “ (These informative annexes are not a part of IEEE Std 1061-1992, IEEE Standard for a Software Quality Metrics Methodology, but are included for information only.)”

B.3. Openness in the Standards Development Process This is perhaps the most important of the practical tips. By IEEE policy, everyone can play at the standards development process. You must let everyone participate independent of his or her qualifications. This can be both beneficial and frustrating to the Working Group Chair. Beneficial, because a fresh point of view, even if not based on expertise in the field, can provide insights never considered by the experts because their noses have been too close to the subject. On the other hand this can be a frustrating situation. Some of these neophytes will join a working group because they have a new job assignment

19 that involves the subject of the standard. You will have a lot of work to do just running the working group, let alone be the one to bring them up to speed, but a mark of good leadership is to lend them a hand. Occasionally, people new to the subject of a standard will be naïve about the background required to make a substantive contribution to standards development. These folks are like students who want to undertake a thesis topic because it sounds “cool”, but have done no reading on the subject. In cases like these, I give the working group member some “homework”: I provide relevant documents and, if this is a revision to an existing standard, provide the original document. This helps a lot in improving the “maturity level” of your standards development process.

B.3.1 Proactive Contributors and People Who Prefer the Sidelines Although, as stated, it is essential to have an inclusive policy regarding standards development, experience suggests there will be a some members who join the parade, take risks, and are not afraid to bear the brunt of criticism for advancing their ideas, while others are content to watch the passing parade and perhaps provide an occasional comment. The latter are important for holding the former in check, in case they get carried away with their own ideas, but the former are probably going to make or break the standard. To illustrate, it is important to call for contributions from all members when a key task is to commence, such as developing an outline. However, the probability is that only a few will volunteer suggestions for the outline. My suggestion is to bond with these people like glue, because not only will the success of the standard depend on them, but your success as Chair, as well. It is important to provide special recognition for these contributors through personal commendation and through the IEEE Standards award process. Once these key members have made their suggestions, it is important to ask the entire working group to not only comment, but to give them a second chance to make suggestions, as well.

B.3.2 Electronic Communication versus Face-to-Face Meetings Although email and web sites are important resources to support the activities of working groups, electronics will never substitute for the benefits to be derived from in-face meetings. The optimal situation is to combine electrons with the body language of in-face meeting, where real understanding can be obtained of sometimes very divergence views. Of course, not everyone member will have the travel funds or absence of schedule conflicts, but just getting a few key people into a room for a few days to hammer out major parts of the document will result in significant progress. Again, it is important to distribute the results of these meeting to all hands and request feedback. This is where minutes of the meetings are very important not only for keeping the working group up to date, but for future possible reference during balloting and for interactions with the Standards Board, as well.

20 B.4. Guidance for Editing Standards Documents for Balloting

B.4.1 Introduction As a Working Group Chair, you should be aware of process and technical factors that could significantly affect your ability to achieve a technically accurate and readable version of your standard for balloting purposes. These factors are covered under “Process Factors” and Technical Factors” below. Note that the two categories are related, but are discussed separately for clarity of discussion.

B.4.2 Process Factors While it is traditional in journal, magazine, and book document preparation processes, both within and outside the IEEE, to provide significant document preparation support, you may not find this to be the case in the IEEE Standards Association (SA) pre-ballot editing process. Traditionally, outside of the SA, the publication process uses the concept that the author is responsible for the technical content of the document and the manuscript editor, who is an employee of the publisher, provides editing support for style, format, and clarity of expression issues. Based on recent experience, it has been learned that this level of support may not be provided by the SA editing process. Therefore, in order to ensure the accuracy and readability of your document, it is recommended that you do the following: 1) Attempt to obtain the services of a Working Group Editor with good word processing skills; this person may be you! This individual should possess the types of skills described in “Technical Factors” below. Note that in the future, it is possible that increased support may be available from the SA editing process, but to be on the safe side, you should assume the worst-case scenario. 2) Provide a pdf file, using Acrobat Distiller (not Acrobat PDF Writer), to the SA Editor that represents the document exactly the way you want it for balloting. Be sure to include the required Header, Footer, and Statement as shown below (the italics indicate information to be inserted). The reason for using Acrobat Distiller is that it will convert Microsoft Word mathematical symbols correctly, whereas this is not always the case in using Acrobat PDF Writer. (Note, however, that there is a charge for Acrobat Distiller.) As an option, and to be on the safe side, you could request a hardcopy printout, by express mail, of the document that will be submitted for balloting by the SA Editor. Note, however, that IEEE ballots are now conducted electronically; it is the pdf file, not the printed version, that will be distributed for balloting.

B.4.3 Header (on each page) IEEE project number/Draft number Date

21 (e.g., P982.1/D3 4 July 2003)

B.4.5 Statement to Appear Just before the Abstract Sponsored by the Software Engineering Standards Committee of the IEEE Computer Society Copyright © year by the Institute of Electrical and Electronics Engineers, Inc. Three Park Avenue New York, New York 10016-5997, USA All rights reserved. This document is an unapproved draft of a proposed IEEE Standard. As such, this document is subject to change. USE AT YOUR OWN RISK! Because this is an unapproved draft, this document must not be utilized for any conformance/compliance purposes. Permission is hereby granted for IEEE Standards Committee participants to reproduce this document for purposes of IEEE standardization activities only. Before submitting this document to another standards development organization for standardization activities, permission must first be obtained from the Manager, Standards Licensing and Contracts, IEEE Standards Activities Department. Other entities seeking permission to reproduce this document, in whole or in part, must obtain permission from the Manager, Standards Licensing and Contracts, IEEE Standards Activities Department. IEEE Standards Activities Department Standards Licensing and Contracts 445 Hoes Lane, P.O. Box 1331 Piscataway, NJ 08855-1331, USA

B.4.6 Technical Factors Typically, you would be using some version of Microsoft Word to produce your document. Word, in turn, is a member of a Microsoft Office suite, which in turn, is a member of a Microsoft operating system (e.g., XP Professional). There are many versions of these programs. Now, couple this with the various versions of Adobe Acrobat, plus PC, Macintosh, and Unix hardware, and you have a large combinatorial problem of potential incompatibilities! In addition, recognize that the SA Editor may be using different hardware and word processing software

22 than what you are using (e.g., Macintosh and its version of Word). This situation motivates the need to provide a pdf file to mitigate the risk of incompatibilities.

B.4.7 Non-Textual Information in the Ballot Draft In addition, if you use Word to produce your document, and you have non- text requirements, the following are some tips for maximizing the probability of producing an accurate and readable document for balloting:

Excel spreadsheet: If you have created a spreadsheet as a separate document that must be included in the Word document, copy just the columns of interest in the spreadsheet, and use Paste Special in Word, selecting the Formatted Text (RTF) option. This procedure will paste the spreadsheet into Word as a table that you can manipulate, if necessary, using Word’s table commands. Equations: Using Word’s Equation Editor or, preferably, the add-in MathType, use at least 12-point font to ensure that the equations will be readable. Mathematical Symbols: To help ensure that these symbols display on other platforms the way you intend them to display, use the Word Insert Symbol menu, choosing the character set Symbol to insert your symbols. Experience has shown that using this Microsoft default symbol set will result in the correct symbol on another platform, whereas using a different character set could result in the mathematical symbol “infinity” looking like a file cabinet symbol on another platform!

B.4.8 Converting from Word to PDF As mentioned previously, use Acrobat Distiller to do the conversion and make a detailed comparison of the pdf file with the Word file, watching in particular for accurate conversion of equations, mathematical symbols, and embedded graphics (e.g., Excel plots and Visio drawings).

23 Annex C. Guidance for Preparation of IEEE Standards

Prepared by Catherine Berger, IEEE-SA Standards Editorial Staff

C.1 Preparation of the Ballot Draft The IEEE has created a Word template that aids working groups with the format requirements of the IEEE-SA Standards Board. You can download this template from the following URL: http://standards.ieee.org/resources/glance_at_writing_new.html.

It should also be noted that the IEEE publishes standards using Adobe FrameMaker. Using FrameMaker will decrease inconsistencies considerably. FrameMaker templates are also available on the IEEE website. Note, however, that there is a charge for FrameMaker.

C.2 Summary of the Balloting Process

The IEEE Standards website is a great place to find the tools you need to prepare and ballot your standard. The “Standards Process at a Glance,” at http://standards.ieee.org/resources/glance.html, walks you through the process. The IEEE Editorial Staff has assembled the following quick reference summary of the Sponsor Balloting Process.

C.2.1 Invitation Stage For this part of the process IEEE does not need the document in pdf format. However they do need to make sure that proper format is followed. Please refer to the IEEE Style manual for proper format and tips provided in this documentation. (Complete both forms concurrently) 1) Submit Draft for Pre-ballot Editorial Review (20 days) http://standards.ieee.org/resources/development/balloting/pre- ballot.html

24 2) Request a Ballot Invitation (30+ days) http://standards.ieee.org/resources/development/balloting/bal-inv- form.html During the pre-ballot review period an IEEE editor will review the document for proper format only and send the working group chair a review with possible changes that should be made prior to sending it out for ballot.

C.2.2 Sponsor Ballot Stage For this part of the process the IEEE editors will need the draft document in pdf format. Please review the tips included in this documentation to ensure that proper format has been followed. When you submit the pdf file make sure that there is a designation in the upper right hand corner, that the copyright statement appears on the bottom of every page, and that it is clearly marked a draft. 1) Initiate Sponsor Ballot: (30+ days) http://standards.ieee.org/resources/development/balloting/bal-sub- form.html 2) Upload balloting files: (Draft must be in pdf format) http://standards.ieee.org/eprocess/upload_balloting_file

During the balloting period the WG chair (and possibly whoever they have designated to receive comments) will get an email as each balloter casts their vote and submits comments. See ballot resolution below for further tips on how to organize comments submitted.

C.2.3 Ballot Resolution You will receive all votes and comments via e-mail. To help you organize and address them you may want to use the multiple comment templates located at: http://standards.ieee.org/resources/spasystem/balloting_comments_ template.xls Cutting and pasting the comments into this spreadsheet will allow your ballot resolution committee to review and record their response to each comment. This spreadsheet can then be used in the Recirculation ballot to show how the committee addressed all the comments. Remember when making changes to the draft, highlight them by using bold text, strikethrough, and change bars. In a Recirculation you only ballot on those changes made to the document. Also remember to change the designation to the next version and date (i.e., P1234/D2 month year)

25 C.2.4 Recirculation Ballot Requesting recirculation is nearly identical to the process for requesting the initial sponsor ballot (see C.2.2). 1) Go to the following URL to Initiate Sponsor Ballot: http://standards.ieee.org/resources/development/balloting/bal-sub- form.html

2) Upload the files below to: http://standards.ieee.org/eprocess/upload_balloting_file  Draft document in pdf format (showing highlighted changes)  Comment Resolution  Cover letter

C.2.5 Other Useful Links The following additional links may also be helpful. 1) Standards Development Online (A step-by-step guide through the process) http://standards.ieee.org/resources/development/index.html 2) Extension Request (If your PAR is expiring this year, you will need to file for an extension.)

http://standards.ieee.org/resources/development/initiate/changes.ht ml 3) Working Group Chair Change Form (If you have recently taken over the working group position for a project and are not listed on the PAR you need to complete this form.) http://standards.ieee.org/resources/development/initiate/changes.ht ml 4) To Join an Existing Balloting Pool (If you wish to receive invitations to ballot on other SESC projects, complete this form.) http://standards.ieee.org/db/balloting/ballotform.html

26 Annex D. List of Changes from Version 1.0 Version 2.0 of this document, dated July 11, 2003, contains the following changes from Version 1.0, dated April 7, 2000.

1. Inclusion of references to IEEE Standards Association (IEEE-SA) 2. Addition of statement (clause 1.2) that IEEE P&Ps take precedence over SESC P&Ps 3. Inclusion of most recent publication dates for IEEE Official Documents (clause 2.1) 4. Inclusion of most current URL for SESC web page 5. Use of most current reference numbers for SESC procedures 6. Updated list of Other SESC Documents (clause 2.4); some obsolete documents have been deleted 7. Modification to clause 6 to reflect changes in the IEEE PAR 8. Inclusion in clause 6.1 of a new requirement to brief working group members on the IEEE patent policy 9. Revision to clause 6.8 to discuss online balloting of standards 10.Some minor editorial changes 11.Inclusion of Annex B, “Practical Guide for Developing IEEE Standards” 12.Inclusion of Annex C, “Guidance for Preparation of IEEE Standards”