Critique of Executive Summary of Cost-Effectiveness of EU Chemicals White Paper
July 15, 2002
Study of Cost-Effectiveness of the Proposed EU Chemicals Policy
The following comments from the American Chemistry Council are based on a reading of the executive summary of a study undertaken by Risk Policy Analysts, Ltd. (RPA) on behalf of the European Commission (EC). The Council has not read the entire report because it is currently unavailable. Nevertheless, because implementing legislation is being developed and because the executive summary is sufficiently detailed, it is appropriate to provide comment at this time.
The RPA study reinforces some of the conclusions drawn by others on the impact of the proposed chemicals policy. For example, testing costs clearly represent the largest cost associated with the information provision components of the proposal, much greater than the cost of registration, authorization, or program administration.
The Council commends RPA for developing a survey and incorporating the survey results in its analysis. Data based on survey results (e.g., the number of substances on the market, the number of site-limited intermediates, the number of uses per substance, dossier preparation costs, characterization of SMEs, and the percentage of chemicals that may be withdrawn from the EU market) are superior to the assumptions that others have used to estimate impacts. The study has greater validity than it otherwise would have because of the time and effort taken to survey the business community.
The Council also commends RPA for developing a more detailed cost model than any other that has been developed to assess the impact of the White Paper. For example, the study identifies costs of data generation already attributable to other EU legislation and/or regulation. Other studies did not subtract these costs from the total and therefore overestimated the cost of the program.
Despite these contributions, the study is not as useful as it could be. The authors could greatly improve the study by making the following changes:
- Instead of the focus on the direct costs associated with information generation, the study should instead investigate losses to consumers and producers due to increases in the prices of chemical products and the removal of chemical products from the European market.
- Consider alternatives that might provide greater net benefits than the approach specified in the White Paper (e.g., a streamlined registration process, a tiered approach to testing, elimination of deadlines for authorization, and an exemption for articles).
- Certain key assumptions (choice of discount rate, per-chemical cost of testing) should be changed to better estimate the costs associated with information generation and provision.
- The entire study should be subject to independent peer review and public comment.
Focus on Social Costs and Benefits
The executive summary states that the purpose of the study is to help inform the Commission on the “relative cost-effectiveness of different mechanisms and procedures for implementing the objectives of the White Paper.” The most useful study would focus on the social costs and benefits of the proposal. Unfortunately, aAs written, the executive summary describes a relatively narrow focus: the direct compliance cost to the business community of information provision.
The social costs of the proposed policy described in the White Paper include compliance costs, transitional costs, administrative costs, and losses in consumer and producer surplus.[1] The social benefits of the proposed policy include reduced anxiety over chemicals management, health and environmental benefits resulting indirectly from information developed under the program, and any gains in consumer and producer surplus.
Analyzing all such benefits and costs would be a very resource-intensive exercise, but considerable simplification is possible. For example, the Council believes that the study should not focus on the cost of risk management actions other than the cost of decisions under the authorization process (e.g., use of information gleaned from evaluation to craft better air pollution regulations). Such risk management decisions are more appropriately examined under other relevant regulatory programs. The Council also believes that the proposed policy will not result in gains in consumer and producer surplus.[2] Certain costs (transition costs, administrative costs) and benefits (from reduced anxiety over chemicals management) are likely to be relatively low (compared to testing costs) and therefore not worthwhile candidates for extensive analysis.
This reasoning narrows the social costs (compliance costs associated with information provision and losses in consumer and producer surplus) and benefits (health and environmental benefits associated with decisions not to authorize chemicals for specific uses) to a manageable number. A further simplification is possible when it is realized that compliance costs are a component of losses in producer surplus.
Estimates of compliance cost associated with information provision (the focus of the study) are likely to represent a lower-bound estimate of losses in producer surplus, assuming generally competitive markets and stable consumer demand for chemical products. The remaining costs (associated with losses in consumer surplus) and benefits (health and environmental benefits associated with decisions not to authorize a chemical for specific uses) have been largely ignored in the executive summary and presumably in the study itself.
Losses in consumer surplus arise when the price of a product increases and/or the value of a product decreases. To estimate losses in consumer surplus associated with price increases, the Council suggests RPA focus on classes of consumer products most likely to experience a price increase as a result of the proposed policy.[3] To estimate losses in consumer surplus associated with a decrease in product value, the Council suggests that the RPA focus on chemicals subject to authorization.[4]
RPA should consider carefully the benefits that might arise from the proposed policy in terms of improvements in human health and environmental quality. Only direct benefits—those associated directly with a decision not to authorize a substance for a specific use—should be considered. As part of this assessment, RPA should consider any risks that might arise from a market response to a decision not to authorize a substance for specific use.
The Council is convinced that the proposed policy—as described in the White Paper—will harm consumers. This conclusion is based on the fact that, under the proposal described in the White Paper, chemicals will be banned from the EU market if information needed for registration and/or authorization is not provided to the government within the prescribed timeframe. Because the timeframes noted in the White Paper are ambitious, there is a high probability that low-risk chemicals and low- risk chemical uses will be banned. Consumers will be forced to buy products and/or services that provide less value and/or are more costly. The issue is not whether consumers will be harmed, but to what extent.[5]
Consider a Wider Range of Alternatives
A focus on social costs and benefits would help to evaluate alternatives outside the scope of the White Paper. In particular, the Council encourages the EU to examine the consequences of alternatives to the proposed registration and authorization processes, which are driven by deadlines. If information is insufficient to support registration or to make an authorization decision, the EU will ban that use of the substance from the marketplace. Such an outcome would hurt consumers if the chemical or chemical use in question poses relatively low risk and substitutes for the chemical aren’t adequate (in terms of price and performance) or available. The study should consider changes to the White Paper that would avoid the possibility of substantial losses to consumers.
Another alternative worth examining is an exemption for articles. The Council was disappointed that no attempt was made to shed light on this issue. The executive summary states: “It has not been possible within this assessment to consider the complex issues surrounding the treatment of substances in articles within REACH and, thus, to address the impacts that any proposals might have on competitiveness and trade for downstream user sectors.” RPA should undertake such an analysis, even if it is difficult. Just as with the failure to consider alternatives to authorization deadlines, RPA has ignored not analyzed an extremely important issue that has the potential for large-magnitude, adverse impacts. By ignoring tThis alternative, the study fails to addresses an issue that should be the focus of EC deliberations.
Because it acknowledges the importance of testing on compliance costs and the vulnerability of SMEs under the proposed policy, the study should consider alternatives to reduce the cost of testing to companies. Several suggestions have been made that would allow key information to be developed at a fraction of the cost of the testing regimes envisioned in the White Paper. For example, some have suggested the idea of postcard registration. Others have suggested a tiered approach to information development. Such alternatives should be explicitly considered.
The study should be revised to consider the impact of these alternatives on social welfare.
Reconsider Certain Key Assumptions
RPA should reconsider many of the assumptions used in its study. This section describes some of these assumptions and suggests alternatives.
Choice of Discount Rate. The study uses a discount rate of 3%. The study should instead use a range of plausible discount rates. The lower value should correspond to the social rate of time preference. The higher value should be a multiple (perhaps a factor of 3) of the lower number to account for costs that diminish private investment to account for the shadow price of capital. The value selected for the study, 3%, seems to represent the lower end of this range. In the United States, regulatory agencies are instructed to use 7%, which is considered a compromise between those who advocate for a lower value and those who advocate for a higher value.
Evaluation (testing) of Chemicals Produced in Low Volumes. The study assumes that chemicals produced in low volumes will not require evaluation involving expensive, substance-tailored testing. A more realistic assumption would be to assume that a certain number of low-volume chemicals would trigger a requirement for further evaluation. To determine the number of low-volume chemicals that might require chemical testing, RPA should make an assumption based on EU experience with new chemicals.
Cost of Individual Tests. The study acknowledges that the cost of testing chemicals is based on data from one company. Given the fact that testing costs represent 88% of the estimated total direct compliance costs, the Council believes that data on per-chemical testing costs should be robust. Unfortunately, the study relies on only one source for its per-chemical testing cost data.
The Council used a recent survey of a handful of member companies and testing laboratories to obtain estimates of the costs of specific tests and the costs of generating certain data. This information was used to estimate the cost of base set testing (basic information requirements), level 1 testing, and level 2 testing. Table 1 provides a summary of the ACC results compared to the study results; the ACC estimates are higher in every case.
Table 1. Comparison of Estimates of Testing Costs.
Description / RPA Study ($) / ACC Study ($)BIR testing, >1 t/y / 28,881 / 43,590
BIR testing, >10 t/y / 142,574 / 232,666
Level 1 / 386,216 / 1,716,292
Level 2 / 628,728 / 2,547,616
More specific information is provided in the appendix. One possible explanation for the discrepancy between the ACC estimates and those of the RPA is that the Council focused on costs in the United States. It may be that EU testing costs are generally lower than that of the USA, but the difference is large enough to raise questions about the magnitude of the RPA estimates. Another reason for the discrepancy difference is the fact that the White Paper is not specific enough in its description of the specific tests that will be employed. Consequently, judgment must be used to identify the specific tests to be required.
Number of Uses Subject to Authorization. The authors of the study assume only one use per chemical subject to authorization and justify this assumption by not accounting for savings associated with the formation of testing consortia.[6] The Council does not believe such an assumption can be justified absent analysis. Because analysis is needed to justify the assumption, then RPA might as well use a cost model that (1) accounts for a more realistic number of uses subject to authorization and (2) accounts for savings associated with the anticipated formation of consortia.
Laboratory Capacity. The study assumes that the current cost of testing will remain constant after the proposed policy is adopted. The Council believes that the cost of testing will rise dramatically in the short term given the limitations on laboratory capacity. The Council believes RPA should assume some increase in testing costs, at least in first year or two of program, to account for the sudden increase in demand resulting from adoption of the policy. RPA should survey laboratories throughout the EU in order to gain an understanding of existing laboratory capacity, the likely increase in prices that will occur after imposition of the proposed policy, and the potential impact of a lack of laboratory capacity on meeting the deadlines outlined in the White Paper.
Use of the Mean for Non-Normally Distributed Parameters. The study inappropriately uses the mean for certain costs relating to authorization (preparation of a risk assessment and a socio-economic justification). Because the distribution of costs for authorization is likely to be positively skewed, this approach has the effect of underestimating the cost of authorization. The Council suggests two alternative methods to correct this problem. Preferably, RPA should specify the underlying distribution and use Monte Carlo analysis to estimate the cost of the program. The next best approach would be to use a range of values for the cost of risk assessment and socio-economic justification or the median value.
Publish the Survey Results
The executive summary describes some of the survey results that were used in the analysis. If the study does not include a question-by-question statistical summary of the survey responses, the study authors should include one in an appendix. Such information is useful for many who wish to better understand the characterization of entities affected by the proposed policy. RPA should specify those instances in which their own judgment was used instead of the survey results to generate estimates of direct compliance costs.
Subject the Revised Study to Peer Review and Public Comment
The Council does not know if the study has been subject to independent, external peer review. Such an exercise is valuable. In a guidance document developed for the OECD[7], RPA wrote the following about peer review:
Peer review of socioeconomic analyses may be important to ensuring that the analysis is robust and will stand up to external scrutiny. The aim of such reviews is to validate both the data and assumptions used within an analysis, and the manner in which those data are analyzed. . . . A peer review is likely to be of most value in those cases where there are significant trade-offs involved in the choice of risk management options . . . . In particular, when the costs of risk management are expected to be high and may have significant impacts on a particular industry sector, a number of sectors or the economy more generally, it is likely to be an important part of the process.
There is no indication that the RPA study was subject to peer review. The Council strongly recommends that the Commission subject the study to external, independent peer review and public notice and comment. Such steps can only improve the credibility of the study.
Appendix: Comparison of Test Cost Estimates
ACC compared its estimates of testing costs with those used in the RPA study.
The executive summary references the tests listed in Annexes to the Directive 67/548/EEC but only presents a total cost for the base set of tests, level 1 testing and level 2 testing. It does not specify which tests would be done. The Council used expert judgment to identify the tests that would be appropriate based on the annexes to Directive 67/548/EEC. The source of the per-chemical test costs was a survey of a handful of member companies and U.S. testing laboratories.
The Council determined the total cost for each level of testing (Table 1) only after identifying the specific tests involved. For some of these specific tests, the Council did not obtain an estimate of cost. The estimates of total cost, therefore, are less than that anticipated.
For the base set of tests (Tables 2 and 3), the ACC estimate is higher than, but of the same order of magnitude as, the RPA estimate. Larger discrepancies were found between the estimates for Level 1 and Level 2 testing. The difference may be due to the cost estimates for the chronic toxicity and carcinogenicity test (see Tables 4 and 5).