DOCUMENT RESUME

ED 392 879 CE 070 813

AUTHOR Blalock, Ann Bonar, Ed. TITLE Economic Competition, Restructuring and Worker Dislocation. INSTITUTION and Administration (DOL), Washington, D.C. PUB DATE 95 NOTE 166p. PUB TYPE Collected Works Serials (022) Reports Research/Technical (143) JOURNAL CIT Forum; nli Sum 1995

EDRS PRICE MF01/PC07 Plus Postage. DESCRIPTORS Adult ; Agency Cooperation; *Competition; Coo.lerative Planning; Demonstration Programs; *Dislocated Workers; *Economic Change; Educational Needs; Education Work Relationship; Employment Level; Employment Patterns; Employment Qualifications; Foreign Countries; *Industrial Structure; International Programs; Training; *Labor Force Development; Labor Market; National Programs; National Surveys; Partnerships in Education; Performance Based Assessment; Program Evaluation; *Public Policy; Resource Materials; Retraining; School Business Relationship; Self Employment; Standards; State Programs; Structural ; Training; Trend Analysis IDENTIFIERS Defense Expenditures; Defense Workers; European Community; Perestroika; USSR

ABSTRACT In thie issue various concepts of economic change are examined. Part I looks at economic change as a context for focusing on workforce issues. Parts II and III discuss what isbeing done to address this contemporary problem. This issue contains the following papers: "Presidential Perspectives on Economic Change and Restructuring: An Interpretation"; "Understanding and Adjusting to Economic Change"; "Challenge in Restructuring: Efficiency vs. Equity"; "Understanding the Context of Dislocation"; "Worker Dislocation: A Review Article" (Ernst W. Stromsdorfer); "Evaluating Dislocated Worker Programs: U.S. Department of Labor Studies in the 1990s"; "Ot:her National-Level Studies in the 1980s and Early 1990s Relevant to Dislocation"; "Self-Employment as an Employment Strategy-- for Dislocated Workers"; "Defense Downsizing: Some Implications"; "Studying Defense Conversion in Affected Communities"; "Programs To Ease the Adjustment to Defense Cutbacks"; "Findings from the Defense Conversion Adjustment Demonstration Project" (Mary G. Visher, Deborah Kogan); "A Commentary by Secretary of Labor, Robert B. Reich"; "An Interview with Salvdtore Zecchini"; "An Interview with Gary K. Jackson and John E. Dryer"; "Economic Reform in the European Union"; "The OECD's (Organization for Economic Cooperation and Development) Employment Study"; "The Persistence of Unemployment in the Postindustrial Era"; "Labor issues in European Consolidation"; "The Political Economy of Borrowing'"; "A Selective Overview of Soviet Reform Policies"; "Gorbachev's Perestroika"; "The USSR in the Transition Period"; "Some Thinking about E&T (Employment and Training) Coordination at the National Level"; "Measuring Performance in Prograns and Coordinated'Systems"; "More about Performance Management"; "Using Performance Standards in Establishing Workforce System Accountability"; "Performance Management Systems: Evaluation Principles and Standards"; "A Brief, Selected History of Coordination Efforts"; "Establishing Comprehensive Workforce Development Systems: State-Level Efforts in the 19908"; "Contemporary Coordination Efforts: The Core Data Elements Project"; "Comprehensive State Workforce Development Systems"; "The National Governors Association's Performance Management Project"; "References on Economic Change, Restructuring and Working Dislocation" (contains 314 references); and "Federal Research: An Update." (MN) - Evaluation Forum

Economic Competition, Restructuring and Worker Dislocation

U.S. Department of Labor EmplOyillent and Training Administration

Issue 11, Summer 1995

0 0

U S DEPARTMENT OF EDUCATION Office 1 Educahonai Research anc Improvement EDUC TIONAL RESOURCES INFORMATIOtt CENTER (ERIC) his document has been reproduced as recelvsd from the person or organization originating It 13 Minor changes have been made to improve reproduction quality

0Points of view or opinions staled in this document do not necessarily reprosent official OERI position Of p011Oy.

BEST COPY AVAILABLE -

Material contained in this publica- Evaluation Forum is a national jour- Editor and Reviewer tion is in the public domain and may nal for program professionals in re- Ann Boner Blalock be reproduced, fully or partiallly, lated human service fields: basic Admiralty Inlet Consulting without permission of the Federal education, , Hansville, Washington Government. Source credit is re- employment and training, welfare, Guest Reviewers quired only to reproduce any copy- labor market and labor force Ernst W. Stromsdorfer righted material contained herein. change, and economic develop- Department of ment. The purpose of the journal is Washington State University This material will be made available to stimulate interest in policy analy- Pullman, Washington sis, policy research and program to sensory impaired individuals Kay Thode evaluation as useful tools for upon request. Welfare Consultant Voice phone: 202-219-5782 policymaking, planning and the im- Seattle, Washington TDD* phone: 1-800-328-2577 provement of social programs. It is understood that the ideas and opin- Technical Services Kathleen Shankman *Telecommunications Device for the ions expressed in the content of the Center for Governmental Studies Deaf. journal are those of the respective writers and do not necessarily rep- Northern Illinois University resent those of the funder or those DeKalb, Illinois of the contributing public interest organizations. Sponsor and Funder Office of Policy and Research Employment and Training Administration U.S. Department of Labor Contributors to Journal Distribution National Governors' Association Washington, D.C. National Aosociation of Counties Washington, D.C. National Alliance of Business Washington, D.C.

;

; vagua ion oru

Economic Competition, Restructuring and Worker Dislocation

Issue 11, Summer 1995

U.S. Department of Labor Robert B. Reich, Secretary

Employment and Training Administration Tim Barnicle, Assistant Secretary

Office of Policy and Research Gerard F. Fiala, Administrator

A review journal for employment and training professionals focusing on policy analysis and evaluation research in occupational training, education, work and welfare, and economic development.

r:

4 Table of Contents

Economic Change and Displacement Features 2 Presidential Perspectives on Economic Change and Restructuring: An Interpretation President Clinton's view of economic change and related issues, based on an anthology of his speeches. 7 Understanding and Adjusting to Economic Change A review of The Urban Institute's Rethinking Employment Policy. 11 A Challenge in Restructuring: Efficiency Vs, Equity A review of two informative books on economic and employment policy in the 1990s published by The Brookings Institution. 15 Understanding the Context of Dislocation A brief overview of four major books on technological change, economic restructuring and employment. 16 Worker Dislocation: A Review Article A guest review of three W.E. Upjohn Institute books on worker displacement, by labor economist Ernst W. Stromsdorfer. National Studies of Dislocated Worker Programs 21 Evaluating Dislocated Worker Programs: U.S. Department of Labor Studies in the 1990's A summary and commentary on U.S. Department of Labor process and net impact studies of programs designed to assist dislocated workers. 29 Other National-Level Studies in the 1980s and Early 1990s Relevant to ,Dislocation A review of analyses and evaluations of efforts to retrain and reemploy dislocated workers, conducted by the U.S. General Accounting Office, the Office of Technology Assessment, the Congressional Budget Office, and other sources. 33 Self-Employment As An Employment Strategy for Dislocated Workers A discussion of self-employment programs in the U.S. and Europe, and a review of evaluations of the major U.S. self-employment projects. Defense Conversion and Dislocation 37 Defense Downsizing: Some Implications Contextual information about defense conversion, and a brief review of several Office of Technology Assessment rep rts on conversion issues. 40Studying Defense Conversion in Affected Communities A summary and analysis of a major study of defense adjustment and conversion, sponsored jointly by the U.S. Departments of Commerce and Defense. 43 Programs to Ease the Adjustment to Defense Cutbacks An overview and discussion of the plethora of assistance programs for workers and communities impacted by defense downsizing.

41.0011111111M13Mir Evaluation Forum, Issue 11 Summer 1995 1ii 5 46 Findings from the Defense Conversion AdjustmentDemonstration Project A guest review ofe U.S. Department of Labor's evaluation ofthe demonstration, by Mary G. Visher, d-tor of the evaluation, and Deborah Kogan, its co-principal investigator.

Commentary 53 A Commentary by Secretary of Labor, Robert B. Reich A commentary on the status of the Americaneconomy, on employment and dislocation, and on income inequality, with implications for poacy.

Interviews 57 An Interview With Salvatore Zecchini An interview with internationally-respected econc_nist, Salvatore Zecchini, Assistant Secretary-General of the Organizstion for Economic Cooperation and Development in Paris, on economic restructuring in Central andEastern Europe and its policy implications.

62 An Interview With John Dryer of The Boeing Company,and Gary Jackson off the International Association of Machinists andAerospace Workers An interview with the prime movers in developinga remarkably successful corporation-based partnership model for assisting dislocated workers,The Boeing Reemployment Program.

Evaluation Issues At the International Level and Activities Economic Reform in the European Union 68 The OECD's Employment Study A review of OECD's Job Study, its implications for employmentpolicy, and its policy recommendations.

72 The Persistence of Unemployment in the Post-IndustrialEra An overview of two analyses by OECD economistson the reasons for long- term unemployment in the EU.

76 Labor Issues in European Consolidation A review of a Brookings Institution book by three labor economistson the potential effects of EU consolidation on labor forces and labormarkets.

77Political Economy of "Borrowing" A review and analysis of a provocative book about the transnational borrowing of employment and training ideas between the U.S. andGreat Britain, by a Brookings Institution economist anda British political scientist now at the Rand Corporation.

WOW ..11 Evaluation Forum Issue 11.Summer1995 Economic Reform in the Former Soviet Union 83A. Selective Overview of Soviet Reform Policies A context for understanding current economic restructuring in Russia, a review of Edward Hewitt's classic, Reforming the Soviet Economy. 88 Gorbachev's Perestroika . Ananalysis of Gorbachev's reform ideas. 90 The U.S.S.R. in Transition An overview of a fascinating book published by the OECD, Structural Change in Central and Eastern Europe. At the National Level 94 Some Thinking About Employment and Training at the National Level A selected history of human service coordination/integration efforts over the past few decades. 97 Measuring Performance in Programs and Coordinated Systems A review of two informative monographs by economist:, at the University of Baltimore on performance standards, and by the Office of Technology Assessment on the use of UI records in evaluation. 103 More About Performance Management A review of several useful reports on "accountability" produced by the Council of Governors' Policy Advisors and the National Institute for Literacy. 105 Using Performance Standards in Establishing Workforce System Accountability Review of economist Burt Barnow's excellent analysis of the pros and cons of using performance standards, in Evaluating Welfare and Training Programs edited by Charles Manski and Irwin GarfinkeL 109 Performance Management Systems: Evaluation Principles and Standards A summary of key elements of the American Evaluation Association's new standards for evaluation practice, in James Sanders' The Program Evaluation Standards published by Sage. At the State and Local Level 115 A Brief, Selected History of Coordination Efforts A discussion of state and local efforts to coordinate their employment and training programs in the 1970s through 1990s. 118 Establishing Comprehensive Workforce Development Systems: State- Level Efforts in the 1990s 120 Contemporary Coordination Efforts: The Core Data Elements Project An overview of the joint efforts of the U.S. Departments of Education, Labor, Health and Human Services and Agriculture to develop a core set of variables, operational definitions, and quantitative measures cutting across multiple employment and training programs.

Evaluation Forum.Issue 11 Summer 1995 123 Comprehensive State Workforce Development Systems A summary of ideas in Northern Illinois University's Building State Workforce Development Systems Based on Policy Coordination and Quality Assurance, the major analytic resource for NGA's Performance Management Project. 127 The National Governors Association's Performance Management Project An in-depth summary and perspective on the NGA project.

Resources 142References on Economic Change, Restructuring and Working Dislocation Books, articles and reports on the above issues, and relevant international ii references. 153Federal Research A selected ur date on employment and training-related research in the federal agencies.

ANIMMICNIMOMM Eilemorial to SarLevita775-61 wenurPowib raarimmallammoamrt I

All

Evaluation Forum Issue 11 Summar lggS Features

Editorial Intrbduction The changed international trade re- nesses operate in the global economic alities. marketplace. The terms "economic efficiency, revi- The end of the Cold War. Illustrations talization, restructuring and reform' The development of advanced high- Specific examples of such changes have now become well accepted within are everywhere apparent. We have in fact, techology products and markets. the postindustrial vocabulary seen unprecedented partnerships es- warmly embraced. We freely insert The new alliances between devel- tablished among some of the cable these concepts Into our narrative piec- oped and developing countries. television industry's largest companies, es as if they had bean commonplace which will give customers a new range through the centuries and there Is an Distinctions of telecommunications products and international consensus about their In understanding what is meant by options. These partnerships will not meaning. But do they have that consis- "restructuring" we need, of course, to only affect wireless communications but tent a history and is there agreement distinguish between cyclical and non- also one-stop shopping for many ser- on how they are defined? The easiest cyclical causes for economic reorgani- vices. answer is "No." zation. Reduced domestic demand for The rise of Hewlett-Packard in the With many international economists, certain products, trade defid.ts, and the computer printer market has been a "economic reform" has been summarily shrinkage of market share In interna- dramatic story. The company now holds replaced by "economic restructuring*, tional competition are largely cyclical approximately 55% of the world market and subsequently adopted by the rest phenomena. Firms are pushed to re- for inkjet printers, and has retrieved the structure by forces over which they may of the world. However, the meaning U.S. lead in related technologies. H-P's continues to vary. This eminently fash- have minimal or no control. The other success has made it one of the two ionable phrase may describe an array restructuring scenario is influenced fastest-growing major U.S. multination- of postindustrial phenomena: more by the efforts of management als. The company has attributed its suc- and/or labor to increase economic effi- The creation of joint ventures. ess to its heavy investment in laboratory ciency and within business- breakthroughs, its drive to capture mass The merging of previously separate es and industries. businesses and industries. markets and sustain rapid product inno- It is important to recognize also that vation and price cutting, its trust in local The consolidation of product devel- most restructuring efforts are motivated creativity and accountability, and the cut- opment thinugh new business part- primarily by economic preservation, ting of its internal operating costs. nerships. which has the effect of staving off the Military technology has been applied mass unemployment that would occur The development of "niche" markets. successfully, in the absence of the Cold with the complete failure of businesses War, to domestic and international mar- The expansion or diversification of even though it frequently involves kets. Scientists who helped the Navy products by a single multinational. significant levels of unemployment and create a sophisticated submarine-track- worker dislocation at the time, and diffi- The downsizing of corporate work- ing computer system are now applying cult readjustments for many of the forces. similar advanced digital processing workers affected. technology to the identification of trou- Causes The reality is that economic competi- ble spots in jet engines, nuclear reactor Some of these activities have been tion is no longer contained within na- coolant pumps, and electricity-generat- characteristic of the economies of in- tional or regional boundaries. New eco- ing steam turbines. This translation from dustrialized nations for some time. nomic powers have emerged within this military to civilian markets has made new They have fueled economic growth and altered environment, to compete with economic efficiencies possible. created . However, the postindustri- the U.S. for international customers, In the Northwest, we see Oregon';' whose needs and desires have also al period has brought with it new chal- restructuring of the timber industry. Tra- lenges due to heightened international changed. Amencan businesses and in- ditionally, Oregon has been the top tim- economic intera3pendence. The basis dustries have had to rise to this occasion, ber-producing state. The drastic curtail- if they were to support general economic for these challenges lies in a number of ment of logging in federal forests developments: health, and the life quality Americans represented a major challenge to the have attained since World War II. The movement of postindustrial na- state's economy. But Oregon managed tions from manufacturing to services. A variety of "restructuring" strate- to produce five billion board feet each gies had to be designed and tested in year using wood from tree farms rather The revolution in information and this context. Some have been success- than the 300-year-old trees from public telecommunications. ful in preserving economic progress land, and the mills are producing more The new political positions affecting while protecting the gains of the Ameri- products out of this timber by using social spending and trade relations. can labor force, and others have not. parts that were discarded previously. In But these new economic methods have the restructuring process, Oregon lost The Increased concern about the clearly revised the way American.busi- planet's physical environment.

E valuation Forum.issue 11 Summer 1995 1 _east _ teee 3 47

Features Econornio Chana and Dis lacernent ..,,..

nearly 15,000 jobs in forest products permanent job loss. Many of the work- ganize and finance new products. Mat- over the past five years, but gained ers laid off are skilled high-wage work- tell Inc. recently announced a restruc- nearly 20,000 in high technology busi- ers but their occupational expertise pro- turing plan that would eliminate 4.5% of nesses. vides few if anj opportunities to its woratorce, which was expected to Dislocation transition to other occupational areas. reduce Matteli's expenses by $25 mil- Many companies have tried to ease the lion in 1995. hi 1594, Kmart planned to Despite restructuring successes, it is impact of downsizing, but markets end clear that the postindustrial period has dose 110 startle and eliminate 6,neo profit marginc in emerging economic ar- jobs, including 10% of its managenal had its downaide. Although we are en- eas inevitably intluence the extent of positions, related to its loss of market joying general economic growth, there downsizing. And in some cases mass is a long-term increase in income in- share to competitor& The Boeing Com- layoffs have been inevitable if compa- pany's own series of layoffs was due equality in the U.S. The trend toward a niee are to survive and prosper. Some greater gap between the wealthy and both to mimed customer demand and dislocated workers can be retrained restructuring in the interest of retaining the poor is attributed by many experts successfuliy for other occupational a competitive edge in i.vorid aerospace to wage stagnation and erosion, which niches, but frequently at lower . markets. And these are but a few of the in turn is correlated with the shift to a And some fall into . service economy, Increased interna- many examples of companies that have had to include 'nyoffa In their efforts to tional competition, the decline of union- The recent IBM layoffs are an exam- reorganize and compete. ization, and the decline of the minimum ple of tha dislocation often liwolved in wage. corporate restructurino. Cray Research in this issue of Evaluation Forum, we Ince a supercomputer maker, reeently explore some of tne issues raised In And certainly the downsizing of laid off 1,000 workers to cut its manu- workforces is a major restructuring is- this introduction. In Part I we look at facturing costs. American Express economic charge as a context for fo- sue. Worker dislocation unquestionably planned to eliminate as many as 6,000 represents another reason for rising in- cesing on woraforce Issues. In Parts If jobs in an effort to recover from several and Ill we look specifically at worker equality. Cost containment In the inter- lean years, and to save $500 million in est of economic efficiency has led to dislocation and what Is being done to expenses which could he usod to reor- address that contemporary problem.

Economic Change and Displacement

Editorial Introduetion the international economic environment mists specializing in restructuring and on that development, and the impact of dislocation issues. We conclude Part I The purpose of Part I of the Features economic change on workers affected with a guest review of three important Section is to provide some background by it. In serving this purpose, we pro- books that focus more exelusively on information on the recent evolution of vide a number of perspectives, from the wotker dislocation. the American economy, the influence of President's to those of various econo-

Presidential The knowledge and understanding a municating key themes with the intent to president possesses about the economy reflect as accurately as possible what ideas Perspectives on and its larger context, '.'4,tct his policy were conveyed throngh that medium. Economic Change positions and perspectives on economie issues, influence policy development in Economic History and Restruc. uring: a uniquely powerful way within the In remarks at the signing of the NAF- framework of a tri-partite governing An Interpretation TA side agreements in September, structure. This president has paid un- 1993, President Clinton commented that usual attention both to economic prob- Perhaps the most significant public after the second World War we refused lems and economic development possi- policy issue of the 1990s has been gov- to let history repeat itself. Rather the ernment's response to economic bilities, to the role of the U.S. in the global economy, to domestic workforce and other Western powers change to the cyclical changes we came together to shape a new era. The development, and to ways in which have come to accept and tolerate as an American economy was rebuilt. NATO government and the private sector can inevitable characteristic of a free mar- was established. We carried out the ket economy, and the economic restruc- work together to assist workers placed Marshall Flan at risk in a new economic environment. to reconstruct war-rav- turing that has been occurring in this aged nations abroad. New institutions period of international competition. Therefore we begin this section with were formed to foster peace and pros- The former has produced variations in insights drawn from an anthology of perity such as theUnited Nations,the employment. Frequently the latter has President Clinton's speeches prepared for International Monetary Fund,theWorld resulted in permanent job loss and users by the White House Press Office, Bank,and theGeneral Agreementon worker dislocation. excerpting from that anthology and corn- Tariffs and Trade. I.M.01011.101=INIMMIL 2 10 Evaluation Forum 1031.10 11 Summer 1995 Features Ennnid: Chang3 and Displacement.

A These actions, le President said, of change which economics, technolo- made in America were subject to com- helped to usher in foyer decades of robust gy and information flow have imposed. petition at home or abroad from foreign economic growth and collective =tufty. produeers and foreign providers of ser- When you live in a. time of change, vices. Whether we saw it or not, our Yet the Cold War was draining we he remarked, the only way to recover hari devoted trillions of dollars to it. daily lives were touched everywhere by your security and to broaden your hori- the flows of commerce that crossed na- Now the, Soviet Union had disintegrated. zon was to embrace change, adapt to it, The nuclear shadow was weeding with tional borders. Capital clearly had be- and move forward. The only way we come global. Services had become glo- Star? Jaadfl. Democratic governments could recover the fortunes of the mid- were emerging around the world. bal. Information had become global, dle class, so that people who worked This was his message. The President reminded his audience harder and smarter could at least pros- that at the end of World War H an un- per more, he believed, was to adapt to His position was that the economy challenged Ameelea was protected by the many changes that were occurring. was now so globalized and change so institutionalized that no government of the oceans and by our technological su- He encouraged the audience to rec- periority. We chose, then, he said, to any nation could promise to protect opize how far we haa moved from an people from the changes in the world help create a watid of free nside sup- industrial age built on gears and aweat ported by institutions that wonld faa- economy. Tio if change WU inevitable, to embrace an information age demand- we had to help people view change as tate it. Global tralle subsequently grew ing skills, learning and flexibility. In from $200 billion in 1950 to $800 bil- our friend rather than our enemy, a this process, record numbers of Ameri- source of security rather than a source lion in 1980. As a result, jobs were cre- cans were succeeding in the new global ated. That decision to create a ustem of insecurity. And we in.ad to do that his a ecenomy. The economy had generated way that promoted those institutions of of exoanded, freer global trade played a aimost six miilion new jobs since 1993. major role, he helieved, in producing soziety that were most important to us, We had the lowest combioed rate of principally our families and communities. the prosperity experkneed by the U.b. unemployment end in twenty- middle class after World War IL five years.usinesses were more pro- The changing nature of work, he This new period of history repre- ductive. We had brought the deficit said, was placing enonnous demands sented a world America helped bring down to expand trade and give citizens on working people. The average work- into being, he suggested through the more of the tools they needed to obtain er today in every kind oi work had to period of the Civil War, two world an education and to rebuild their com- be able to work with more information, wars, the Cold War, the establishment munities. to be more creative, to solve more of global economic and trading mis- problems on his or her own initiative. sions, and the efforts to build a family Workers, Families a.%1 Therefore, the President said, we had to of nations and establish other instru- Communities In tho transfer more responsibility to workers ments of peace and harmony, of New Economic Era at the grass roots level. They had to progress and democracy. learn more skills, and process and ana- However, the President also stated lyze more information than ever before. But the President also acknowledged in his remarks at the NAFTA signing Since every American had to face these that while the nation as a whole had en- that Americans were continuing to raise forces, and every American family aid, central questions about our place and joyed peace and prosperity in this era, he felt the job of government ought to too many people were working harder our prospects in this new world we had be to empower people to make the most for less. While businesses were restruc- done so much to produce. They were of such forces. asking questions such as "Will we and turing and growing more productive our children really have good jobs, and competitive, too many people still When the family is under economic first-class opportunities, a world-class could not be sure they had a job next stress, he proposed that workers within education, quality affordable health care, year or even next month. And far more families either had to learn more and safe streets?" After having Dilly defend- than our material riches were threat- become more productive, get a better ed freedom, he realized, they want to ened, he insisted our children, our job, or work harder for less, trying to know if they will share its bounty. families, our values were at risk. The be as sturdy and resilient as the times common bonds of community which are. The latter was what had happened For the last two decades, he said, the had been the great strength of our to millions of American families for the middle class that was created and en- country from its beginning had become last twenty years, he said working larged by these policies had been under badly frayed. harder for less. The average working stress due to changes in the global en- family was spending more hours at Speaking at American University in vironment, the growth of technology, work today than twenty-five years ago, the winter of 1993, the President point- and increasing economic competition. for about the same hourly wages, ad- ed out that we were woven now inextri- If we had learned anything from the justed for inflation. collapse of the Berlin Wall and the fall cably within the fabric of a global of the governments in Eastern Europe economy. Imports and exports, which The President saw 1973 as the wa- and the former Soviet Union, he com- accounted for about one in ten dollars tershed year in American life. From the mented, it was that even a totally con- now represented one dollar in every end of World War II until 1973 family trolled society cannot resist the winds five, Nearly three-quarters of the things income doubled. We lived in an era of ftwnswwwwe lorwomftwoomea Evaluation Forum ilane 11 Summer 1995 1 1 A.. ftsifeeneete:

Ratures Emonornic Chant e and Dielacement

prosperity that we almost came to take And into that vacuum had rushed job loss. Businesses do not choose lo- for granted. The middle class grow ever gangs, drugs and violence. cations based solely on wages, he said. larger and more secure. People just as- They choose them on the basis of the sumed that they could obtain jobs they Acknowledging that it was difficult would hold for a lifetime, that they to maintain a sense of security and opti- skills and productivity of workforces, would always do better every year than mism that a country like ours needed in on government attitudes, on roads and they had the year before, that they order to lead the world into the future, railroads to deliver products, on the availability of markets for those prod- would be able to afford to send their he identified a plethora of needs: ucts. Those locations were our strength, children to college, have a comfortable The need for pro-family policies, he believed. , own their own house, and like the family and medical leave take care of their parents. But the reali- policy enacted into law. Some of the specific provisions in ty was, he said, that since 1973 most NAFTA, he said, were important in that Americans had worked increasingly The need to pass humane welfarere- respect for example, provisions that harder for the same or lower incomes. form that encouraged people to be removed some of the current incentives for people to move their jobs He had other disturbing information successful parents and workers. across the border, making it harder than before for to convey. Between 1972 and 1992, The need for a national network of while the work year became longer for businesses to relocate solely on the ba- manufacturing extension centers to sis of very low wages or lax environ- most Americans, wages stagnated. The help small firms accomodate tonew mental rules. The environmentalagree- 75% of people who did not have col- challenges and acquire new technol- lege degrees felt it profoundly. Those ment applied trade sanctions against ogies. who began but did not finish college any of the countries that failed to en- force their own envirenmental laws. saw their wages fall by 9% just since The need for telecommunicationsre- 1979. For those who did not go The agreements also ensured that Mex- on tn foems that would help the U.S. de- ico would enforce its laws in areas such college, wages fell l';%. And although velop the information superhighway. he pointed out that the U.S. traditional- as worker health and safety, child labor and the . in addition, ly generates new jobs through interna- The need to reform U.S. job training they protected our industries against tional trade, he also recognized that programs, especially the unemploy- unforeseen surges in exports from ei- some were lost each year to competi- ment system. The President remind- ther one of our trading partners. tion from countries with lowerwages ed Ns audience that eighty percent or higher quality, or sometimes unfair of the people losing their jobs today In his view, we had now createda trade practices. do not get called back to their old trade agreement that moved beyond the traditional notions of free trade, putting Median family income today, he jobs. We had become locked intoa the environment and human resource added, was only $1,000 higher than it 1950's system when we needed one considerations at the center of this and was twenty years ago. And more signif- for the 21st century that encouraged icant, the growth in future trade agreements. At thesame income inequality continuous retraining and place- time, the President felt such trade between those who were educated and ment in the workforce. agreements imposed new obligations those who were not had escalated dra- on American government. Government matically. Even though there were 50% International Trade was obligated to make sure that U.S. more people in the work force of mi- and the Economy nority origin with four years of college workers were the best prepared and trained in the world. or more, the aggregate racial gaps in in- The President also commented that come were deeper because the educa- open and competitive commerce would We now know, he said, that theav- tion gap had grown so great and be- enrich us, spur us to innovate, forceus erage 18-year-old American will cause inequalities in income had to compete, connect us with new cus- change jobs up to eight times ina life- escalated in the last several years. Once tomers. His perspective was that trade time. Over the past ten years, for the Americans looked forward to doubling promotes global growth without which first time in history workers losing their their living standards every 25years. no rich country could hope to grow jobs often found they could not return At present productivity rates, it would wealthier. It enabled our producers to to their old job and had to find a new take 100 years to double living stan- prosper, who were themselves consum- occupational niche. Consequentlygov- dards, he predicted until our grand- ers of services and raw materials. ernment needed to developa new reem- children's grandchildren are born. He understood that many Americans ployment systemto replace the tradi- Our society, he said, had suffered were worried that expanded trade tional unemployment system. We had to be able to tell that subset of workers unbelievable stresses, as broken homes would move jobs out of the U.S. be- who became dislocated by moreopen and unwed mothers had becomemore cause of the difference in wage rates. trade policies that there was a job train- commonplace. In places devastated by They had seen this happen. But the President pointed to the eighteen out of ing and reemployment program avail- poverty and despair, we had seen the able to them. absolute collapse of families and work nineteen serious economic studies by itself, and of the sense of community. both liberals and conservatives that had Once we had put our own economic concluded that there would be no net house in order, he said, we must make

Evaluation Forum 12 Issus 11 Summar 1995 Features Economic Changernent

ing business beyond our borders. For the first time in a decade we had trade a priority element of American nine months of manufacturing job security, part of an integrated economic Increase our investment in education growth. For the first time in nine years best to program. Then we must do our and training and in technology the annual vote of international econo- exercise leadership among the major fi- and defense conversion. mists was that the U.S., not Japan, was nancial powers to improve our coordi- the most productive economy in the nation on behalf of global economic NI Bring the benefits of free enterprise to world. growth. areas which have been isolatedfrom it, in our inner cities and rural areas. But the President also believed we Also, he proposed, we must promote were dealing with some long-term eco- the steady expansion of growth in the Make the federal government more nomic problems. The average hourly developing world, not only because it efficient and effective. peaked about it would wage in the U.S. actually was in our interest but because twenty years ago, and working people help these countries as well. These di- Implementing these strategies, he believed, would require a much more had been losing their rections, he said, constituted an agenda steadily for about ten -tars, the only aggressive, innovative partnership with for American action in a global econo- advanced country in tla world where the private sector. Government does my, as well as for our ownprosperity. that was the case. So there were genu- not have the capacity to "save" theU.S. Therefore the best investment we could ine reasons why a lot of hard-working economy in an economic environment make, he was convinced, was in the Americans did not feel more secure or one resource rooted within our own that is global in scope, he said. But gov- ernment can create the right climate, the happier with the good statistics and borders in the education, skills, rea- growth rates. They were still not sure right conditions, and can empower peo- soning capacio, and creativiO, of Amer- that these numbers guaranteed them a ple so that they can compete by taking re- icans. good future and a good job, the ability sponsibility for themselves and their fam- to keep their children's health insur- American Economic Progress ilies. He felt that increasing changes in ance or put aside money for theircol- In February, 1993, speaking to a the world made this imperative. lege education. They were not sure that Joint Session of Congress, the Presi- The President indicated we were not the society was going to be able to dent explained that a number of condi- alone in facing economic problems in solve the social problems that violated tions over the 1980s had brought us to this new era. Every advanced country our values and conscience. a point of decision: twodecades of low was experiencing a commonphenome- And we had growing poverty. We productivity, low growth and stagnant non: even as economies grew,there had to find a way, through both public wages; persistent unemploymentand were pockets within them thathad not and private sources, to enhance the ; years of huge gov- been benefited by economic recovery, ernment deficits and declining investment where investment had not taken hold, lives of the American people who lived in poverty, by bringing investment and in our future; exploding health care costs where jobs had not materialized. This jobs to poverty areas. We could not, he and lack of coverage for millions of was particularly true for innercities said, repair the American family until Americans; legions ofpoor children; and and isolated rural areas.The first thing we provided the structure, thevalues, education and job training opportunities we needed to do in the U.S., he sug- the discipline and the reward that work inadequate to the demands of an unusu- gested, was to change the job mix, in- ally competitive global economy. creasing the proportion of high-skill, provided. high-wage jobs.'And we needed to pro- We needed to recognize also, he The economic plan he recommended vide special incentives for people to in- said, that well over half the women for turning this around involved four fundamental components: a shift of em- vest in isolated an-as, using empower- who were mothers in this country were ment zones, the enterprise community in the work force. We had to decide phasis from consumption to invest- ment; honoring work and family in ev- concept. that as long as the economy mandated decisionmaking; In 1994, according to the President, this, and the economic pressures of the ery part of our public time did, we had to find ways for peo- reducing the federal deficit honestly we fount: ourselves in themidst of the and credibly, and earning the trust of first investment-led, low-inflation, pro- ple to be both successful parents and successful workers, and ways of orga- the Ameru..an people by paying for this ductivity-driven economic expansion in nizing ourselves differently with regard with cuts in government waste and in- three decades. Something fundamental to , health care, family leave, efficiency, in government spending, was beginning to happen, he was con- and with fairness to those bearing the more than half the new jobs and tilt income of people who had chil- vinced dren and who worked but still did not burdens of these changes. created by our economy in 1994 were above the average wage. This was more make enough money to support them. He felt the key elements of this ap- high-wage jobs in one year than in the proach were these: Education and Workforce previous five combined. Investment in Preparation in a Postindustrial Reduce the deficit. what it new equipment was eight times Economy Expand trade and intensify U.S. had been in the previous four years. For the first time since 1979 the U.S. led Speaking at the University of North government efforts to become a the world in the sale of automobiles. Carolina in 1993, the President pro- partner with the private sector in do- Evaluation Forum Issue 11 Summit 1995 13 ---7-111

Features Economic Change and Displacement

posed that economic security couldno with a common love for thiscountry longer be found in a particular job. It Goals 2000 set national skill standards and the values of freedom and mutual as well, to ensure that our workers were must be rooted in a continuing capacity respect. He felt we had to find a way in to learn new things. That condition re- better trained. It gave states the flexi- _ this age and time to restore that kind of bility to meet educational and skill quired a system of life-long learning, discourse and that kind of respect. We beginning with higher standards for standards according to their own needs could not afford to engage in the citi- and ideas, stimulating well-plannedex- schools. Almost two decadesago we zenship of division, distraction and de- began education reform efforts that in- perimentation. He felt strongly that in- struction. We had a future to build, novations needed to be encouraged and cluded higher standards, he said. Now through education. we were trying to adopt a whole new rewarded, evaluated for their efficiency approach in a national effort to raise The President had a special opportu- and effectiveness, and if successful educational standards. This, he felt, nity to talk about the meaning of edu- adapted for use elsewhere. This would was an American imperative. At the cation for American young people ata be a departure from our past There had never been any national standards. same time he believed we must insure junior high school in 1994. Heex- There had never been a formal that every young person had theoppor- plained that the Goals 2000 proposal way to tunity to obtain a college education. embodied ideas gleaned frommany measure whether or not educational and skills acquisition goals were being And for the three-fourths ofyoung peo- years of work on educational issues. It met. ple who did not obtain a four-year col- established what kind of educationevery In the past, he said, everyone who lege degree, we needed to merge the child needed in every school in America. was willing to work could fmd work. world of learning with the world of It set out some goals that were expected Today, more than ever before, whether work, to offer young people classroom to guarantee that young people would be you had a job or not, how much you and on-the-job training. prepared to compete successfully in the could earn in a job, and whatyour fu- 21st century wherever they were, and In a talk at Rutgers University in ture was, depended on how muchyou whether they came from poor, middle could learn not just what you knew 1993, the President reminded the audi- class, or wealthy families. ence that in the midst of World War II but how much you could learn. Re- President Roosevelt proposed the GI One of those goals for theyear 2000 search was indicating that people who Bill of Rights, which offered returning was for every school in America to of- graduated from college made twiceas veterans the opportunity for education fer a disciplined environment condu- much as those who did not. The world, in return for their service to our country cive to learning. Another was to in- he said, was increasingly smaller, changing very fast. People in war. Thanks to the GI Bill,more crease the high school graduation rate were con- by 90% nected financially and by communica- than eight million veterans receivedan an international standard. advanced education. Half a century lat- Still another was to make sure thatev- tion networks that were unheard ofear- er, the enduring legacy of the GI Bill ery adult possessed the knowledge and lier. Therefore we had to be ableto was the strongest economy in the skills needed to obtain and retaina learn new things continually. good job, a job as good as people had world, and the broadest, biggest middle The President's perspectiveon our class that any nation had enjoyed. in oth..:r postindustrial countries. future was that we must view ourselves The national service plan, he point- At the signing ceremony for the more as an extended family, a large ed out, was constructed on thesame Goals 2000: Educate America Act in community, and look at all the people principles, namely that when people March of 1994, the President remarked in this society as anenormous resource. give something of invaluable merit to that this legislation set world-class edu- If we did so, we would reinvigorateour their country they ought to be rewarded cational standards for whatevery child educational and training systems. He with the opportunity to further their ed- in every American school should know said we needed to make sure thatour movement into the 21st century ucation. The national service planen- for reading and writing, math and was abled American students to borrow the science, history, geography, foreign grounded in an old idea, that all Ameri- money they needed for college and pay languages, civics and economics, and cans have not only a right but a respon- it back as a small percentage of their the arts. It also said something about sibility to rise as far as their talents and own income over time. The President teachers. We had to prepare them bet- determination can take them, andto considered this especially important af- ter, enable them to continue to learn. give something back to their communi- ter a decade in which the cost of a col- And it said something aboutparents. In ties and their country. We neededa new set of understandings, he thought, lege education had goneup even more a world in which families were under rapidly than the cost of healthcare. increasing stress, the President saidwe about how we can equip peopleto meet must find ways to help them become the challenges of thenew economy, At the 75th anniversary of the more involved in their children's edu- how we can change theway govern- founding of UCLA in May, 1994, the cation. And the legislation acknowl- ment works to fit a different time, and President asked graduating seniorsto edged that we needed more research and how to repair the damaged bonds inso- look back at the history of the Constitu- innovation so we could generalize theuse ciety. The future was notan inherit- tional Convention. To look at howpeo- of successful models and strategies. ance, he said. Rather it was an opportu- ple assembled wildly different points of nity and an obligation, somethingevery view and argued heatedly, but always Besides academic standards, the generation had to create. President reminded the audience that

Evaluation Forum aIssue 11 Summer 1995 Features Economic Chen e and Us lacement

II_ and addressing the problems of disad- technology and service jobs of the Understanding and vantaged workers, the structurally un- future. Adjusting to employed, and those experiencing dis- Changes in our approach to poverty, crimination in the labor market. Economic Change given growing income inequality. There was a significant expansion of These changes provide the context Economists' perspectives on the programs to train and retrainworkers, for the perspectives of other economists economy, how it works, how it is increase UI benefits, create public ser- in Rethinking Employment Policy, changing, differ with their theoretical vice jobs, and ensure equal opportuni- ty. Even though the problems that training and orientation. Although this Ray Marshall's Perspective intellectual attribute confuses us, it also prompted this policy change have not expands our minds. In this article we faded away, the times have changed, Former Secretary of Labor, Ray look at perspectives embodied in sever- they point out. In the 1980s, the combi- Marshall, concentrated his chapter on al chapters of a book published by the nation of a conservative president, fis- "The Implications of Internationaliza- Urban Institute, Rethinking Employ- cal pressures to cut budgets, and nega- tion for Labor Market Institutions and ment Policy, edited by D. Lee Bawden tive public perceptions of certain Industrial Relations Systems." It is and Felicity Skidmore. The book is one programs shriveled employment policy. Marshall's thesis that the international- in a series of publications sponsored by The editors predict prophetically in this ization of economies and the globaliza- the Institute's Project on the Federal introduction that the American public tion of markets between 1950 and 1980 Social Role funded by a number of pri- is not likely to support a more aggres- have been the most important economic vate foundations. Although the book sive and expansive policy, if based on trends since World War II. He supports was written ha 1989, prior toadditional the full array of past strategies. this contention with adequate evidence. changes in the American economy and Meanwhile, they insist that some of By 1987, he says, over half of U.S. cor- the role of the federal government, its the troublesome problems of the past porate profits derived from overseas. International trade accounted for one- insights nevertheless apply usefully to remain to be solved. In fact there is the 1990s. third of all U.S. cropland, one-fourth of more concentrated structural unem- farm income, and one-sixth of all jobs. ployment, expanded underemployment, The book takes a retrospective look Nearly 70% of all goods manufactured at what is known and remains to be un- an increase in worker dislocation, and continuing discrimination in the work- in the U.S. was competing with im- derstood about dealing with domestic ports. Close to 25% of the growth in and international issues facing the U.S. force. They propose that the solutions the U.S. workforce during the 1970s in- to these problems must be crafted in the labor market in this decade. The chal- volved immigrants and refugees. lenge the editors pose at the beginning context of important social, economic of the book is to find a method for and demographic changes that have These changes, he says, have altered 1) maximizing the potential of our taken place since the 1960s, and must the effectiveness of traditional macro- workforce without creating inflation, take advantage of what has been economic policies. The integration of and 2) targeting job training and em- learned over the past several decades. markets globally has transformed labor ployment programs to those who can The changes they claim we must re- market institutions and industrial rela- benefit the most from them, so that our spond to are these: tions systems, as well as influencing resources are used efficiently. Changes in the composition of the management practices. Bawden and Skidmore identify two workforce that is, more older The Evolution of Economies major themes that guide the book. One workers and women requiring the Since World War II is that U.S. employment policy has not adjustment of retirement policies, Marshall exelains that after the met the goals envisioned in the early wage structures, working hours, and WWII, governments viewed an open, 1960s as achievable. The other is that services such as child care. we now know enough from analyses expanding trading system as the best and research that shaping a more effec- Changes in and way to encourage the growth of world five future employment policy is possi- policies respon- economies. Many experts had linked ble and should be done. sive to less overt, more subtle forms protectionism and trade restrictions of racial and sexual discrimination. with the and the war Employment Policy History itself. As a preventive measure, the Changes in our educational system, U.S. and other countries worked to- In their introductory chapter, the ed- given that over one-fourth of the gether to form institutions, policies and itors suggest that the employment and adult population lacks a high school procedures to encourage and maintain training policies of the 1960s and early international trade and finance. This 1970s, which represented a policy education, and may lack the basic skills that support training and re- free-trade system became known as the change as well as increased funding, Bretton Woods system. generated a new interest in identifying training for the expanding high-

7 EvaluationF0f1M7 ISMti Summer 1995 Features Economic Chan e and Dis lauement

It was organized a.ound three basic accompanying this development. Inter- ment of the American economy that principles: I) free trade, 2) fixed ex- national economic rules that worked for business and industry's ability to com- change rates, and 3) autonomy in do- the 1950s and 1960s no longer applied. pete successfully in world markets had mestic economic policy. The world's Countries are not as effective in di- been declining for over twenty years, welfare was considered to be increased even in high technology sectors. Con- by a ccmpetitive, free-trade, open-mar- recting their policies concerning the in- ternational economy, Marshall says, clusions from other analyses elaborated ket system in which each nation would on this assessment. Marshall cites two concentrate on producing things which and the effects of domestic economic policies have changed. The impact of studies in the mid-1980s by the Brook- would give it the greatest or the least ings Institution and the New York advantage economically. This idea was economic stimuli in a period of eco- nomic depression, for example, limits Stock Exchange that reach cther con- referred to as "the doctrine of compara- clusions. These were the major find- tive advantage." domestic economic expansion by in- creasing foreign imports a serious ings: Alongside this doctrine, a critical in- problem for the U.S. during the 1981- An overvalued dollar, due to huge fluence was the power of the U.S., ac- 1982 when foreign imports U.S. budget deficits, is the main cording to Marshall. Our domestic cut into increased demand. And the cause of our economic problems. economy had been strengthened by consequences of U.S. budget deficits Given this scenario, the analysts World War II, having benefited from are not under as much control. This technological development during the lowers U.S. living standards and can concluded that the economic boom war, which supported an enormous raise inflation, he says. It also forces of the 1950s and 1960s was "abnor- growth in productivity and total output. other countries to keep their interest mal" and the economy would return This economic dominance translated to rates high. These destabilizing effects to "normal" as soon as the budget policy dominance. It was therefore in have brought pressure on policymakers was balanced. our national interest to advocate for a to reconsider the Bretton Woods sys- relatively open and expanding world tem. Trade was the factor most responsi- economy. ble for a net increase in jobs in the The reality has strayed from this sys- 1970s we gained mc-1 employ- But even as the Bretton Woods sys- tem's theoretical underpinnings. Com- ment from trade than we lost to it. tem encouraged growth in the interna- petition, Marshal says, does not direct tional economy through the 1960s, cer- domestic markets, therefore countries Trade was not the main reason for tain events were to begin to tear it do experience unemployment. There- declining employment even in the apart, Marshall explains. These were fore many industrial countries have industries experiencing the greatest some of those events: now supported strategies that both cre- declines. ate comparative advantage and improve The abandonment of the fixed ex- U.S. industrial performance was change rate system. the industry mix. And Marshall claims that economic activity is now basedon strong relative to other countries The massive increase in foreign in- national and enterprise strategies, not first or second in the world in 23 out vestment and related flows of capi- simply on the interaction between of 40 manufacturing industries in taL short-run market forces and the maxi- 1982. mization of profits. The growth of the Eurodollar market Most job losses were concentrated in (dollars held and traded abroad). Japan, for example, is more interest- four key depressed industries which ed in strengthening national power, A large increase in U.S. debt and the accounted for only 20% of total productive capacity, and market share, manufacturing output. The 36 indus- debt of non-oil-producing develop- he says, than in "short-run profit maxi- tries that dominated manufacturing ing nations. mizing." Also, economic change is not gradual, Marshall points out. Trade is actually added jobs between 1977 Large-scale movements of workers no longer limited to "goods at the mar- and 1982. among countries in response to em- gin." Whole technologies are exported, ployment and income opportunities. The growth in productivity had been and countries share the production of slowing sin6 the 1960s, but in man- products. The result, he says, is poten- ufacturing it had been particularly Some Implications of tial job loss, wage suppression, and less strong. Internationalization satisfactory working conditions. Marshall sees both positives and In this generally more optimistic negatives in this internationalization of The Potential Loss of the perspective, poor production perfor- markets. The positives are increased ef- U.S. "Competitive Edge" mance was seen to be correlated with ficiency and expanding knowledge, Marshall reports that the President's energy price shocks, inflation and eco- which raised living standards. The neg- 1985 Commission on Industrial Com- nomic uncertainty, none of which were atives are the destablizing influences petitiveness conveyed a dour assess- thought to interfere with performance.

Evaluation Forum Issue 11 Summer 195 16 Features Econprement

Marshall's Viewpoint ed macroeconomic policies and strate- The analysts pointed to the expansion gies that avoid over-valuing thedollar. of real plant and equipment spending, Marshall draws from both perspec- And finally we need to reform our in- increased funds for research and devel- tives, acknowledging that the U.S. is dustrial relations system, to improve opment, increased efficiency due tothe still the strongest world economy and economic stability and employment and accumulation of work experience by likely has the highest average produc- reduce unemployment. baby boomers, a better educated work- tivity levels, even though we appear to force, and a significant improvement in be- Most of the features of the Bretton be losing that advantage rapidly. He all labor-management relations tied to the lieves that macroeconomic policies and Woods system were characteristic of need to become more competitive. the expensive dollar are important fac- industrial nations, and some of these tors. But he thinks the pessimists are features persist. However, Marshall But there was a remaining residueof have correct in being concerned aboutmain- teases out certain elements that pessimism, exemplified in the Harvard been unique to the U.S. and must be Business School's analysis. The main taining market positions "without sacri- ficing return on assets, or real wages or considered in developing 3 new indus- conclusions, according to Marshall, trial relations system: were the following: incomes" that is, he feels our living standards are going down. His position American unions were considered A more realistically-valued dollar is is that analysts are on target in taking the bargaining agent for all employ- not the answer, because the structure seriously how we are doing vis-a-vis ees, whether union members or not. of trade has changed. The economic countries like Japan. Productivity However, the legal right of workers recovery from the 1981-1982 reces- growth is the key to maintaining com- member- petitiveness, in Marshall's mind, and to vote for or against union sion has led to slower growth in ex- ship created competition for em- ports and faster growth in imports thereforefor sustaining high wages and profits. ployees' loyalty betv, een union and than in previous recoveries. The re- non-union sectors, and between failed to turn of business assets has He also sees the composition of in- unions and employers. keep pace with the rising cost of dustry as an important influence. Stud- capital. ies that treat all industries the same are Collective bargaining was decentral- missing critical information, he says ized, emphasizing wages, hours and Energy prices were an important "there are dynamic and symbiotic rela- working conditions in particular factor in trade deficits in the late tionships between production activities firms, industries and labor markets. 1970s and 1980s, but there was also and technology transfer." Also, he re- a decline in non-energytrade bal- minds us that the linkages between eco- U.S. employers were generally hos- ances as well. nomic activities and technology are cu- tile to unions, despite the affinity of mulative. A lack of competitiveness in the American labor movement for Market share in constant dollars is key industrial sectors may lead to their the capitalist system and its flexibili- not the only, or the mostsignificant loss, which then removes from remain- ty in responding to the interestsof measure of economicperformance. ing businesses the technological sup- employers. Performance is determined also by ports, the learning accumulated, andthe the relative values of exports and networks developed that are required A "scientific management system" imports. Our share of total world ex- for them to stay competitive. was used by employerswhich was ports has declined. generally authoritarian and adversar- Marshall's Recommendations ial, allowed only minimal participa- The U.S.'s share of high-technol- Having established that the Bretton tion in decisionmaking by workers, ogy exports has fallenalso. Woods system no longer works in the and provided minimal job security. U.S. productivity growth has postindustrial era, what does Marshall Union politics were not class-con- suggest? One answer, he says, is to re- been far below our competitors, scious, rather they focused on jobs strict the growth of the international even though their economies and economic security rather than on trading system an answer he rejects have remained depressed. political base out of hand.Trade restrictions,he says, an organizational or The pessimists have discounted can worsen economicproblems be- for the working class. indicat- many of the optimists' tenets, cause it produces retaliatoryaction. Management felt they had a right to ing that the evidence shows that in manufacturing, productivity growth has The right answer, in Marshall's lay off workers during economic deteriorated "despite improvements in view, is to promote an open and ex- dowaturns since the expectation was both capital formation and capital-labor panding trading system guided by 1) that business cycles would keep lay- internationally acceptable rules that can costs." offs temporary, and the UI system be enfnrced, 2) international economic would provide adequate temporary policies, and 3) international economic assistance, institutions. Also needed are coordinat-

9 Evaluation Forum Issue 11 Summer 1995 17 Features Economic Chan e and Dis lacement

The American system was more Mechanisms within firms that help economists, is generally supported by flexible than most European indus- absorb demand shocks, such as bo- the Administration's economic and in- trial relations systems in adjustingto nus compensation systems, sharing dustrial relations system reform ideas, change, due to: of production with Third World and has been taken seriously by the The principle of exclusive bar- countries, subcontracting, and the larger corporations. However, the im- gaining. use of temporary workers. pact of these ideas on reducing disloca- tion has yet to be determined. The more decentralized bargain- The ability to shift resources from ing system. non-competitive to more competi- Anothv Perspective: Goals for tive industries. Nationa.( Employment Policy The lower degree of unionization. la An emphasis on lifetime employ- Isabell V. Sawhill's perspectiveon The greater ease with which em- ment, , the con- economic issues is also enlightening. ployers could close plants and lay centration of collective bargaining at She defines four goals for employment off workers. policy: the enterprise level rather than at the The greater openess to immigra- industry or sectoral level, and the The provision of jobs i.e. the re- tion and imports. annual adjustment of wages (pre- duction of unemployment. cluding internal competition The greater displacement of la- on the The creation of more good jobs part of employers and unions.) bor, resulting in pools of under- i.e. the improvement of productivity employed workers. Heavy emphasis on consensus- and earnings. building at every level, rather than These elements supported a longpe- The provision of assistance to the on detailed regulations. riod of relatively high growth inpro- disadvantaged i.e. making the duction and output, and movedmost A bonus wage payment system that poor more self-sufficient. union members into the middle class. keeps wage increases in bounds. But the system tended to be inflation- The improvement of the functioning ary. The bargaining approach tended to A flexible job assignment and train- of labor markets using means that raise wages by playing one employeror ing process that increases workers' guarantee economic efficiency and union against another. Long-term union job security and employers' willing- the fair treatment of workers. contracts with cost-of-living adjust- ness to finance long-term education She views these goals as serving ments and annual increases in benefits and training for their employees. tended to inflate the compensation dual purposes: human welfare in soci- base. UI and other income maintenance But Marshall does not leaveus here. ec, eni the economy's long-term programs reduced the impact of labor He points out that the U.S. has higher growth and productivity. The strategies she recommends for achieving the four supply on wages, just as multiplewage- overall productivity and living stan- earner families did. dards than Japan, and some sectors of goals are these: 1) macroeconomic pol- policies reduced employers' incentives the Japanese economy, hesays, are not icies; 2) job creation policies, suchas to avoid wage increases and unions' particularly productive or efficient. subsidized employment in the public and private sector; 3) education; 4) in- willingness to hold them down. Asa Nevertheless, the Japanese systempro- comd transfers; and 5) government result, Marshall says, insufficientatten- vides rising real incomes and much reg- tion was focused on the level ofpro- more job security. And it is more egali- ulation. She considers the public educa- ductivity and the degree of economic tarian in terms of income differentials. tion system the strategy of choice for efficiency. Beyond that, Japanese workerssee a creating better jobs through improved strong relationship between improve- productivity. In providing jobs overall, Marshall looks to the Japanesesys- ments in productivity and the employ- macroeconomic policies are primary, tem for answers. These are the major ment benefits they receive, whichgen- she believes. Income transfersare im- characteristics that capture his atten- erates a considerable incentive. The portant in assisting the disadvantaged, tion: cooperation between public and private even though concern about work disin- Highly integrated, consensus-based sectors makes it possible for businesses centives and long-term dependencere- main. economic policies emphasizingup- to be satisfied with lower rates ofre- grading of the Japanese industry turn, which then encourages long-term With respect to the goal of improv- mix. planning. ing the functioning of labor markets, Marshall's diagnosis of our prob- she gives attention to worker protec- 31 A management system supportive of lems and possibilities has garnered tions and benefits that is, to protec- labor-management cooperation and wide support from other respected tive labor laws; anti-discrimination participation. laws; job security; retirement, dzsability and unemployment insurance;

=s1W 10 .11011111 18 Evaluation Forum Issue 11 Summer 1995 Features. Econ2Lnis la ice ent

11111 rights; and notification of layoffs. More jobs can be created if Americans A Challenge in While the protections and benefits are would accept lower wages, she points of critical importance to workers, she out. Alternatively, wage levels couldbe Restructuring: says, they may also reduce economic increased at the risk of greater unem- Efficiency Vs. Equity efficiency. Employers find it more dif- ployment. The choice is more disturb- ficult to adjust to changing market de- ing, she suggests, in an economy that is In this article we review ideas in two mand. This is the perennial tension be- vulnerable to increased foreign compe- books whose topics are relevant to the tween equity and efficiency that tition, since our standard of living can perennial dilemma posed by efforts to economists have been struggling with be greatly influenced by that choice. increase economic efficiency without since Adam Smith. We could, she says, compensate for jeopardizing economic equity. Resolv- choosing higher wages through a com- ing this dilemma has important impli- Resolving Problems bination of 1) trade protectionism, 2) a cations for worker dislocation, since in Meeting Goals stimulative fiscal and monetary policy dislocation is frequently a product of that reemployed displaced workers, or What Sawhill suggests is that we find efforts to increase economic efficiency 3) a decline in the value of the dollar but often leads to increased income dis- more flexible and less costlymethods but these solutions would; she be- parity. for resolving the tension, weighing lieves, lead to an erosion of the pur- costs vs. benefits more carefully. The chasing power of higher wages and de- Both books are Brookings Institu- tion publications. We first give atten- current labor market and industrial re- tericnate the standard of living. lations system, Sawhill says, "evolved tion to Growth With Equiv Economic as a requirement of a modern economy Whatever the strategy, Sawhill feels Policymaking for the .Next Century, built on the principles of specialization we must make special efforts toassist written by Martin N. Baily, Gary Burt- .t and large-scale production." These the disadvantaged. Her perspective on less, and Robert E. Litan in 1993. And princi)les served us well, she recogniz- programs to reduce youth unemploy- then we turn to A Future of Lousy es, greatly increasing productivityand ment and welfare dependency is that Jobs? The Changing Structure of U.S. Amenca's standard of living. adverse economic and demographic Wages, edited by Gary I3urtless in forces are more responsible for slow 1990. However, little attention was paid in progress than the programs developed this system to workers' rights and pro- for that purpose and that some of Themes in Growth With Equity tections, which led to unionization and these programs have been, in fact, quite The authors' position is that the to an array of protections and benefits. successful. And the industrial system that initially American economy exhibits two worri- in pro- gave us prosperity was ill-suited,she A Brief Commentary some trends: slow improvement claims, for a world in which the labor ductivity growth, and growing income force is educated, workers make de- These selected perspectivs on the inequality. Short-term fluctuations in mands, services and information re- economy provide important lessone the business cycle and in unemploy- place standardized goods, and capital about potential relationships between ment are not their primary concerns in and technology are widely available. economic policies, unemployment and this book it is long-term patterns These changes have created a climate dislocation, despite the differences that concern them: among labor economists in the nature for workplace reform in the direction of What will economic recovery mean? worker satisfaction and participation, of the evidence they interpret, their and new styles of corporate leadership. conclusions from this information, and How can we boost the rate of long- How successful this new direction will their recommendations for change. On term growth? be, Sawhill says, is yet to be estab- one point there is agreement among How can income disparitles be re- lished. them: it seems clear that the industrial relations system of the 1950s and duced? Some Conclusions 1960s is no longer viable in the 1990s These questions framc a fascinating and must be revised substantially in the One of the most difficult trade-offs book that provioes insights based on an context of postindustrial international increasingly large body of evidence in achieving the goals Sawhill proposes competition. is between employment and wages. about long-term economic performance and its implications for equity issues. The authors' approach, they want us to understand, is "eclectic" and "cau- tious." No single policy is envisioned as the magic bullet. They viewtheir recommendations as modest, in the sense that they are designed to increase the probability of achieving progress in resolving productivity and income ine- quality problems.

4a..mwa> forme, 11 Evaluation Forum Issue 11 Summer 1995 19 Features . Economic Chanand Dis4lacement

A premise is that appropriate poli- By faster growth, the authors mean How strung the evidence is from re- cies for increasing economic growth both a more rapid rate of advance in to- search and expert opinion in sup- and bringing greater equity to the dis- tal output and a faster rate of increase porting the wisdom of the proposal. tribution of income are inevitably inter- in output "holding constant the related. The economy, they say, can amount of inputs used to produce that Of these proposed policies, labor grow faster if the low-skilled and low- output." A labor force that works more policies are considered the most sup- incomed can produce more and higher intelligently to produce goods and de- portive of both growth and equity. quality goods and services per hour liver additional services with thesame The full complement of recommended worked. Skilled workers, they contend, amount of work effort is at an advan- policies is presented in Figure 1. have greater flexibility and mobility, tage. Advantage is achieved by raising The authors' proposals are designed which means they can move from de- labor productivity. clining to expanding industries with to speed economic change by increas- shorter periods of unemployment fol- Economic growth refers to the ing productivity growth, white easing lowing job loss. If unemployment is so growth ofpotential output, "the output the burdens of change for workers and reduced, the economy can be pushed to that can be attained when labor and their families. But the authors remain lower levels of unemployment without capital are fully employed and inflation humble about the impact of such pro- serious levels of inflation. At the same is not accelerating." Economic geowth posals, hoping for substantial improve- time, the problems we associate with is spurred, the authors say, by advances ments if not dramatic results. They are poverty will not be sapping the econo- in technology, by improvements in convinced it is time to develop a com- my if its resources as much. workers' skills, and by progressive prehensive set of interelated policies as management practices. When the au- well as for realism. Inaction, theysay, , With respect to income inequality, thors talk about income inequality, they is an unacceptable strategy. the authors state unequivocably that are referring to inequality in pre-tax in- their interest is not limited to strategies comes as well as after-tax incomes The issue of Earnings inequality that reduce poverty, rather they are in market incomes, particularly wages. Gary Burtless' edited volume, A concerned about changing the condi- tions that lead to low earned income In commenting on their comprehen- Future of Lousy Jobs? speaks directly among the least skilled. However, they sive policy model, they acknowledge to the equity issue. In the introductory recognize that some policies that speed that the research evidence does notsup- chapter, Burtless identifies the prob- lem. Over the last two decades, the long-run growth do not support equity, port all of their recommendations share of all income obtained by the and policies that are consistent with eq- equally well. But this does not deter uity do not necessarily increase eco- them. Using a medical analogy, they richest fifth of families in the U.S. has nomic growth. point out that one cannot always expect increased by 3.3% to 43.7%. The share existing scientific information topro- received by the poorest one-fifth has Their main contribution to thinking vide conclusive answers about alterna- fallen 1% to 4.6%. Incomes of the about these complex issues is a com- tive treatments, but this is hardly area- group of families just below the richest prehensive framework and strategy a son to abandon the search for evidence rose slightly, while the incomes of mix of policies and programs for en- or to avoid making policy suggestions. those groups of families just above the couraging long-term economic growth And so they do make them! poor fell. These shifts, Burtless con- while maintaining equity. This policy cludes, have reversed the trend over model is based on assumptions about An Agenda for Growth much of the postwar period, a trend the underlying causes of slow growth and Equity that saw this gap shrinking. and increased inequality, such as: .. Three broad themes run through the Causes of inequality The impact of oil price shocksover large set of proposals in the authors' the last two decades. model: increasing and reallocating in- Since the main source of income for most Americans is wages, one cause Reduced innovation. vestment; assisting workers; making hard choices;and implementing for rising income inequality is thought An increase in regulation. change. The proposals subsumed under to be the decline in middle-class jobs these themes are organized into several with high to moderate wages. Not all Deteriorating educational perfor- policy categories: innovation, labor, in- economists agree that this is the major mance. vestment, and trade and foreign invest- cause, but most agree that average Stronger foreign competition. ment policies. For each proposal in the earnings growth among production model, there are qualifications: workeis has stopped in recent years af- A stagnant minimum wage. ter rising strongly over the early post- w Whether or not the policy contrib- Declining unionization. war period. This sharp decline since utes to economic growth or to eco- 1973 is related, Burtless says, to the The model also depends on one's nomic equity, or to both. slowing of productivity growth, but definition of growth and productivity. How substantial the proposal's ben- again economists are not of one mind efits may be in resolving one or the about why this change has occurred. other, or both. IlliNIMPMINMONNIMR 12 Evaluation Forum Issue 11 Summer 1995 20 Features Economic Change and Displacement

tion or aggregatelevel of labor supply, One problem is defining and measuring among women Given thisresult, one could propose that middle-class jobs and 2) the changing pattern of demand productivity and wage inequality. For for U.S. workers. example, Burt less proposes that an ide- are declining for men, but notfor wom- al measure of worker compensation en. Composition and Demand Shifts would include both wages and fringe Between 1973 and 1987, the earn- benefits per hour or per forty-hour ings gaps between well-paid and mod- The rise in the number of poorly- workweek, but the Census does not col- erately-paid workers, and between paid jobsmightbe related to an in- lect enough information to calculate moderately-paid and poorly-paid work- crease in employers' use of lessskilled but Burtless says this is con- this. ers have grown over time,particularly labor trary to the evidence. If employers were Another way to measure inequality in the 1980s. The number of men with earnings close to the median wage level hiring less skilled workers, the wages of is to look at the distribution of wages these workers would be elevated relative and calculate the trend in earnings for has shrunk. Jobs near the middle of the male distribution have disappeared. to those received by better skilled, more workers at selected points in the earn- educated workers. However, the opposite ings distribution. This method indicates Burtless points to two reasons for these changes that have been explored by pattern occurred in the 1980s. Also, job- that inequality is rising among men but lessness among the less skilled should in- falling or remaining fairly stable economists: 1) shifts in the composi-

. _ Figure eeomm-ended py.the.Authdrs Innovation Policies penalties for paying di- , to provide peer review. Institute tax Strengthen basic science research: rectors in cash. Authorize oversight on the part of Fund precompetitive research. Reduce risks of innovation: the National Science Foundation. Promote commercial R&D by dou- bling the R&D tax credit. Adopt modest 'tort" reforms, ac- Re-examine the allocation of the companied by strengthened R&D Lengthen business time horizons: current federal budget. and enforcement by regulatory aStrengthen precompetitive R&D: Encourage equity ownership by agencies. Fund only projects with private bankholding companies. Continue to allow joint R&D. sector financial support. Institute tax penalties on executive Fund a broad diversity of projects. director stock options with short ex- Create a Civilian Technology Board ercise periods.

Labor Policies Corps and JTPA. Improve skill training of schools. Establish public-private labor au- Encourage firms to link worker pay Encourage schools to provide high thority to develop tests and creden- tials for non-college occupations. with division of company-level perfor- school transcripts to prospective mance. employers. Institute pay-or-play training re- quirement for non-college-educated Cushion economic shocks: Require national standardized workers. Upgrade national and state employ- tests. Tie compensation and promotion of Establish national ment services. program for non-college-educated teachers and principals to perfor- Establish a non-cause-related pro- mance of students on national workers. gram of earnings insurance. tests. Establish improved R&D programs for such public training efforts as Job Improve skills of current labor force:

Investment Policies Allow banks to meet requirements Raise investment share of total out- Enact a permanent 5% credit for equipment investment. through investments in community put: development banks. Reduce, and ultimately eliminate, Spend somewhat more on infrastruc- federal budget deficit. ture, concentrated on upgrading ex- Encourage particular types of invest- isting facilities. ment with spillover benefits to the Encourage regional.industry "clus- rest of the economy. ters" by states or localities.

Trade and Foreign InvestmentPolicies Use liberalized and more rational practice" status as carrot to obtain Encourage further multilateral liber- better access to foreign markets. alization of trade and foreign invest- treatment under U.S. "unfair trade ment. .1=17. is Summer 1995 Evaluation Forum Issue 11 21 Features . Economic Change and Displacement

crease under this theory, yet unemploy- alive to the number in the worst-paid aIt is extremely unlikely that part- ment rates for the less skilled have risers trade and service sectors. Between 1967 relative to those more experienced. time jobs or welfare state programs and 1977 this pattern was similar formen have contributed to changes in the and women. From 1977 through 1987, The declining wages of less skilled wage structure. men may be due to a variety of shifts in the loss ofgoods-producingjobs contin- the structure of labor demand, Burtless ued, but was more rapid for men than for The direct and indirect effects of indicates. Demand for less skilled female workers. This may explain, in supply-side demographic trends workers could have decreased due to part, the diverging patterns of wage gain have contributed in a significant collapsing demand for the goodsor ser- for men and women. way to wage developments. vices produced by industries in which The declining importance of labor e The long-term trend toward higher less skilled labor is concentrated. Un- unions may be another contributor to skilled workers displaced by this shift the patterns observed in the data. joblessness, particularly among would have been pushed to seekem- Unions assist workers in raising the prime-aged men, has contributed, to ployment in other industries, forcing earnings of moderately-paid workers some extent, to earnings inequality. down the wages paid to unskilled labor and frequently reduce wage inequality. Changes in the industrial structure and in those industries. Changes in the tech- Also, the erosion of the value of the the decline in unionization havecon- nology of production may have de- minimum wage has affected the relative tributed importantly to inequality. The pressed the demand for less skilled labor earnings of people at the lower tail of in selected industries or across industries. the wage distribution. deteriorating quality of American schools has left a generation of work- Or the decline in average wagesre- Btutless says a striking trend in the ers poorly prepared to deal with the ceived by unskilled workers could be wage structure in recent years is the challenges of modern employment. due to sharp employment losses in in- continuing and pronounced drop in dustries such as mining and durable earnings among less educated men. In The changing structure of demand manufacturing that normally paid high an era in which an increased proportion for skilled and unskilled male work- wages to less skilled workers. Fewer of the workforce has some schooling ers within industries has contributed unskilled workers will be employed in beyond high school, workers who have to inequality. high-wage industries. And by directing had no more than a high school educa- displaced workers into low-paying in- tion appear to face growing handicaps Burtless concludes that if the de- dustries, the average wage received by in the labor market. This is due, Burt- mand for unskilled labor has drcpped, low-skilled workers in those low-wage less reports, both to shifts in labor de- the obvious policy response is to im- .41 industries could be depressed. mand and in labor supply. The most prove the qualifications of the less skilled to match the developing Changes in the labor supply, accord- important factors in increasingwage re- ing to Burtless, can have two kinds of ef- differentials between less skilled and quirements of the job market. "If the nation has too many unskilled workers, fects on the wage structure. An increase more skilled workers appear to be: rather than too many bad jobs," in the number of less skilled workers will says Changes in industrial structure. Burtless," both efficiency and equity raise the proportion of all workers who will be served by improving the skills receive low wages. Increased competition Declining rates of unionization. of workers now lodged at the bottom." for less skilled jobs may depresswages The fact that the number of college- relative to those received in more skilled educated young men entering the A Commentary on the occupations. Both of these effects have, Brookings Books in fact, occurred. Some of the differences workforce failed to keep pace with the rising demand for highly educat- between men and women may reflectsex These two books are usefulcompan- segregation in the labor market and dif- ed workers. ion pieces in gaining a better under- ferent trends in labor demand. If moder- Some economists have proposed standing of the trade-offs betweeneco- ately-paid men are concentrated in de- that the rise in part-time employment in nomic efficiency and equity. Clearly clining industries, Burtless points out, the U.S., and the effect of government there are substantial differences in the "middle-class jobs could be disappearing income maintenance has contributed to perspectives of different economists, in for men even as the distribution ofwages wage inequality. However, Burtless re- terms of theories, the assumptions driv- among women remains virtually un- ports that the research does not support ing those theories, the interpretation of changed." this contention. research findings, and strat..s to re- solve efficiency and equity problems. Major Patterns Burtiess' Conclusions The phenomenon of dislocation is caught in this web of information. The Meanwhile, the pattern of employ- The evidence from research is quite ment gains and losses emerging in sta- data on reemployment wage rates for persuasive, Burtless believes,on sever- increasing numbers of dislocated tistical analyses generally confirms the al points: proposition that the number ofjobs in workers, which typically involvea de- pression of their previous better-paid industries has declined rel- Wage inequality has risenamong wages, must men, but not among women. be figured into these analyses and the patterns emerging from them. 22 Evaluation Forum Issue 11 Summer 105 Features Economic Change and Displacement

mental changes in economic systems about collaborative restructuring ef- Understanding the in the past two decades. Dislocation has forts that is, from the transnation- Context of been a major legacy, requiring new al borrowing and adaptation of roles for workers, management, labor ideas. Dislocation and government in economic develop- The essays in the book identify both The purpose of this short article is to ment and a coherent economic re- structuring strategy. the challenges and the opportunities encourage you to read further in the lit- that exist in developing alliances that erature on restructuring and disloca- The book examines three main is- can have economic value. The authors tion. Four books worth the effort are sues: 1) the extent to which work orga- seem to agree that the best role for the the following: nization and market share, or competi- federal government is to facilitate and Technology and Global Industry: tive position, are influenced by coordinate activities at the regional, technical innovation; 2) how these Companies and Nations in the state and local level, and provide an in- changes are conditioned by government formation clearinghouse on best prac- World Economy, edited by Bruce R. policies, and 3) what role collective Guile and Harvey Brooks and pub- tices. The most important conclusion bargaining relationships between seems to be that without some kind of lished in 1987 by the National Acad- unions and companies play in the eco- institutional base for supporting eco- emy Press. nomic restructuring process. A series nomic restructuring, short-term politi- a Technology and Employment: Inno- of case studies of restruc- cal pressures will overwhelm the long- turing efforts, both in Europe and the vation and Growth in the U.S. Econ- er-term needs of industrial economies. U.S., are used to extract lessons for omy, edited by Richard M. Cyert American policymakers. Technology and Employment and David D. Mowery, also pub- lished by the National Academy A number of themes permeate the The Cyert book grew out of the Press in 1987. book. An interest in team-based work Committee on Science, Engineering organization is one, emphasizing the and Public Policy of the National Innovation and the Productivity Cri- participation of workers in areas tradi- Academy of Engineering, prompted by sis, by Martin Neil Baily and Alok tionally reserved for managerial deci- the Committee's interest in the impact K. Chakrabarti and published by sionmaking. Inierfirm linkages is an- of technological change on employ- The Brookings Institution in 1988. other interest among companies, ment opportunities, productivity and their suppliers, and their clients. Locat- life quality. The theme is that employ- Economic Restructuring and Emerg- ing the development of industrial poli- ment losses due to a decline in interna- ing Patterns of Industrial Relations, cies at the subnational or regional level tional competitiveness are likely to be edited by Stephen R. Sleigh and is another emphasis, encouraging a mu- grewer than any that result from tech- published by the W.E. Upjohn Insti- tually-reinforcing network of key ac- nologieal change, and that such change tute for Employment Research in tors. is essential to the preservation and ex- 1993. And these are some of the lessons: pansion of U.S. employment and wag- es. Themes and Recommendations In the absence of a national industri- al policy, the efforts of state govern- The book contributes a significant We merely provide glimpses of the ments have created a common direc- array of policy options and recommen- wealth of information and insights in dations in three areas: public policies tion that supports an economic these books, as an incentive to readers to assist worker adjustment to techno- to look for more. restructuring process integrating so- logical change; public policies to sup- cial goal-setting while retaining the port the development and application Economic Restructuring and efficiency of market competition. of advanced technologies; and im- Emerging Patterns ol Governments cannot influence eco- provements in private sector manageri- Industrial Relations nomic development positively with- al expertise in evaluating and imple- menting new technologies. The Sleigh book was based on a out sponsoring joint efforts among year-long series of seminars sponsored business, workers and unions. With respect to worker adjustment, by the City University of New York, the recommendations deal mainly with ending in a major national conference Transnational economic alliances changes in Title III of JTPA, particular- in 1990. The purpose of the book was and investment have blurred the na- ly to broaden the range of employment to summarize and comment on innova- tional identity of corporations and services, including job counseling, tive economic restructuring jointly un- made it more difficult to develop a skills diagnosis, job search assistance, dertaken by corporations, unions and national industrial policy. and placement services. There is strong government. Sleigh says that the eco- support for increasing funds for basic nomic shocks of technological change A national industrial policy needs to and job-related skills, and for expand- and global competition created funda- be developed, however, based on a ing income support during retraining. synthesis of what has been learned Instituting a program of federally-pro-

41=1111. 15 Evaluation Forum Issue 11 Summer 1995 23 Features . Economic Ch e and Dis!acement

vided direct loans and loan guarantees Referring to an OECD study, how- administered by states or local areas is ever, the authors propose that diffusing Worker Dislocation: felt to be a critical resource for retrain- and refining already-developed tech- A Review Article ing or relocation. Also recommended is nology is the key to high productivity a series of demonstration projects to growth.They therefore argue for con- test program designs, which are rigor- tinued support of basic science, for Ernst W. Siroinsdorfer ously evaluated. "middle-ground" and commercial re- search, and an improved educational Department of Economies innovation and the system that appreciates the knowledge Washington State University Productivity Crisis and skills required for technological This Brookings book is one in a se- innovation. There are important mes- sages here for workforce development. ries of publications on productivity is- "You should learn a trade. They can sues. Funded in part by the National never take that away from you." Science Foundation, it suggests that the Technology and slow pace of innovation may be influ- Global Industry Anonymous factory worker, ential in slowing productivity growth. Brooks and Guile begin the book by in advice to his son. Unlike some other economists, the au- identifying two characteristics of global thors propose that the size of the col- economic change: 1) the U.S. no long- As the above quote suggests, work- lapse of productivity growth in the U.S. er dominates world economic affairs ing men and women are in fear of los- should define this as a national crisis. but rather is only one element, even ing their jobs due to economic change, They claim that had the growth in out- though significant, is an increasingly whether it be from technological inno- put per hour after 1965 equaled the av- global economy, and 2) the growth of vation, foreign competition, shifts in erage rate from 1870 to 1965, output other national economies has permitted demand, government regulation, or oth- would have been more than 20% higher production and distribution to become er sources. There are epic tales of in 1985 than it actually was. Unless increasingly transnational. Technologi- highly skilled workers attempting to there is an increase in productivity cal advances have played an important outdistance technological change growth, they say, Americans will suffer role in emerging economic and techno- through increased effort and other stagnant or falling living standards. logical integration, increasing the ca- means, with the demise of the glass In addition to labor productivity, pacity of transnational systems that car- bottle blowers at the turn of the century two other indices of production perfor- ry information, goods and people. coming to mind most sharply. There mance are given attention: output per Focusing on the activities of multi- are tales of violent resistancethe ac- unit of capital used, and multi-factor national companies and the policies of tion of workers themselves creating production. The owners of capital did industrialized nations, the book seeks evocative new words for our Ian- poorly after 1965. And the collapse of to answer these questions: guagesabotage and Luddites.There multi-factor productivity growth, in the is the modern image of the "Rust aggregate, has been startling, with no What are the effects of changing Bowl", characterizing the upper indus- clear message that it will increase. The technologies on the production and trial Midwest. distribution of goods and services in authors place this in context by refer- Scientific interest in the impact of ring to the data on other industrialized the global economy? economic change on worker displace- countries, which reveal a similar and How do technological advances con- ment began early in this century, fol- sometimes more severe slowdown. tribute to shifts in the relative com- lowed by seminal articles by Haber Although the authors focus on the petitive advantage of nations, re- (1925) and Myers (1929) in the 1920s. role of innovation, they find that infla- gions and firms? Case study monographs of worker ad- tion, , oil price fluctuations, justment to plant shutdown began oc- and other economic interferences in the a How do governments and enterpris- curring in significant numbers soon af- 1970s also affected economic perfor- es respond to the dynamics of tech- ter the end of World War II, but mance. They suggest that there are two nological advance in a global econo- systematic empirical research on the kinds of strategies for utilizing technol- my? problem of worker displacement did ogy for economic growth: one expands not begin until the 1960s in response to the technological frontier, developing What are the likely consequences of waves of restructuring that began in the leading-edge industries (characteristic their efforts? post World War H era to hit auto manu- of the U.S.); the other diffuses technol- Although the book directs our atten- facturing, meat packing, the railroads, ogy that has already been developed (as tion to the management side of eco- coal mining, and other industries (See in Europe). The Japanese are now using nomic issues, there are keen insights to Borus, 1964; Schultz and Weber, 1966; both strategies. be culled for workforce preparation and Somers, 1968). Even then, the issues dislocation between the lines.

AWRY /IMES 18 24 Evaluation Forum Issue11 Summer 1925 Features Econote at_ icalsylient

cash flows to the program of one bil- were not fully understood, whether one velopment Act of 1959 became the first lion dollars a year and historical expen- was concerned with the measurement national legislation to provide for inter- of the displacement loss to the individ- vention in the worker readjustment pro- ditures since 1959 that exceed many ual and society; the time and money cess. This legislation focused on poli- tens of billions in current dollars! cies to aid in the economic adjustment costs of finding new employment; is- The Dimension of the Problem sues of social compensation to losers process. No direct cash compensation from change; whom to taain in new for loss of human capital was contem- Perhaps part of the historical reason skills, how to train them, and what plated. The first effort at direct cash for this tolerance has been our failure skills to train; or, how to provide reli- compensation for displacement came until recently to compeehend the full able data to inform social policy on any with the trade adjustment assistance dimension of the problem. The studies of these issues. program in the Trade Expansion Actof by Leigh and Jacobson et al. indicate 1962. Compensation was to be paid to that the numbers affected are large and The three volumes discussed in this those workers damaged by liberalized the losses per worker are severe. essay carefidly and thoroughly assem- international trade. The intent was, in Working with different data sets and ble much of the methods of analysis part, to reduce political resistance to re- slightly different assumptions, both and knowledge that we possess today duction in trade restrictions; however, Leigh and Jacobson et al. estimate sim- concerning the above issues. The Costs the program concept had strong theo- ilar annual totals of workers hit by dis- of Worker Dislocation by Jacobson, retical support in the compensation location. A key assumption in such es- LaLonde and Sullivan (1993) provides principle of theoretical welfare eco- timates is to specify a minimum length one of the most conceptually compre- nomics. Trade adjustment assistance of employment in a firm in order for a hensive and empirically complete stud- has gone through several modifica- laid off worker to be counted as a dislo- ies of the patteen and quantity of wage tionsfirst by simplifying eligibility cated worker. For one estimate, Leigh loss due to economic change and dislo- for compensation in 1974 and then hav- sets a minimum of at least five years of cation. Duane Leigh's study, A.ssisting ing eligibility and payments tightened employment at a firm prior to disloca- Displaced Workers, is a comprehensive up in 1981. tion. This assumption yields a total for review as of 1989 of the policies de- As pointed out by Leigh, a major displaced workers of 3.2 million a year signed to ease the burden to workers of over the time period 1979 to 1984.For economic change and presents the most problem with this legislation is the dif- ficulty of identifying whu the losers are the period 1980 to 1986, assuming a six recent evidence as to their relative ef- year minimum job tenure, the estimate fectiveness. Back to Work, by Howard from liberalized international trade (or any other structural charge to the econ- of Jacobson et al. is 2.6 million. Tak- Bloom, reports with thorough method- ing the midpoint of this range gives an ological detail on one of the very few omy) and how much they lose. Current estimate of approximately two per cent classical random tests of programs de- policy is now summarized in the Eco- nomic Dislocation and Worker Adjust- of the economy's labor force each signed to aid displaced and disadvan- yearmore than 20% in ten years taged workers. These studies progress ment Assistance Act of 1988 (Leigh; damaged by dislocation. from the most to the least general, and pp. 93-94). This latest revision was are discussed in that order. motivated by evidence that prime age Leigh's estimate is that the typical males were having difficulty adjusting displaced worker loses about 20% of Reasons for Concern to economic dislocation (Jacobson et his or her pre-displacement wages. al.). During fiscal year 1991 the federal Policy makers and analysts are inter- More conceptually sophisticated analy- government spent nearly one billion sis using longitudinal data that incorpo- ested in who gets damaged by econom- dollars to help displaced workers under ic change, what the time pattern of eco- rates a comparison group suggests that this act. the initial annual loss is closer to 40% nomic loss is, and what the total Moreover, the aver . present value of that loss is. A key rea- Yet, to anticipate what is to come, (Jacobson et al.). son for this interest is to determine very little is known about the broad age total present value of losses whether to intervene in the economy outcomes of various forms of labor amounts to about $80,000. This is a and assist the dislocated. The decision market interventions to aid displaced private loss to the individual but it is to intervene is in part a function of the workers other than job search assis- also a real loss of economic welfare to total earnings lost per dislocated work- tance. As Leigh's thorough review of society as well. As Jacobson et al. point out "displacement t.s a major set- er and the distribution of that loss the literature shows, even less is known across groups in the labor force. Sec- about the fine detail of optimal pro- back for experienced workers. " ond, if society wishes to compensate gram design for different types and Using the Jacobson et al. estimates, mixes of treatments relative to their tar- the losers to any economic change, it the total annual cost of displacement to A needs to know who the losers are and get populations. It is difficult to com- society is a present value of $208 bil- $.'; how much they have lost. prehend this long-standing tolerance of lion (2.6 million workers a year times partial knowledge and outright igno- $80,000 per worker). What if these es- After extensive Congressional hear- rance of these policy outcomes. Yet, timates are four times too large? The ings in the late 1950s, the Area Rede- such tolerance exists, even with recent

NOMIVIVARFACCAQIIMIENNI.111111 411111MMEMP .171.011Mai 17 Evaluation Fotum Issue 11 Summer 1995 9

Features Economic Change and Displacement

loss is still $52 billion a year! Hopeful- Policies to Mitigate vouchers, to encourage firms to hire ly, the economic changes that cause Earnings Loss displaced workers. this dislocation provide society as a whole with benefits that equal or ex- With the possible exception of some 3. Job search assistance programs, in- of the wage premiums due to unioniza- ceed the losses. However, ignoring cluding the components of: melt- Food Stamps and other transfer pay- tion, unanticipated' economic disloca- tion that reduces or eliminates any of reach; orientation; assessment and ments, the direct compensation that is testing; job search work shops; paid by society's beneficiaries to this the four sources of wage premia above result in a loss of human capital or situ- counseling; and job development change is only about a billion dollars a and job matching. year, as noted above. ational productivity. This in turn is translated into a loss of earnings power. 4. Advance notice and on site delivery These losses of productive capacity Why Such Losses Occur can of services. be seen as property right losses, and, The above estimates are large and hence, compensation is justified (some 5. Economic alternatives to plant shut- dramatic. What is their source? Con- would say required) on the grounds downs, including: subsidies of vari- ceptually, does the total sum represent that no one be made worse off bya giv- a potentially compensable loss? Jacob- ous kinds to employers; assistance to en economic change and at least one enable employee buyouts of firms; son et al. draw a careful taxonomy of person be made better oft The point is the source of these losses. Wage pre- and subsidies to workers to start up that for society to benefit, there must be their own enterprises. mia can derive from the following a net gain after all compensation for sources or combination of sources: costs is accomplished. The first four policies are intended to 1. Some workers gain wage premia due Leigh presents a complete taxonomy aid the transition of dislocated workers to new jobs and replace lost human to a collective bargaining agree- and critique of the policies thatare in- ment. tended to replace lost earnings power capital. The final group of policies rep- resent efforts at direct job creation. 2. Some workers may have firm-, in- due to dislocation. He makes the point that while the most widely publicized The evidence as to the relative effec- dustry-, or occupation-specific hu- group of displaced workers come from tiveness of each of these policy choices man capital. unionized, high wage jobs in the manu- is limited to say the least; however, facturing sector, in fact displaced work- there is better evidence on some initia- 3. Some firms may offer higher wages tives than on others. and be able to select higher produc- ers overall are a very diverse group, tivity workers as a result in certain differing by gender, ethnic origin, in- Occupational Training situations. dustry, occupation and labor market lo- cation. These characteristics imply dif- Occupational training programs pro- 4. Some firms and certain sectors of ferent types and mixes of policies for vided at the state level include a variety the labor market employ promotion each subgroup of displaced worker of innovative components, including practices that preclude lateral entry. (Leigh). vouchers; active participation of em- ployers to design training, choose train- While he focuses on program poli- Leigh suggests that wage premia due ees, and hire the trained workers; per- to collective bargaining agreements cies that have been implemented and evaluated in the several states, the fol- formance-based contracts; and the may not represent a component that diversion of unemployment insurance lowing program policies represent the justifies compensation. However, there funds to pay for training. Leigh points is evidence that unionized workers full range of alternatives in today's pol- icy toolkit: out that the net impacts of these pro- work more intensely and are morepro- grams largely have to be inferred from ductive than similar non-unionized 1. Formal classroom occupational evidence based on evaluation of federal workers and that, in response to higher training, with remedial or adult basic programs since the state initiatives gen- unit wage costs, firms will also select education where also needed. erally have not built in evaluationcom- more productive workers as a result. ponents that incorporate comparison To the extent that this is true, then 2. On-the-job training, with director groups. An exception to this is the union wage premia lost due to disloca- indirect wage subsidies, including Texas initiatives discussed by Bloom. A tion are compensable. Real productivi- ty has been destroyed, not just union- There is now evidence with strong induced wage rents. In sum, if any of ii-here is a strong theoretical argument that internal validity based on the National only displacement losses due to unanticipated .17TA Evaluation conducted by Abt As- the four conditions are negatively af- economic chnnge be compensated. One can ar- fected due to displacement, both short gue that where change is predictable the market sociates Inc. in cooperation with the term and long term losses can result. will provide a compensating wage differential in Manpower Development Research Cor- the form of a wage premium to balance out the poration that retraining has a positive expected capital loss. See Hamermesh, Cordes return for adult men and adult women. and Goldfarb, 1987. Sixteen Service Delivery Area (SDA) sites were judgmentally chosen with 26 1111NOWINCe 18 EvaluatIon Forum Issue 11 Summar 1995 ,

Features Economic Change and Displacement

workers and no necessary costs to em- random assignment to treatment and sonable to the process of provid- advance noticeappears control within each site. As with the ing reemployment services in order to ployers. Thus, to be a useful social policy. sites evaluated by Bloom, adult women gain the most effective use of social re- sources. The notion is to presentall fared better than men. Other, non-ex- Job Preservation through perimental evaluations of training re- workers withjob search assistance first, withholding initially any wage Subsidies and Employee ported by Leigh, such as the Downriver Buyouts Community Conference Economic Re- subsidy, training voucher or other adjustment Program, are too method- training services. Workers with fewer How appropriate are policy inter- ologically flawed to allow any defini- disabilities due to the dislocation will ventions designed to keep threatened tive judgment as to program outcomes be able to find jobs first. This strategy firms operating? Leigh discusses three (Leigh). was followed in theDownriverproject alternatives: mentioned above and positive returns 1. "Assistance to allow current owners Bloom's (1990) evaluation of three to job placement services were identi- Texas sites providing reemployment fied, within the (severe) methodological to keep operating or to attract new services for displaced and disadvan- limitations of that analysEs. Leigh reports owners from outside the firm. taged workers demonstrates conclu- that in general job search assistanceser- 2. Assistance to encourage employee sively that with the appropriate politi- vicesappear to pay off socially. cal will and careful attention to buyouts of existing facilities. experimental design and execution of Related to job search assistance is 3. Assistance to enable displaced thereemployment bonus initim've that design in an on-going program, it workers to start up their own small wherein a cashbonusis paid to workers is possible to properly evaluate such in- businesses." itiatives. His exhaustive account and to induce them to search more diligent- record of design, program and evalua- ly for work. Experimental results for Evidence on all these initiatives is tion management, and analysis provide Illinois suggest such a policy does limited to a few qualitative studies. a blueprint for federal and statelegisla- work, but non-experimental results in Thefirst initiativeis generally sub- tors who with to discover whether pub- New Jersey suggest otherwise. The sumed under programs to establish "en- lic investments of this type are effec- weight of the evidence should go to the terprise zones." The notion is to stimu- tive. We agree with his statement that experimental results, at this time, con- late economic development by the Texas evaluation "...demonstrated ditional on new evidence developing. reducing government regulation and the feasibility of conducting a high- lowering taxes in the zones. The ap- quality, randomized field experiment at Advance Notice parent reality of state enterprise zones, several sites simultaneously, within a Early intervention in the job reloca- though, is that total but not marginal modest budget and limited time frame" tion process makes intuitive sense. tax rates are reduced, and the apparent (Bloom). Workers are helped thereby to maintain benefits of increased government activ- ity within the zones are reduced by the Of course, the problem at the federal their self-esteem and job search ener- cost of complying to regulations in or- level is not one of inducing an individ- gies, and firms can be induced to coop- ual program site to agree to an experi- erate in any job development and re- der to receive any assistance. mental evaluation. TheNational JTPA training efforts more effectively. These Thesecondstrategy relies for its ef- Evaluationhas demonstrated this. The concepts are incorporated in the Work- ficacy on the increased productivity in- problem at the federal level is to mar- er Adjustment and RetrainingNotifica- duced in workers who now own the shall the political focus and energy to tion Act of 1988. Leigh points out that firm. For a variety of reasons, such a bring aboutnationally representative early notice does appear to reduce the consistent successful result has yet to experimental evaluationswithin the private costs of adjustment to workers, be established in practice. And, there is context of on-going programs. Given largely due to the fact that some work- the further downside that the workers the billions of dollars that have been ers quit before the plant closing as a re- typically have taken their diverse port- spent delivering subsidized employ- sult of locating a new job prior to sev- folio of retirement investments and ment and training programs since the erance from the threatened job. converted its capital value to a single Area Redevelopment Act was initiated However, it is possible that advance egg in a single baskettheirthreatened 36 years ago, it is not unreasonable to notice can disrupt the production effi- firm. Such an action simply is not pru- expect such an initiative. ciency of a firm in a variety of ways, so dent, especially if the original owners that gains to workers are then offset by are not willing to maintain asignificant Job Search Assistance losses to stockholders. Plausible argu- stake in the firm. As Leigh carefully points out, it is ments in the evaluation literature exist Thethirdstrategy has several prob- difficult to identify initially which as to both positive and negativeeffects lems with it. First, if the new business- workers laid off due to economic on worker productivity and firm costs es would have started up anyway, change have suffered losses of human due to advance notice. However, the which has to be a possibility, those re- capital. This being the case, it is rea- weight of evidence suggests gains to ceiving the subsidy get a windfall gain.

19 Evaluation Forum MO 11 Summer1095 _

Features Economic Chan e and Us lacement

No net social benefit results. Second, education should be offered, yet states chinery in the Men's Clothing In- since such a start-up firm would be have slighted this initiative in favor of subsidized, it would have lower costs economic development activities. Fi- dustry." The Journal of Political and could conceivably displace an ex- nally, there simply is not enough evi- Economy. April. isting, otherwise efficient, firm or dence at this time to offer such subsi- firms. Harnermesh, Daniel S., Joseph J. Cord- dies to firms in an effort to maintainor es, and Robert S. Goldfarb. 1987. increase jobs. And workers who can As to the third point, few additional "Compensating Displaced Work- jobs may be generated and, indeed, in- succeed in difficult entrepreneurialen- deavors are least likely to need the ser- ersWhat, How Much, How?" in ducing inexperienced individuals to at- Labor Market Adjustments in the tempt to be entrepreneurs may just ex- vices tendered to truly dislocated work- acerbate failure. ers. Pacific Basin. Peter T. Chinloy and Ernst W. Stromsdorfer. Editors. As a concluding point, given the Boston, MA: Klewer-Nijhoff. An Overall Judgment very large real losses to workers as well Convincing and detailed evidence as the large amount of public resources Jacobson, Louis, Robert LaLonde and now exists on the extent and severity of devoted to assisting these workers, it is Daniel Sullivan. 1993. The Costs of earnings loss due to economic changes reasonable to expect more systematic Worker Dislocation. Kalamazoo, of all types. The social costs arevery evaluation of program effects, which MI: The W.E. Upjohn Institute for large indeed and warrant a systematic includes more use of experimental de- Employment Research. development and valid testing of poli- sign, but at the very least, the use of cies to deal with the problem. state-of-the-art quasi-experimental de- Leigh, Duane E. 1989. Assisting Dis- sign. placed Workers: Do the States Have The survey by Leigh reveals a very a Better Idea." Kalamazoo, MI: rich and diverse set of policy options, The W.E. Upjohn Institute for Em- only the highlights of which arepre- sented in this survey. But the fact is Bibliography ployment Research. that but for a few of the policies, we Myers, Robert J. 1929. "Occupational Bloom, Howard S. 1990. Back to know little of their efficacy. Job Adjustment of Displaced Skilled search assistance should be the core Work: Testing Reemployment Ser- vices for Displaced Workers. Workers." The Journal of Political readjustment policy. It is relatively low Economy. August. cost and avoids the problem of having Kalamazoo, MI: The W.E. Upjohn to distinguish truly displaced workers Institute for Employment Research. Schultz, George P. and Arnold R. We- from those unemployed due to season- Borus, Michael E. 1964. The Econom- ber. 1966. Strategies for the Dis- al, cyclical or short term frictiokial placed Worker. New York, NY: ic Effectiveness of Retraining the causes. The role of retraining is' less Harper & Row. clear. It seems clear that employers in Unemployed. New Haven, CT: The local labor markets must be involved in Yale University Press Somers, Gerald G. Editor. 1968. Re- the training process. Training should Haber, William. 1925. "Workers' training the Unemployed. Madison, be offered only for well specified needs WI: The University of Wisconsin Rights and the Introduction of Ma- and in the types and quantities of work- Press. ers needed. Apparently, more remedial

2 8

20 Evaluation Forum Issue 11 Summw 1995

11Ps CV.,

Feature:. Natbnal Studies of Dislocated Worker P rams

National Studies of DislocatedWorker Programs 's tcrtoriantriod uctioti. numbers of plant closures and mass adequate, sufficient responses. layoffs increased concern about dislo- The authors discuss a variety of pro- In a comprehensive paper on dislo- cation. In 1982, Title ill of JTPA re- grams and pilot projects spawned by cation by Robert W. Bednarzik, a senior sponded to that concern. And states anxiety over increasing dislocatien. economist at the U.S. Department of began to finance their own retraining Five*demonstration projects had been Labor, and Malcom Lovell, Director of programs for dislocated workers. Title undertaken under Title ill over an eight- the Labor Management Institute at III was followed by the Economic Dislo- year period beginning in 1980. All were George Washington University, the au- cation and Worker Adjustment Assis- evaluated. Bednarzik and Lovell's sum- thors discuss the problem of dislocation tance Act (EDWAA), which was to in- mary of the results for OJT and class- and its major causes, and review na- crease the EatT system's provision of room training in the five projects shows tional projects seeking to reduce this assistance to the dislocated. a mixture of success and failure for the emerging characteristic of postindustrial There was reason to wony about same interventions in different projects. societies. dislocation, the authors contend. From A consistent finding, however, was that Written for the Competitiveness Poli- 1985 to 1989, 4.3 million workers had training, per se, did improve the earn- cy Council in 1992, the paper provides become dislocated. Most were in man- ings of dislocated workers over time, an excellent summary of expert opinion ufacturing, but dislocation was spilling and job search assistance seemed to arid research regarding dislocation is- over into the service sector. EDWAA improve both employment and earn- sues in the 1980s, and the programs and TAA were serving about 150,000 ings. developed to address those issues in dislocated workers a year out of over A study of ten exemplary dislocated the U.S. and Europe. The authors place 800,000. In 1986, the Secretary cf La- worker projects under Title lii provided retraining and reemployment strategies bor's Task Force on Worker Dislocation important insights about service strate- in the context of the need to make sub- took a serious look at the problem gies and service delivery processes, to stantial improvements in the quality of again, as well as the programs devel- which the authors give appropriate at- the American workforce to remain inter- oped to resolve it. Not surprisingly, they tention. But by 1989 EDWAA had re- nationally competitive. found that the workers at highest risk of placed Title lit, emphasizing "rapid re- dislocation were those in high risk in- They locate the beginning of adjust- sponse," labor-management dustries operators, fabricators and ment assistance programs in the area cooperation, a more powerful role for in mining, construction and substate areas, linkages with Ul, the of public transportation. The govern- manufacturing. Minorities and youth ment had concluded in the aftermath of Employment Service and TM, and were at particular risk, Bednarzik and longer-term retraining. There the paper consolidating certain transportation sys- Lovell point out, because of their con- tems that workers losing jobs in the ends, and our series of articles on na- centration in import-sensitive industries. public sector should be compensated tional evaluations begins. The well-con- These workers experienced severe ceived and implemented studies re- by receiving assistance in finding new structural barriers to reemployment. ones. Similarly, changes causing job viewed in these articles tell us much loss in the area of trade were followed Later In the 1980s we began to see about an ongoing national dislocated by the Trade Adjustment Assistance middle managers and other white collar worker program, EDWAA, about ED- program (TM). workers in the ranks of the dislocated. WM demonstration projects, and about Public policymakers began to ask DOL studies of self-employment. The surge in unemployment in the themselves if EDWAA and TM were early 1980s related to unprecedented

A

U. -o JTM was accompanied by formal gov- One result was theconducting of Evaluating ernment interestin performance man- some of the most sophisticated and sci- Dislocated Worker agement and evaluation, likely related entifically viable evaluations of ongo- in part to the decentralization and ing and demonstration programs in the Programs: U.S. privatization of employment and train- employment and training field that had Department of Labor ing programs to states and local areas. ever been undertaken. Most of the pro- Among other purposes, federal moni- grams and projects evaluated have fo- Studies in the 1990s toring and evaluation guidelines and cused on the economically disadvan- standards were to compensate for this taged. However, in the 1990s the U.S. In the 1990s, theissue of worker devolution of power by maintaining Department of Labor (DOL) launched a displacement and dislocation became a quality, consistency and equity. They number of evaluations of Title HI pro- more serious concern, growing more were to assure that a transfer of authori- grams and demonstrations, as well as problematic with increasing economic ty and administration to states would pilot tests of new strategies addressing competition and risk. Title III of JTPA not result in a redirection of programs dislocation. was created to respond to this fluoresc- toward state-specific agendas and away ing phenomenon in postindustrial life. Historically, dislocated workers from Congressional intent. were a new target group for services,

'21 Evaluation Forum Issue 11 Summer 1995 111,11111111,..."11.

Features National Studies of Dislocated Worker Programs

encompassing workers of all socio-eco- delivery systems. All but one of the government has been sensitive to dis- nomic levels who were experiencing outcome studies utilized either random placement due to trade conditions as 271 permanent job loss in the occupations, assignment to produce equivalent treat- early as the 1930s. The Trade Expan- industries and businesses that had sus- ed and control groups, or a constructed sion Act of 1962 began the recent his- tained them over time. Employment comparison group of individuals con- tory of compensating workers for trade- and training policymakers, planners sidered similar to those receiving pro- induced job loss thorough "trade and program designers faced a new gram services. adjustment assistance." Although the challenge. And evaluations now began TAA program put in motion by this to investigate the nature and effects of There is no question that measure- ment is a special challenge in process legislation was reduced drastically in programs responsive to the middle the 1980s, legislation in 1986 reempha- class. studies. Many of the variables describ- ing program implementation that are sized adjustment services, 1988 legisla- likely to exert the greatest influence on tion expanded the use of training to The Status of Evaluation speed readjustment, and reemployment Research program outcomes are difficult to mea- sure. The variability in these measures services were subsequently given more This article briefly reviews a subset across process evaluations reduces attention. With the advent of EDWAA of the DOL-funded evaluations of pro- one's ability to compare studies. in 1988, coordination between TAA grams for dislocated workers. The and the new program was mandated. 1990s witnessed increased government The measurement of outcomes has This linked TAA with a more disci- commitment to the use of experimental been limited traditionally to a small set plined and sophisticated service deliv- research designs for estimating the ef- of easily quantifiable employment and ery system. Clearly the legislative evo- fectiveness (return on investment) of earnings variables, to which measures lution of TAA reflected the growth in ongoing programs and demonstration have been added to provide informa- the population of dislocated workers in projects. At the same time there was tion on interim outcomes such as the this society. new support for learning more about completion of educational and skill the influence of organizational struc- competencies, and longer-term out- The Process Evaluation tures, functions and processes in pro- comes such as job retention. of TAA grams, and service delivery characteris- However, even these standard out- The purpose of the evaluation was to tics, on the outcomes of these programs come variables are not always defined learn more about 1) the certification, that is, a growing interest in process identically. And less tangible and more notification and application processes, evaluations, or studies of program im- inclusive sets of outcomes for individu- and 2) the delivery of reemployment plementation. als, such as an increase in work motiva- services. Conclusions and recommen- Where possible, the DOL encour- tion and productivity, greater self-es- dations are presented in Figure 2. aged a "methodological partnership" teem in the workplace, or less stress in between process and net impact evalua- one's family life, are rarely given atten- The Evaluations of the tions when studying programs, which tion, even though these outcomes may Implementation of EDWAA be highly significant. permitted evaluators to offer better ex- The design of EDWI-LA programs planations of the potential causes of Conceptualization differences also was dependent largely on the planning program outcomes. The process and net reduce theimparability of evalua- preferences and expertise of the states. impact evaluations of the WA State tions, even tiough studies may be fo- Consequently it was important for DOL SEED program are an example, as are cused on similar issues and target to know, from a public policy stand- the twin process/net impact evaluations groups. The underlying assumptions point, what designs seemed most ap- of the Trade Adjustment Assistance evaluators make in defining and fram- propriate to the needs of dislocated program. In other cases, process evalu- ing the research, and the paradigms workers and what configurations ap- ations preceded potential outcome stud- (propositions) that flow from these as- peared to produce the outcomes intend- ies, or were considered most appropri- sumptions subtly influence the con- ed by the Congress. Conclusions from ate when the focus was mainly on struction of research questions. the EDWAA evaluation are presented implementation issues. These caveats should be kept in in Figure 3 and recommendationsare Figure 1 provides a quick overview mind when looking across the conclu- presented in Figure 4. of the kinds of evaluations conducted sions and recommendations of evalua- The Phase II report on EDWAA im- by DOL on dislocated worker pro- tors regarding these programs for dislo- plementation was reviewed inan earlier grams, the major reseach questions for cated workers. issue of Evaluation Forum andso is which answers were sought, and the not given detailed attention in this arti- '71 methodologies used. The Evaluation of the Trade cle. The methodology of the evaluation Adjustment Assistance is quite unique. The research was guid- The Results of the Studies Program (TAA) ed by an innovative, thoughtful and DOL's process studies emphasized Although our concern about the dis- useful model describing the elements of similar aspects of programs: their orga- located in the 1990s has been more dra- "a responsive service delivery system" nizational dimensions and their service matic than in earlier periods, the federal for dislocated workers. Information

22 Evaluation Forum Issue 11.summer 1995 Features National Studies of Dislocated Worker P

..;1 Dot: biatuitions

The St. Louis Metropolitan Re-Employment Project An Impact Evalu- ation. Research and Evaluation Report Series, Employment and Training Administration, 1993. Type / Design Main Research Questions Methods Net impact What are the net impacts of the services re- Data Collection evaluation / ceived by participants on their earnings, receipt U! wage record data quasi-experimental of Ul, and their reemployment over time? Telephone interviews j,= design Program records Data Analysis Treated group was compared with random- ly-selected sample of St. Louis residents fil- ing for Ul.

.; esdY,Xeltir, PliqpGlialod etch' Evaluation of the Trade Adjustment Assistance Program: Process Report. Office of Strategic Planning and Policy Development, Employment and Training Admirgstration, 1992. Type / Design Main Research Questions Methods Process evaluation / What is the nature of services provided? Data Collection Descriptive design To what extent did 1988 amendmente im- II Aggregate quarterly quantitative data on prove program? payment and service activity under each pe- tition. What were "best practices" in terms of service delivery? Unstructured on-site interviews with central office staff and some local staff. How does TAA interact with other dislocated worker programs? Participant survey. Data Analysis Comparison of delivery of services over time, among states, by petitions.

Mario',e,yeftitc.-FP19),PAPArtlEMPPRIMIcfniisiSMOLTNIVIP491111ameggositaiikimAciOmmew international Trade and Worker Dislocation: Evaluation of the Trade Adjustment Assistance Program. Office of Strategic Planning and Policy Development, Employment and Training Administration, 1993. Type / Design Main Research Questions Methods Net impact What are the differences in the characteris- Data Collection evaluation / tics of TAA caseloads and other dislocated Participant survey quasi-experimental workers? Telephone interviews with random sample design What kinds of services were provided by of recipients comparison with participants in other pro- Labor market data grams? Data Analysis What effect did TM services have on the re- Comparison of client characteristics, servic- employment experiences of TM participants, es received, and client outcomes for TM as compared with other dislocated workers? treatment group with matched comparison What effects did the 1988 amendments and group of Ul claimants. changes in Title ill have on TM?

Figure 1 continues

10.11 Evaluation Forum Issue 11 Summor 1995 23 Features . National Studies of Dislocated Worker Pr. railis

Study of the Implementation of the Economic Dislocation and Worker Adjustment Assistance Act. Research and Evaluation Report Series, Employment and Training Administration, 1992. Type / Design Main Research Questions Methods Process evaluation/ What is the nature of EDWAA policies and Date Collection descriptive design organizational practices? Telephone interviews What are the operational links between Site visits: discussions and interviews with 1) state and substate functions and 2) de- state, substate, and service providers sign and delivery of services? Data Analysis Cross-site analysis of qualitative and quantita- tive data from program MIS and other sources.

wo, , .,,,,,v; ,. . 1r, T,- , ; Vibblooklipgri'VnA99gen1/400.4..V..41)Au Study of the Implementation of the Economic Diilacetion and worker Adjustment Assistance Act: Phase II, Responsiveness of Services. Research and Evaluation Report Series, Employment and Training Administration, 1993. Type / Design Main Research Questions Methods Process evaluation I What types of services are provided? Data Collection descriptive design To what extent do these services respond to Site visits. clients' needs? Discussion with state-level staff. What are "best practicee regarding Program interviews with substate staff. designs and service practices? On-site observation by evaluation team. What problems are occurring re: service de- Discussion with participants. livery policies and practices? Review of client files. Data Analysis Comparison of collected Information with an evaluator-constructed ideal-type "Conceptu- al Model of Responsive Dislocated Worker Services" to judge extent to which programs fit this model. ee. it Web :katheilitsitSUzantiii. 'fvk*reittiori..teb:Oratfkoian,alt andalotwiyi Study of the Implementation of EDWAA: Report on the Survey of Substate Areas. Research and Evaluation Report Series, Employment and Training Administration, 1993.

Type / Design Main Research Questions Methods Process evaluation/ What is the nature of the organization and Data Collection descriptive design operation re: EDWAA services suth as type Worker Adjustment Annual Program Report: of clients targeted, characteristics of services for types of clients served, recency of , provided and their possible effects on out- educational level, previous wage level, barri- comes? ers to employment, program outcomes. What is the orientation of programs and ex- Survey data. tent to which they are consistent with the Review of state plan.s. themes of their legislation, and what is the Labor market information. possible effect on outcomes? To what extent is EDWAA coordinated with Data Analysis Title HA of JTPA, with what effects? Survey results and other data were analyzed What Is the probable effect on outcomes of against the coneeptual framework of proposi- the level of responsivenese to client needs? tions regarding ways substate areas organized their pmgrams, the types of clients served, the types of responsiveness to needs, and the out- comes achieved, using statistical methods.

24 32 Evaluation Forum .issue ii Summar 1295 Sea: ee...*a. =-13,

Features National Studies of Dislocated Worterina_rt _Ms

Vat Evaluation of the EDWAA Job Creation Demonstration.Research and Evaluation Report Series, Employment and Training Administration,1994. Type / Design Main Research Questions Methods Process evaluation / How effective are CommunityDevelopment Data Collection descriptive design Corporations in expanding employment oppor- Site visits. tunities for dislocated workers through linkages inteiviews with staff from 11 substate pro- between entrepreneurial training and other eco- grams in eight states. nomic development activities? interviews with state-level EDWAA staff in five states. Conferences. Telephone interviews. Data Analysis Analysis of an array of quantitative and qualitative information.

,se isykafir "Wry'Stephen Walsh, 'AridElonald D'Andoo Serving Dislocated Farmers: An Evaluation of the EDWAAFarmers and Ranchers Demonstrations. Research and Evaluation Report Series, Employment and TrainingAdministration, 1994. Type / Design Main Research Questions Methods Process/outcome How effective are the service categories used Data Collection evaluation, descrip- in the demonstration? DOL WAPR data. five design On-site interviews and discussions. Data Analysis Services received compared with those re- ceived by other EDWAA participants. Relationship analyzed between services and participants' short- and long-term outcomes.

-Wandner I:Stephenfed4. .04 TM- 74' Self-Employment4Programs for Unemployed Workers. Pt;rts 1-4. Unemployment Insurance Occasional Papers, Employment and Training Administration, 1992,* Type / Design Main Research Questions Methods Process evaluation/ How viable is self-employment as reemploy- Data Collection Quantitative data on clients and services via descriptive design ment option for the unemployed? DOL's Participant Tracking System. N What is the relationshipbetween 1) early pro- vision of business assistance services and On-site discussions and interviews with self-employment allowances to Ul claimants, SEED personnel using standard protocols, and 2) successful outcomes in self-employ- via monitoring visits. ment? On-site group interviews with SEED paracipants. On-site observation by evaluation team. Data Analysis Cross-site analysis of quantitative/qualitative data on organizational arrangements, service delivery. Net impact evalua- How effective are SEED services in terms of Data Collection tion/experimental self-employment and earnings? Ul wage records; telephone survey. design; cost/benefit Data Analysis study s The self-employmentand earnings out- comes of SEED participants (offered busi- ness start-up training, a waiver of the Ul Stephen Wandnor and Jon Messenger; 9.From Unemployed to Self-Employed , work requirements, and periodic payments 2, Washington Self-Employment Enterprise Development (SEED)Demonstration,Terry Johnson and Janice Leonard;3, Massachusetts Unemployment Insurance Self-Employment Demonstra- equal to Ul benefits) were compared with a tion,Jacob Benue, Mohr& Wood, Chris Naplerala. and Terry Johnson; 4,Self-Employment control group that received only Ul payments Progfams for the Unemployed: An Analysis of Program Evaluation and OperationsResearch In and services. Europe and NorthAmerica, Douglas Scott. 25 EvaluationFORM issue11 Slimmer1905 33 Features . National Studies of INMostns

about EDWAA implementation was Figure 2 then judged against this model. The ''EvaluAtion.Corichiiions and RatoMmendatfons major conclusion was that there was Issue Area substantial variation across the study Certification of groups of workers and notification of workers about TAA sites in the level of responsiveness of Conclusions EDWAA to the needs of this target Recommendations Most workers in certified groups did States should obtain lists and con- group. Areas experiencing high levels receive notices about TM but states tact workers who are laid off follow- of dislocation, and special projects 'acked a systematic approach to iden- ing certification, and ask firms to tended to be the most responsive. tifying and notifying potentially eligible notify state agencies of subsequent workers. The third report in the series on ED- layoffs, following up with employers WAA was based on a survey of sub- to assure that this occurs. state areas delivering EDWAA servic- a States should use Ul wage record es. It addresses both process and and other data as a basis for identi- fying potential eligibles. outcome issues. The results regarding service integation were particularly a States should disseminate informa- tion about TM through brochures useful. Those substate areas with the and the media. greatest integration of services with JTPA IIA tended to have these charac- issue Area teristics: Application to TAA 93 They served workers whose profiles Conclusions Recommendations were similar to the economically Just as certification and notification Group Orientations tO TM would disadvantaged. were occurring earlier in workers' strengthen movement toward the spells of unemployment, so the appli- use of TM services. aThey utilized OJT as the primary cation process had been moved for- training option, but provided less re- ward. sponsive services. Issue Area s They provided weaker early inter- Delivery of Reemployment Services vention services. Conclusions Recommendations At this stage in EDWAA evolution, Substantial numbers of TM clients re- Staff needs to assist workers in incentive/sanction systems were in ceived long-term occupational training viewing reemploymentoptions more place in several states, and the evalua- at a variety of public/private institu- realistically. tion indicated that performance incen- tions. Nearly ali clients received Em- II Employment Service staff should ployment Service job search services. tives appeared to increase substate ef- become more informed aboutlocal/ forts to obtain positive outcomes. Also, A small number received payments to regional employment opportunities eight of the study states provided finan- finance job search outside local ar- and appropriate training. cial incentives for providing long-term eas, and some received relocation al- lowances. Clients rarely used avail- TM programs should work more training, which had the effect of in- able remedial education or OJT. closely with unions. creasing the intensity of that training. States should offer placement ser- The main recommendations based on Some workers strongly resisted re- vices as well as Job search servic- the survey focused on developing re- ceiving training. Staff sometimes es. lacked a detailed knowledge of labor sponsive services, organizing services a States should consider consolidat- market conditions and skill needs, and ing TM and EDWA.A. more effectively, and imvoving state therefore were vulnerable to making policies. These rewminendations mir- inappropriate decisions about training rored many of those summarized in the options. revieNi here of the first study. Most states placed little emphasis on Evaluation of the EDWAA post-training placement services. Mandated coordination between TM Farmers and Ranchers and EDWAA was working well al- Demonstration though few sites actually integrated A major demonstration project fo- service delivery. cusing on rural dislocation, this pilot project involved Tour states. The evalu- ation was based on intensive site visits and produced insightful case studies. The demonstration was critical in that little has been known about the

28 Evaluation Forum Issue 11 Summar 1995 :it

Features National Studies of Dislocated Worker Programs

EDWAA Eveluati-4'ccl.t.rsiOrls Issue Area s Implementelon Problems Economic developmedfunds were sometimes used to re- Funds were underused given the extent of the needs, and train workers to stave off layoffs but not to serve already-dis- the services were received by only a small proportion of dis- located workers. Sometimes EDWAA funds were used to located workers. Job search assistance and short-term train- support economic development projects. ing were overemphasized and rapid response capacity was often weak. The requirement for matching funds had unin- Issue Area Coordination with Other Programs tended effects on the type of services provided and the kind Coordination occurred mainly through joint efforts to deliver of workers served. services, particularly training. Sometimes EDWAA and feder- al vocational education funds, or EDWAA and federal finan- Issue Area m EDWAA Resource Use cial aid funds were used together. Title IIA programs and ED- Most states selected Service Delivery Areas (SDAs) as the WAA were frequently co-located administratively. Linkages organizations to receive substate EDWAA funds. States with other human service agencies were considered weaker, lacked adequate data for allocating funds that is, data on such as with welfare agencies. plant closings and mass layoffs, declining industdes, and farmer/rancher economic hardship. Only half the states felt issue Area Basic Readjustment Services resource allocation was consistent with the needs of the dis- Pre-layoff services, rapid response and basic readjustment located. services were often coordinated by a single agency. Some substate areas did not provide basic readjustment services if Issue Area Targeting dislocated workers were not receiving training. Local areas While most states used inclusive definitions of eligibility, tar- were struggling to develop appropriate assessment strate- geting priorities and methods varied considerably across gies for this target group. Only one-third of the substate ar- states. eas were using a case management system. Classroom training was often perceived by dislocated work- issue Area Service Delivery Systems ers as offering better reemployment opportunities than OJT. Most states supported the federal guidelines regarding ex- However, classroom training was highly localized and diver- penditure of 50% of funds on retraining, but few states en- sified. Community and technical colleges were the most com- couraged substate areas to request waivers for training or to _7".t.:;, mon providers. OJT positions had mixed benefits, sometimes reward the provision of long-term training. more oriented to economic development goals than to the Issue Area Local Program Organization and Design needs of workers. Few training options involved workplace- Some local areas partially integrated EDWAA with Title IIA based learning. service systems. Half of substate areas had focused on laid- Issue Area State Role off workers, one-third on the long-term unemployed. The ex- The requirement that EDWAA funding be passed through tent of dislocation had directed these priorities. Most sub- states to substate areas tended to weaken the state's leader- state areas complied with the federal emphasis on retrain- ship role in setting program parameters and assuring ac- ing; about half (those with high levels of dislocation) had countability. In retaining 40% funds, however, states frag- emphasized long-term training. mented the EDWAA delivery system, leading to eolicy confu- However, some administrators felt this emphasis denied the sion and a lack of policy coordination.

importance of basic readjustment services, and led to an ...... overuse of OJT. Over half of the local areas emphasized Issue Area Program Accountability classroom training; however, about one-third emphasized Most states did not provide much technical assistance to OJT. Most offered only limited supportive services. Even substate areas. Reviews of reports to determine compliance fewer provided needs-related payments. Most states en- with regulations focused on procedural rather than program- couraged participation by labor and management but only a matic issues. All states had fiscal expenditure reporting sys- third had created formal labor-management committees. tems to insure compliance with federal fiscal rules, but states were not consistent in recapturing funds from delinquent Issue Area Coordination with TAA, ES and Ul SDAs. Most states had nonfinancial agreements with ES and Ul, States tended to retain a strong role in designing and admin- usually negotiated at the local level. Nearly two-thirds of istering 40% activities, budgeting for a variety of activities. states provided EDWAA 40% funds to ES/UI for rapid re- Over half used these funds to contract for specific services to sponse activities. dislocated workers statewide. Issue Area Coordination with Economic Development In terms of programmatic accountability, EDWAA perfor- Coordination tended to be limited to Information sharing re: mance standards had little meaning, since no incentive or potential and actual layoffs. A few states used 40% funds for sanction system had been developed by states. Few had special coordination activities. Even so, companies' layoff even developed criteria for reviewing and approving substate plans were frequently too far in advance of WARN notices to plans, allow for effective layoff prevention efforts.

Evaluation Forum Issue 11 Summor 1995 27 3 5 ee:

e-e2 _

Features . National Studies of Dislocated Worker Programs

FigUte 4. EDWAA-EVauition. RecOnin,ebdatio.ns program than in other decentralized na- tional E&T programs to respond to na- Substate capacity building should design of rapid response, on realiz- tional guidelines intended to maintain be supported. ing the benefits of reduced labor/ the program's consistency, comparabil- aResource management practices management tensions, and on im- ity and accountability nation-wide. 4 should be improved. proving the range and availability of While EDWAA's funding and organi- aRapid response must be assured. services that are responsive to dis- zational framework allowed more ex- located workers' needs. Labor and management should join perimentation, this experimentation did government in the design and im- States need to create a more coor- not always translate to progress in deal- plementation of service delivery. dinated statewide program, linking ing with dislocation. rapid response, , and aA coordinated use of program and early Intervention services, and co- The current renewed interest in fed- other resources should be encour- ordinating plant-specific projects aged. eral block grants consolidating federal- with ongoing local service systems ly-sponsored E&T programs, and the aThe coverage of the program to produce a more seamless deliv- should be extended. accompanying emphasis on states' and ery of services. local level rights in the human services, Longer-term retraining and reem- States need to improve efforts to ployment plans should be included has not yet benefited from a serious identify the needs of dislocated consideration of consistency, compara- in service options. workers and tailor services to those EDWAA administrators and service needs, sharing "best practices mod- bility and accountability issues sur- providers should concentrate on the ele with other states. rounding this proposal. It is possible that in attempting to resolve the issue of duplication and program prolifera- needs of this target group, despite the The four projects varied greatly in tion in the federal system, we may un- dramatic decline in American farms the way funds were used and in the wittingly produce a new form of human service system fragmentation at subna- over the 1980s. Ten percent of the na- average cost per client. tion's farms had disappeared within a tional levels. Under the block grant few years. Historically, urbanization Half of all participants were still policy suggested as of the writing of and the consolidation of farming activi- fanning three months after program this article, states, substate areas and lo- ties had led to a loss of more than four termination; two-thirds were en- cal communities will be encouraged to million farms over the past sixty years. gaged in off-farm employment, design and implement their own ver- But the farm crisis of the 1980s was ex- many having had this kind of em- sions of an E&T system, assisted only by very general federal policy and ceptional in that the movement out of ployment prior to program participa- oversight guidelines. This is likely to agriculture in this decade was no longer tion. mainly voluntary. This unprecedented result in programs whose operation and dislocation was having severe effects Participation appeared to increase results cannot be compared appropri- on agriculture-dependent communities. employment rates, hours worked and ately with those of others, thus denying The demonstration was to identify the hourly wages, but only modestly the Congress the basis for improving employment and training needs of this and there was no evidence that any national E&T policies. group of workers and determine how project was any more effective than Meanwhile, there are important les- best to meet them. another in delivering services or in sons from the EDWAA evaluations. The main findings of the evaluation of the outcomes. We suggest a few: the demonstration were these: Case management systems seemed Programs focused on dislocated Participants tended to be in severe well-suited to this group, and re- workers must be =usually sophisti- financial distress despite high levels training appeared to be as appropri- cated and professional in their re- of education and off-farm jobs. ate for this group as for other dislo- cruitrnent and selection process. Recruitment goals were met despite cated workers. a They should try to intervene prior to the reluctance of farmers to partici- or immediately after layoffs. Commentary pate in employment and training They should provide a wide range of programs. The evaluations of EDWAA reveal income support and service options. the substantial variation across states Outreach efforts appeared to in- and across substate areas in the way na- They should construct individual crease enrollment. tional programs may be implemented. service plans with clients that are Over 80% of the participantsre- Since EDWAA funding policy gave based on competent client profiling, ceived retraining. greatest authority and discretion to sub- on a thorough knowledge of avail- state areas, it was more difficult in this able education, training and related

28 Evaluation Forum Issue 11 Summer 1995 36 Features National Studies of Dislocated Worker Programs

sus services and how to access them, on Other National-Level namic economy pressured by interna- the most up-to-date and comprehen- tional competition. 1 sive labor market information avail- Studies In the 1980s The Office of Technology Assess- able, on the key results of research and Early 1990s ment of the U.S. Congress produced a on dislocation, and on clients' own revealing report in 1987 on one of the strong preferences for assistance. Relevant to U.S. Department of Labor's traditional Dislocation programs for workers dislocated due to They should have vigorous case trade, Trade Adjustment Assistance: management systems to provide on- The problem of dislocation is, his- New Ideas for an Old Program. Re- going support to clients as they take torically speaking, a recent one. The in- viewed intensively in an earlier issue of advantage of the program's services crease in international economic com- Evaluation Forum, this report provided and benefits. petition and rapid changes in insights for revising the TAA program technology have led to unprecedented that contributed directly to later interest These suggestions require a highly business and industrial restructuring in consolidating TAA and EDWAA. trained staff with access to critical data- with long-term effects for workers. Lit- The authors praised TAA's conceptual bases, service networks, and other erally hundreds of analyses and evalua- design for its income-supported long- sources of information relevant to the tions related to the issue of worker dis- term training, and EDWAA's adminis- functions required of them. They must placement/dislocation due both to trative flexibility and broader coverage be able to invest employers and unions ordinary business cycles and to restruc- as well as its quicker and more compre- in becoming an integral part of the de- turing have been sponsored over the hensive service response, and suggest- sign and implementation of these pro- past decade, testifying to the serious- ed that the best features of the two pro- grams. They need to be generally fa- ness and importance of this phenome- grams be combined in a new one. miliar with the characteristics of non in all postindustrial economies. evaluation research so that they are Often overlooked, the report alerted This article alerts us to a few of these competent consumers and contractors policymakers to another aspect of TAA studies and reports. of research on their own programs. it is virtually the sole federal program to provide sustained, intensive technical Selected U.S. General With regard to service options in assistance to small and medium-sized dislocated worker programs, the DOL Accounting Office (GAO) Studies manufacturers. Interestingly, the OTA re- monograph A Guide to Well-Developed A series of studies relevant to dislo- port viewed the negative results of a pre- Services for Dislocated Workers is a cation was commissioned by the U.S. vious evaluation of TAA by DOL's In- veritable gold mine. This guide was in- Congress between the late 1980s and spector General's Office as a less than tended for EDWAA practitioners at all the present. Figure 1 provides a brief fair judgment of its overall effectiveness. levels of government. Information for overview of their purposes and results, Mathematica Policy Research pro- the guide was obtained from 70 sub- and the GAO's main recommendations. state areas and 10 special projects in 24 duced an informative report for the states, and focuses on the design, orga- Selected Studies in the 1980s U.S. Department of Labor and the New nization and provision of services to Jersey Department of Labor in 1989, dislocated workers between Program Stephen Baldwin's report, Technol- The New Jersey Unemployment Insur- Years 1989 through 1992. The mono- ogy and Employment Policy, commis- ance Reemployment Demonstration graph provides detailed information on sioned by the National Commission for Project. This project examined the use the following service options: Employment Policy in 1987, was a of the Ul system for identifying dis- careful expose of the effects of techno- placed workers early in their spell of Early intervention services. logical change on jobs and workers. As unemployment, and the provision of Services to help clients develop ap- a form of structural change, or restruc- early intervention services to speed re- propriate reemployment plans. turing, technology effects are manifest- employment. The net effects of three ed, Baldwin demonstrated, in worker "treatments" were studied: 1) job Basic readjustment services. displacement, retraining, and organiza- search assistance only; 2) job search as- Retraining services. tional changes internal to firms for the sistance combined with training and re- purpose of taking better advantage of location assistance; and 3) job search Although this extensive guide was new markets and production processes. assistance combined with a cash bonus designed for program operators, it is an The benefits of technological change, for early reemployment. All three strat- excellent resource for policymakers, Baldwin pointed out, are not evenly egies resulted in net benefits to both in- planners, oversight personnel, and eval- distributed across workers, firms and dividuals and the society at large. uators. communities. Therefore a major policy implication of such change is the need Selected Studies in the 1990s for effective training and retraining Another useful report by OTA was programs that provide workers with the made available in 1990, Worker Train- skills needed to be productive in a dy- ing: Competing in the New Internation-

.11141110.10. Evaluation Forum Issue 11 Summar 1995 29 ee'ce

Features National Studies of Dislocated Worker Programs

Fig.tir).. 'Selected .dACIFlep.ort.

Purpose and Type of Study Process study: to obtain information on services provided to dislocated workers by TAA and EDWAA In three states.

Results Major Recommendations TAA was serving a larger proportion of hard-to-place A better information systere for both programs should workers than EDWAA. be developed, to enable politymakers to determine TAA participants received more income support and what services are received and what outcomes are were more likely to enter long-term training. achieved. Both programs provided classroom training and OJT, but TM and EDWAA should be consolidated and serve all EDWAA placed more emphasis on OJT. dislocated workers, Irrespective of the source of dislo- cation. Both programs lacked comprehensive Information on participant outcomes, but particularly TM. income support should be provided with retraining. There was only limited coordination between the two Early intervention is essential. programs, partly due to complex TAA regulations. A larger menu of assistance options should be avail- able. ..

Results Major Recommendations Flaws were found in 63% of the petitions filed. Major Certification process should be made more accurate, flaws were: and information more adequate for determining incompleteness and inaccuracy of data collection from elgibility. companies.. States need to be more responsive to workers filing Inadequacy of analyses of trade statistics. petitions. inadequate customer surveys. a TM and EDWAA should be consolidated, and the need Variations in volume of petition filings related to level for certification eliminated. of outreach and assistance provided to workers wish- ing to file petitions.

' -: iffkift"WitatfilQatidW011014FS0 kerlAd atendfratilf ofillcation'Act ARAI) t ealiteitiljeti9gisgteitl2 Purpose and Type of Study Process study: to investigate how well the WARN system was working in 11 states.

Results Major Recommendations While workers were more likely to receive prior notice of a DOL should be given the responsibility and authority to closures and layoffs, nearly half of the employers with vigorously enforce the WARN provisions. 100+ workers were not required to give notice. Some employers appeared to be confused about WARN provisions, even though DOL brochures and seminars sought to educate them. Many employers believed WARN benefited dislocated workers in terms of their reemployment but decreased company productivity.

Figure 1 continues

30 Evaluation Forum Issue 11 Summer 1985 38 Features National Studies of Disioatted Worker Programs

Purpose and Type of Study Outcome.study: testimony summarizing DOL studies, such as inspector General's and Mathematics's,and their impli- cations for policy. Results Major Recommendations need to be assisted systemencelly TAA was not achieving its key goals: a Dislocated workers and conaistently irrespective of the source of disloca- Benefits not equally accessible. tion i.e., whether dislocated under TAA, EDWAA, De- Slow program respanse. fense Conversion/Diversification, Clean Air, Tranaltion Services limited and not well tailored to needs. Assistance Program, Demonstration Centers fa.; Re- Not enough counselln3. training Dislocated Workers, etc. c Waiver provision led to only half of TAA recipients Programs need to be consolidated. receiving training. Limited job placement assistance atter training. inadequate Information system for determining perfor- mane% Separate funding sources have led to separate rules/ regulations which act as barriers to quick, equitable, comprehensive service provisions. lilt;;;A , Purpose and Type of Study Process study: to explore the implications of the new re- employment program Initiatives that specifically target dis- located workers (in an amendment to TAA considered a bridging" program prior to legislation consolidating dislo- cated worker programs). Results Major Recommendations The program had many of TAA's shortcomings. Processes should be developed that eneure that the certification process is quick and fair. Activities in addition to claseroom &raining, such as job search assistanee, should be allowable In considering eligibility for incornu support.

...... , kat ,ipttazattgi 24,3g110402,sej,jitt-m,,,,,,221,41,09,4.14.11.1PAIIMIIIMONOMEMaaraa Purpose and Typo of Study Process/limited outcome study: to learn more akout the im- plementation and outcomes of assistance programs relat- ed to the dislocation of timber workers .n northernCalifor- nia in the late 1970s. Results Major Recommendatione Many workers received wage replacement benefits or Receipt of benefits should be closely tied to partic:pa- severance pay, but few received training. tion in training. Service providers were reluctant to develop retraining New job opportunities must be created in communities affected by major layoffs, in addition to providing re- programs. training and income support. Data on participant outcomes were not adequate enough to determine what happened to workers after receiving services. Data on outcomes for communities were not sufficient to sort out the effects of the programs. Figure 1 continues

INOMMTBIns, 31 Evaluation Forum issue 11i Summer 1995 39 ;

sItt.

Foatures Nadonal Studies of Dislocated WorkeEilmms_____

Purpose and Type of Study Process study: to review in four states DOL's effort to make changes in this transitional program (modeledon TM) to 1) shorten the certification process, 2) include stales In that process, 3) broaden eligibility regulations, 4) tie income support more closely with retraining, and 5) eliminate waivers.

Remits Major Recommendations s Mt had successfully addressed a number of shortcom- More consistent and comprehensive assistance should inge. be assured to all eligible workers. Secondary workers (those indirectly affected by NAFTA) a A better performance monitoring system should be de- still laced problems in accmseing benefits. veloped for participants' services and outcomes. a Some workers were still receiving incomplete assess- ments and remedial assistance, and a limitedmix of ser- vices. Performance management remained inadequate.

_ete al Economy. Also reviewed in more de- placed Workers for the association's to firms that were members of AMA, tail in an earlier issue of the journal, corporate members. The report indicat- the results were particularly useful this OTA study examined sixteen op- ed the following: since the study involved one of the few tions for training policy regarding the Forty-six percent of the firms sur- employer-based surveys conducted in needs of the economically disadvan- the 1990s. taged, displaced workers, workers with veyed reported downsizing within 3,4 special needs, and the general popula- the past year. The importance of an adequate and tion of employed workers. These op- The cuts made by these firms in- comprehensive occupational informa- tions were organized within four issue tion system was emphasized in David creased to an average of 10.4% of areas: 1) reducing barriers to company Stevens' 1993 monograph for the Na- training; 2) upgrading workers' skills; their workforces. tional Occupationalinformation Coor- 3) providing training and technology Sixty-six percent of the downsizing dinating Committee, Occupational In- assistance; end 4) enhancing the quality was due to companies' concerns formation: The "Blue Highways" of and effectiveness of training. The re- about future business downturns. the Labor Market. The paper pointed to port thoroughly analyzed these options, the utility of oecupational information referring to policy analyses and re- For the first time, a majority of the in helping users become more discrimi- search conducted about them, and rec- jobs eliminated by finns in thesurvey nating and skillful in selecting and ommended certain options for policy involved , middle managers, translating published statistics to meet consideration. professionals and technicians. The specific policy, planning, management number of firms offering outplacement and oversight needs, and in heightening A study conducted in 1993 by the services to dislocated workers was at a users' sensitivity to workplace changes Congressional Budget Office examined seven-year peak. The report confirmed they can see are occurring but about lessons learned in obtaining informa- that workforce reductions wereoccur- which they do notyethave confirming tion about the actual experiences of dis- ring more frequently. Fewer than half statistics. located workers. The report, Displaced , of the companies downsizing had in- Workers: Trends in the 1980: and Im- creased their profits or their worker Some Comments on the Studles plications for the Future, detailed the productivity as a result. The larger The recommendations in the reports number, characteristics and experiences companies were more likely to trim of a large saniple of dislocated workers, referred to involve something of a con- jobs than smaller ones, but small busi- sensus on the desirability of: 1) weld- and the implications of this information nesses cut more deeply when they did for federal policy. ing income support to longer-termre- downsize. Firms that had downsized training, 2) providing a comprehensive Another study in 1993 that provided since 1989 (70% of the survey sample!) set of service options, 3) consolidating insights about dislocation was spon- were cautious about hiring new work- programs so that all dislocated workers sored by the American Management ers and had tended to lengthen hours are eligible for a unified program, irre- Association (AMA). AMA conducted a and/or as an alternative. Few- spective of the source of their dislocation, survey regarding workforce reductions, er than half of all the firms surveyed and 4) increasing administrative flexibili- had increased their workforce since producing the 1993 AMA Survey on ty by removing regulatory barriers to ef- Downsizing and Assistance to Dis- 1988, Although the survey was limited fective program implementation.

32 40 Evaluation Forum aIssue it Summa 1005 Features a National Studies of Dislocated Worker Programs

centrated their attention on small busi- tional Ill-related projects testing the Self-Employment as ness development as an economic de- use of reemployment bonuses as incen- an Employment velopment tool, largely to the neglect tives for an early return to work for dis- of microenterprise development. located workers in Washington State Strategy for and Pennsylvania. And in the late Dislocated Workers Rather than serving economic devel- 1980s DOL funded the St. Louis Met- opment needs, self-employment pro- ropolitan Reemployment Project, grams in other industrial nations have The two major self-employment which was evaluated using a quasi-ex- been designed to expand employment perimental research design. projects funded by DOL in 1988, and opportunities. By the mid-1980s the reported on in 1992, in the monograph seventeen advanced industrial countries In 1987, the Department funded a Self-Employment Programs for Unem- who were members of OECD were of- project to test self-employment strate- ployed Workers, were targeted to UI fering formal self-employment pro- gies. Washington State was selected as claimants on permanent layoff. Howev- grams for the unemployed. According the test site. In the same year, however, er, entrepreneurial self-employment to the DOL report, by 1987 nearly 20% the Omnibus Budget Reconciliation strategies had been of major interest in of new business development in Great Act required the Secretary of Labor to Europe for some time, as an option for Britain was due to self-employment; al- test self-employment programs in a coping not only with cyclical unem- most 25% in France. multi-year project involving three ployment and underemployment but states. In this second effort to test self- also with dislocation due to restructur- At the time the DOL report was employment approaches, DOL, the four ing and downsizing of workforces. written two major self- employment states, and the research contractor joint- models had been developed in Europe: The Organization for Economic Co- ly designed the strategy to be tested and operation and Development, for exam- The "French Model," involving the evaluation plan. The design com- ple, housed key experts on programs lump sum self-employment allow- pleted in 1989 also drew heavily from throughout Europe that were utilizing ancs that were to provide part of the insights gained in a study of self-em- self-employment as an alternative to re- capital needed for a microenterprise. ployment projects in Great Britain, employment. OECD was also interested France and Sweden, funded by the Ger- in the distinctive challenge this option The "British Model," involving peri- man Marshall Fund. Concerned about involved for evaluators, sponsoring an odic payments as a form of income potential costs to states, two of the international seminar on the evaluation support while developing and oper- states withdrew from the project, leav- of "Self-Employment and Employment ating a new microenterprise, such as ing Washington and Massachusetts as Creation Schemes" in 1989. UI in the U.S. the leading test sites. The DOL report was very useful in Although the European effort was The Washington SEED Project suggesting how experimental evalua- not rigorously evaluated, descriptive tions can benefit policymakers and pro- information was suggesting that self- The Washington project followed gram designers in developing success- employment was a significant option the French model (lump sum payments) ful self-employment programs. It also for the unemployed and dislocated. while Massachusetts applied the British made an important contribution to the model (periodic payments). Both in- knowledge base regarding European U.S. Approaches volved evaluations using an experimen- entrepreneurial initiatives. tal research design requiring the ran- In 1985, DOL became interested in dom assignment of a pool of UI Self-Employment Programs testing new strategies for dislocated claimants into a group receiving self- in Europe workers, and using an experimental re- employment services and subsidies, search design to evaluate their effec- and a control group not receiving them. The DOL monograph distinguises tiveness. The major elements of these between "microenterprise develop- strategies were: 1) early identification In the Washington project, claimants ment," which is used synonymously and profiling of dislocated workers us- interested in the project, and potentially with "self-employment," and "small ing UI data, 2) referral of dislocated subject to random assignment, were re- business development," which is im- workers to providers of reemployment quired first to attend an orientation ses- portant to the definition of the enter- services, and 3) the linking of UI and sion as a first step in the application prise strategy DOL was testing and reemployment services within a single process. Of the 44,456 UI claimants in- evaluating. Microenterprises were seamless delivery system for these vited to participate in the project, only viewed as sole proprietorships with workers. The set of reemployment ser- 7% attended the orientation. Of the only one or a few employees. Small vices of greatest interest were job group attending the orientation, 61% businesses were defined by the Small search assistance, training, relocation subsequently applied and were random- Business Administration as having less assistance, reemployment bonuses, and ly assigned. Consequently, the project than 500 employees. The DOL report self-employment allowances paired was testing the self- employment strate- reminded us that microenterprises as with business support services. New gy on a small, self-selected group of well as small businesses were growing Jersey was selected as the first test site. claimants. Although the evaluation of rapidly. But pane policymakers at Later the Department funded two addi- SEED was featured in an earlier issue both the federal and state level had con- Summer 1996 Evaluation Forum Issue 11 41 Features National Studies of Dislocated Worker Pro rams

of Evaluation Forum, werepeatsome during the demonstration period, Participants interested in self-em- of the main findings here, which were and reduced total earnings from this ployment will more likely be male, reported in detail in DOL's monograph, form of employment. white, middle-aged and educated First Impact Analysis of theWashington than most U1 claimants, and from State Self-Employment and Enterprise Both gross business income and tax- Development Demonstration. es were significantly higher for the professional/technical/managerial treatment group. occupations. Women, however, are Program Implementation: The likely to be overrepresented in this Process Study The treatment group worked fewer select group of claimants. hours and obtained lower earnings The dislocated workers in the study in Ul-covered employment. The EDWAA Job Creation groups were older and more educat- Demonstration ed than the larger universe of UI If self-employment and wage/ claimants, and more likely to have employment were considered to- With the decentralization of the bulk been in higher-wage white collar gether, the demonstration increased of EDWAA funding to Service Deliv- ery Areas or local community organiza- cupations. Many had an employed the likelihood of being employed tions, the role of Community Develop- spouse and substantial assets. and remaining employed, but had no significant effect on total earnings or ment Corporations (CDCs) and other community-based nonprofits in deliver- Clients achieving five interim out- average monthly earnings. comes received a lump-sum pay- ing services to dislocated workers be- ment equal to their remaining UI Although the demonstration reduced came more important. DOL's multi- benefits. These outcomes were: the length of UI payments, it in- year Job Creation Demonstration was to study this role, and the evaluation 1) completion of the training pro- creased the amount of total benefits was to judge its effectiveness. gram, 2) development of an ap- when the lump sum payment is in- proved business plan, 3) the estab- cluded. The emphasis in this project was on job creation through self-employment. lishment of a business bank account, The evaluators also gave attention These were the main results of the eval- 4) compliance with licensing re- to and job creation for uation; ==I quirements, and 5) the obtaining of other workers. They found that the adequate financing for their pro- demonstration netted 49 new jobs for Participants' characteristics varied posed business. nonparticipants, although these went considerably from one site to anoth- mainly to family members, and there er, but overall they were older and The program treatments clients felt was no effect on job satisfaction. better educated than the typical dis- were most useful were business located worker population. training, the waiver of the UI work The Massachusetts Project The selection of applicants also var- search requirement, and the lump- This self-employment demonstration sum payment. followed the British model. Clients inter- ied greatly, ranging from selection ested in applying were required to attend strategies that involved sophisticated Program Results: The Net a training seminar consisting of a series screening to self selection. The latter Impact Study of training workshops and counseling led to a high attrition rate; the to The project increased the likelihood sessions. An additional condition was former was quite successful. that those receiving self-employment al- of being self-employed by about six Curriculum design using an "adult months, a bit more for women than lowances would have to work full-time on microenterprise development. learning approach," extensive staff for men. and peer support, and substantial Only about 4% of the 26,170 UI Both treatment and control groups technical assistance to individuals claimants invited to attend the seminar appeared to be critical to success. started primarily service sector busi- did so. Of this number, 63% applied nesses. and were randomly assigned. Again the Staff qualifications that included Approximately a third of these busi- study group was small and self-selected. practical business experience and a nesses failed in the first year, but the Although the DOL monograph on self- high degree of commitment were demonstration did not appear to af- employment projects was written prior to important. obtaining final impact results, the evalua- fect this outcome. tors did come to a few early conclusions: Development of business plans was an essential component of training. Average gross monthly earnings from It is likely that self-employment is self-employment were increased by appropriate only for an estimated Lack of access to revolving loan approximately $150/month. 5% of UI claimants. funds and other sources of capital The demonstration reduced slightly was the most serious barrier to II Self-employment businesses will achieving demonstration goals. the likelihood that clients would par- likely be in the service sector. ticipate in wage/salary employment Strategies for designing services and 34 1 42 Evaluation Forum issue 11 Summar 19175 Features National Studies of Dislocated Worker Programs

service delivery systems converged produced marketable skills even if Self-Employment in Europe across sites over the course of the they were not immediately applied to The section of the DOL monograph demonstration, but organizational starting a business. on self-employ ment directs our atten- tion to "European Experience with problems interrupted services and Earnings from self-employment Self-Employment Programs," by led to inappropriate hiring decisions, were generally disappointing. Earn- poor communication with partici- Stephen Wandner and colleagues. It ings varied widely from business to provides a useful if brief overview of pants, and reduced efficiency. business, and net business income European strategies. Although these Coordination with EDWAA was averaged only about $1000/month strategies have undergone policy mixed, whereas the CDCs worked when measured at the six month fol- changes with increasing unemployment well with the UI system. However, low-up and $582/month at the one- and slimmer government human ser- the specter of UI sanctions affected year follow-up. These self-employ- vice budgets, it is important to review data collection for the evaluation, ment earnings were much lower than the strategies that characterized the late since some participants did not re- the $12 average hourly wage the par- 1980s. But we should notice that Cana- ticipants had earned prior to layoff. da and Australia have also been inter- port their previous self-employment ested in self-employment approaches. experiences and understated their Pre-post studies of gross incomes can earnings from businesses begun as be misleading. However, on average, These were some of the characteris- part of the demonstration. it appears clear that dislocated workers tics of European self-employment pro- tend to earn less in their new jobs. grams in 1988: In the short term, the demonstrations Their primary purpose was to pro- were successful in launching new The outcomes for demonstration mote self-employment (microenter- businesses and creating secondary participants who did not start busi- prise development) among the un- employment 45% of participants nesses of their own but rather sought employed, to contribute to solutions started up businesses and 29% found reemployment were more positive, to unemployment. wage and salary employment. matching those of the more tradi- Among the businesses begun, 76% tional EDWAA programs in the The target groups of these programs were still in operation a year later. same areas 79% entered unsubsi- usually involved those eligible for dized employment. income support payments through Comparing the Job Creation Dem- national unemployment insurance onstration with the Washington The evaluators made a number of programs, but some included the State and Massachusetts UI demon- recommendations. If immediate wage long-term unemployed and those re- strations and flve other projects sug- replacement were to be a participant's ceiving social welfare benefits. gested that the demonstration's primary goal, they suggested that self- employment did not appear to be a fea- gross outcomes, such as business The services ordinarily included fi- sible alternative in the short term. If start-up and short-term survival participants and program sponsors were nancial assistance and supportive rates, were on a par with other mi- prepared to absorb a certain level of services. The definition of "support- croenterprise programs undertaken risk, microenterprise strategies were ive" was services designed to assist in the same general period. recommended as offering longer-term workers in planning, establishing and operating the businesses they Service industries were the most benefits. were creating. Financial assistance prevalent type of start-up business- For a small subset of dislocated was offered in 1) lump-sum pay- es, even though only 22% of the par- workers, who were aware of the risks ments equal to all or part of the ticipants had been dislocated from and the effort required, were highly funds remaining in an individual's this sector. motivated and prepared to develop a specific business concept, self-employ- insurance entitlement, which was to Comparing employment outcomes at ment was recommended as a viable support initial capitalization of his follow-up with those of EDWAA strategy. But even workers with these or her business, or 2) periodic weekly substate areas indicated that the qualities needed access to capital as or bi-weekly payments from the insur- demonstration had not produced as well as training to be successful, the ance system to give a worker continu- beneficial employment outcomes as evaluators concluded. Since a self-em- ing income in the early stages of plan- ployment strategy was distinctly differ- were characteristic of EDWAA. But ning and operating a new business. when both placements in wage and ent from a reemployment approach, the evaluators recommended that current The payments often lasted for a year. salary jobs and business start-ups were performance management systems uti- A comprehensive package of servic- considered, the six Job Creation sites lize a set of variables and measures es included a number of business-re- had an average employment rate of more responsive to the uniqueness of lated services, although the ingredi- 74% compared to EDWAA's 67%, microenterprise development, in judg- ents of the package varied greatly The evaluators concluded that the ac- ing its outcomes. from country to country. These ser- quisition of entrepreneurial training 35 Evaluation Forum Issue 11 Summer 1095 43 _ _

Features National Studies of Dislocated Worker P ams

vices encouraged business counsel- programs wherever found, based on an the JOBS program. ing, entrepreneurial training, techni- expert panel developed by the Organi- In considering what to propose as a cal assistance, exemptions from zation for Economic Cooperation and Development (OECD) in Paris. common methodology for evaluating certain business taxes and legal re- diverse self-employment programs, the quirements, and preferential access The conclusion of the authors, who authors turned to the OECD panel. This to business loans and grants outside remain unamed in the paper, is that group of experts identified six major is- the programs. most European programs have been sues that could and should be studied: a In some countries, microenterprise studied using a descriptive research de- sign and survey methods, particularly How important the provision of development was limited to certain surveys of participants. Periodic sur- government assistance for business economic sectors, and to special cat- veys have formed a longitudinal data- start-up really is. egories of the unemployed. base for analyzing process issues and To what extent business ventures Based on selective gross outcome gross outcomes. generated by programs would have studies, both the French and British In general, however, European E&T occurred in their absence. models appeared to have a positive ef- programs were not systematically eval- fect on employment and economic de- uated in the 1980s. Of the 17 OECD Whether the methods used to target velopment. Compared to the actual countries with operating self-employ- services to subgroups within the eli- numbers of unemployed workers ment programs, 10 had not produced gible population are effective. served by American demonstration evaluation reports. Most of the evalua- How long program effects seem to projects, these programs enrolled large tions conducted had been limited to last. numbers of the unemployed. In France, process issues, in the context of weak the Chomeurs Createurs program en- public sector support for and experi- How cost-effective programs actual- rolled 72,000 in 1986. The British En- ence with impact evaluations. In the ly are. terprise Allowance Scheme exceeded E&T field, this may have been related 100,000 participants in 1987. However, to the tendency for European universi- a How programs affect potential fu- the prediction of the evaluators study- ties toproduce manymore excellent ture dependence on unemployment ing the U.S. demonstrations was con- theoretical economists than applied insurance and other income support sistent with the French and British ex- econometricians. programs. perience, namely that under 5% of the unemployed were appropriate for self- In their discussion, a major differ- Appreciative of these issues, the au- employment programs. ence surfaces between contemporary thors of the paper suggest a very simple American evaluation approaches to evaluation paradigm. Program elements Increasing familiarity with self-em- new program initiatives and European are classified either as "inputs" or "out- ployment efforts in Europe led to sub- orientations. With respect to self-em- puts," and the evaluator's task is to de- stantial transnational borrowing of self- ployment programs, for example, Euro- termine how le inputs affect the out- employment concepts. Enough so that pean governments have not been under puts, or at least what kinds of the National Governors' Association pressure to test new ideas through dem- relationships exist between them. In- and the Corporation for Enterprise De- onstration projects. Programs there are puts are: 1) targeting services, 2) vet- velopment jointly sponsored a national more often than not the products of ting service (selecting among the busi- conference on self-employment strate- government agency policies that tend ness plans submitted), 3) financial gies in 1987. The conference brought not to be dependent on the results of services, and 4) support services (an ar- experts from Europe, Canada and else- evaluations. ray of technical business-related servic- where to address self-employment is- es). Outputs for participants are em- Under public pressure to establish sues at the time Americans were tour- ployment and business retention. accountability, aided and abetted by the ing European programs under the Outputs related to economic develop- elaboration of the evaluation research German Marshall Fund. The borrow- ment goals are the generation of eco- ing, however, was mainly by Ameri- field, the pattern in the U.S., particu- larly beginning in the 1980s, was to nomic development activities and the cans from Europeans. creation of new jobs. test innovations or reforms in well- Evaluating Self-Employment planned small-scale pilot projects that Emphasis is placed, in this para- Programs in Europe could be evaluated using experimental digm, on 1) the adequacy of databases or quasi-experimental research designs supporting evaluation, including infor- The Appendix of the DOL mono- prior to full-scale adoption. Exceptions mation on the background and charac- graph contains an interesting UI Occa- existed, of course, such as JTPA, al- teristics of clients, information about sional Paper on "Self-Employment Pro- though its design benefited indirectly the services received, and performance grams for the Unemployed: Program from previous E&T research on CETA indicators, and 2) the collection of data Operation and Policy Research in and other programs. And a large num- over time. The authors also suggest a Western Industrialized Countries." Its ber of experimental evaluations of series of indicators which they classify intent is to propose a common method- work/welfare demonstrations preceded within these categories: 1) employment ology for evaluating self-employment and were fed directly into the design of for the proprietor, 2) enhanced business 44 Evaluation Forum Issue 11 Summer 1995 Features Defense Conversion and Dislocation

bored to the individual and his or her survival rate, 3) contribution to the lo- main lessons are that self-employment business, with ongoing case manage- cal and macro economy, and 4) new job is most appropriate for a very small, se- ment that emphasizes technical assis- creation. They appropriately recom- lect group of dislocated workers who tance as well as other kinds of support. mend the use of measures of job dis- are highly motivated, disciplinedand placement. They review problems with planning-oriented in the way they go This is a somewhat different concept descriptive and quasi-experimental re- about developing businesses supported of microenterprise development than in search designs and advocate for experi- by government income support and Eastern Europe and third world coun- mental approaches in determining pro- training. These program participants tries, where it is offered as a significant gram effects. must also be risk-takers of a sort, will- intervention for large groups of unem- ing to try something new with all the ployed workers. For example, in the Commentary on the Wisdom of attendant anxieties. The results from early 1980s the Chinese government Self-Employment Strategies for the final report may shed more light on stressed self-employment as a major di- Dislocated and (Ahem/Ise the effectiveness of self-employment mension of the liberalization of the Unemployed W4rkers strategies. economy, only to find that business We have learned a great deal from Therefore it looks as if effective re- failures outnumbered successes five self-em- the projects generated by the increase cruitment and screening strategies will years later. Carefully targeted in dislocation in the 1980s and the U.S. be essential to install at the front end of ployment programs, on the other hand, government's strong response in ex- microenterprise development programs. appear to have contributedeffectively ploring a range of service options using Ways to sustain motivation and disci- both to job creation and to economic state-of-the-art evaluation research pline will surely be important. And it is development. principles and methods. Some of the fairly clear that the training must be tai-

Defense Conversion andDislocation flints, and about the Impact ofdefense EditOrial Introduction to the resolution of pressing societal problems. But the changing budget for- restructwing on military personnel, de- tunes of the Department of Defense fense-dependent communities, and de- Cutbacks in the defense budget re- fense-related businesses and indus- lated to the end of the Cold War have have not yielded all the positive effects tries. The following set of articles been heralded by the American public anticipated. Anxiety remains about de- fense readiness in a world complicated explore some of the issues involved in as a bonus, a symbol of a less danger- this phenomenon of the 1990s. ous world, a chance to transfer savings by ethnic, religious and territorial con-

Role of Education in Restructuring De- Background Information Defense Downsizing: fense and Other Industries reminds us At the end of fiscal year 1987, the Some Implications that the U.S. Department of Defense active duty military was at a post-Viet- has been and remains the nation's larg- nam peak of 2,174,217 positions.The The Cold War ended in dramatic est employer. Furthermore, as an em- National Defense Authorization Act of changes in the political economy of de- ployer it is known for its integration of 1991 projected a strength of 1,613,000 fense in the U.S. Its demise spawned a education and training with the culture positions by the end of 1995. This was new vocabulary as well tet ms such of work, and its incorporation of new about a 25% reduction. By 1997, De- as "drawdown," "downsizing,""con- technology within skills acquisition. partment of Defense outlays were ex- version," and "transition."1 The impli- Given rather unique opportunities pected to decline by approximately cations of the end of the Cold War for for cducation, training and retraining 30%. This could represent a loss of as workers affected by reduced defense within the military, unprecedented many as 800,000 military andcivilian spending has been equally dramatic. drawdown and downsizing in the 1990s Department of Defense (DOD) jobs be- The U.S. Department of Education's represent an unusual test of the rele- tween 1987 and 1997, and could affect 1993 report, The Conference on the vance of military training for thecivil- as many as a million additional private ian economy. Since these changes have sector jobs over the next several years. Me Defense Conversion Commission with- involved a major dislocation of work- con- Although the U.S. has always lost in the U.S. Department of Defense defines ers within and outside the military,in- versionas "the process by which the people, and replaced large numbers of military skills, technology, equipment and facilities of novative strategies have been essential personnel every year, this significant defense are shifted into alternative economic ap- to reducing the negative consequences multi-year downsizing effort has trans- plications." It views conversion as "a process of of dislocation. transition." 37 Evaluation Fortum issue 11 Summer1995 45 -

Features 2 Defense Conversion and Dislocation

lated to far greater personnel change personnel, which is important. But we recommended a set ofprinciples and than is traditional, with the potential for still have issues to resolve regarding in- criteria for evaluating these programs, a high level of involuntary separations. voluntary separation. The GAO report with strong emphasis on specifying ob.. Clearly there has been a long-term re- recommended that the Congress con- jectives and using quantitative mea- structuring of the nation's defense pri- sider extending the use of financial sep- sures to judge how well these objec- orities. As a result, the Congress autho- aration incentives beyond 1995, and tives were being achieved. Greater rized a range of assistance efforts in the amending the legislation to include in coordination across old and new pro- 1990s to facilitate downsizing, while the eligible pool for incentives all per- grams was given high priority. Overall, minimizing its impact on the individu- sonnel who have attained six years of the Commission felt that the conversion als, businesses and communities affect- service within the timeframe for which process was, in itself, an opportunity to ed. these incentives are authorized. make improvements. The first priority of the Defense De- This upbeat attitude was reinforced partment has been to achieve voluntary The Role of the Defense Conversion Commission by the Commission's computer simula- reductions. It has successfully drawn tions regarding the longer-term conse- down military forces across various The creation of the Commission quences of the drawdown. These were skill and seniority sectors using strate- within DOD was an important step in generally optimistic. Analysts predicted gies that will have the least negative dealing with the issue of dislocation. It that the drawdown's main short-run ef- impact while maintaining the level of provided a framework for monitoring fect would be a slight downwardpres- experience needed. However, as the and judging the process and conse- sure on output and a temporary in- U.S. General Accounting Office (GAO) quences of the defense drawdown. It is crease in unemployment. But if defense pointed out in their September 1993te- a key source of recommendeons for spending were to be replaced mainly by port, Military Downsizing, significant resolving problems, and for embracing investments in plants and equipment, additional reductions will result ina new possibilities associated with con- human capital, and research and devel- larger proportion of involuntary reduc- version. opment, the prediction was that eco- tions. A recent Bottoms-UP Review of nomic growth might actually speed up DOD needs and programs for 1995 The Commission's 1993 report, Ad- over the longer-term. Since general through 1999 supports this conclusion. justing to the Drawdown, explains its economic growth has been such an im- We should keep in mind, however, that assessment goals and methods, which the impact of this current drawdown is now provide general guidance to feder- portant tool in reducing the impact of al efforts to make the conversionpro- drawdowns, the Commission felt the considered by experts to be smaller Administration and the Congress than that of the drawdowns after the cess work. The Commission's four goals are: should give special attention to invest- Korean War and Vietnam, and less se- ment tax credits, banking reforms, defi- vere than other recent economic dislo- 1. To facilitate the transition by encour- cations. cit reduction, education reform, there- aging economic growth over the rnoval of trade barriers, tax reform, and long run. Strategies for Limiting increased spending on infrastructure. 2. To preserve defense capabilities. involuntary Separation Assisting the Dislocated Early release of personnel prior to 3. To ease the impact on workers, com- The complex network of assistance the normal end of their enlistments has munities and companies. efforts funded by the Congress in the been one strategy. Another has been fi- 1990s is very difficult to sort out. Em- nancial separation incentives to induce 4. To improve government policies and ployment and training administrators, people to leave. The Defense Authori- programs in terms of their efficiency zation Act of 1992 and 1993 supported and effectiveness. managers and evaluators clearly need to learn more about the many federal the use of financial incentives through The Commission recognized thateco- efforts to assist the defense conversion 1995 for those who had completed six nomic growth was a critical factor in transition. And these efforts need great- years of service at the time the law was facilitating conversion and alleviating er integration. enacted. New early retirement options its negative effects, and took the posi- have been used also. On the involun- tion that this growth should come from In the meantime, one important tary reduction side, DOD has tightened redirecting defense resources into in- source of information is du. Office of the quality standards for retention and vestment rather than consumption. Economic Conversion Information has created special boards thatcan (OECI). This office is the result ofa mandate the retirement of selectedper- With respect to easing the impact of collaborative effort on the part of the sonnel from among those already eligi- conversion, the Commission felt broad- Economic Development Administration ble for retirement. er use needed to be made of integrated and the Economic and Statistics Ad- community-level planning, encouraged ministration within the U.S. Depart- The careful shaping of a smaller and supported by the federalgovern- ment of Commerce, and the Office of military force has favored the retention ment. To improve existing government Economic Adjustment in DOD. The of the more skilled and experienced assistance programs, the Commission OECI manages a major information

Evaluation Forum Issue 11. summer 1925 4 6 Features m Defense Conversion and Dislocation

clearinghouse on defense conversion live duty military, and in the DOD ci- large-scale dual-use conversion relevant to workers, communities and vilian sector. This was 5.1% of total projects, believing that the differences industries. This automated system pro- national employment. The authors pro- between defense and civilian cultures vides information on current press re- pose that as many as 2.5 million de- and practices are too great. Producing leases and legislation, publications and fense-related jobs could disappear by components and subsystems rather than resource materials, worker adjustment 2001. They point out, however, that not final products, the report states, is more programs, and related issues. all of this shrinkage means actual job likely to launch dual-use production. loss, since approximately three-fourths Government technical, marketing and Another source is the Logistics of this downsizing will be through at- financial assistance is viewed as impor- Management Institute (LW), a private trition. tant in helping firms move into more nonprofit research and development or- commercial activities. ganization established in 1961 and The larger problem, they suggest, is funded by DOD. Its new Center for De- the extent of employment losses in any A second OTA report in 1993, De- fense Conversion and Economic Re- one period, large declines in any given fense Conversion: Redirecting R&D, newal produces policy, economic, and period making readjustment more diffi- also offers useful insights and recom- fiscal management analyses and offers cult. Eight of the fifty states are above mendations about converting defense access to a number of databases on eco- average in their dependence on the de- industries and their research and devel- nomic adjustment challenges. The data- fense industry. And even with the opment efforts. The latter is a signifi- bases focus on two sets of issues: de- present level of economic recovery, it cant issue in that the defense complex fense activity, and federal, state and is hard to replace the well-paid jobs de- has supported a disproportionate share local adjustment programs. fense manufacturing has offered. There of U.S. research and development, is also the issue of minorities. Of all some of which has important civilian Congressional funds for assistance employed black men between eighteen applications. It has established major programs have flowed mainly to the and twenty-nine, 10.6% are in the mili- civilian industries, and has afforded DOD and from there to other relevant tary compared with 5.4% of white high-quality, well-paid jobs. Some of federal agencies, such as the Depart- males. The shrinkage of the defense es- this R&D provided a large market for ments of Labor and Commerce. One of tablishment may represent a major loss technologically advanced goods and the Logistics Management Institute's of employment opportunities for minor- services. Relevant to these spinoffs, the major assignments in providing input ities. report admonishes the government not to the Defense Conversion Commission to waste the valuable human, institu- has been to prepare an exhaustive Com- The Larger Picture tional and technological resources for- pendium to Assist the Transition. This merly assigned to defense purposes. compendium provides a description of The OTA report also indicates that each program, its sponsor, the organi- communities at serious risk due to de- This second OTA report also sug- zation responsible for its implementa- fense cutbacks are not receiving as gests ways to re-use these defense-re- tion, and the amount of its funding. much assistance as they often need. lated resources, such as the develop- This information helps considerably in Federal economic development pro- ment of new environmental protection sorting out the spectrum of programs grams have lacked funding perennially. occupations, the creation of renewable for different kinds of recipients. Defense-dependent communities re- energy sources, the development of ceived an additional 50 million for high energy-efficient and non-polluting Transition assistance in the form of 1991 through 1993, but this was still transportation systems, and the produc- retraining and reemployment for af- much less than the amounts available in tion of a new generation of high-per- fected workers has been considered the past. And states have had difficulty formance computers and networks. As critical. However, a broad-range strate- sustaining innovative economic devel- with most phenomena of the proportion gy appears to be needed that simulta- opment projects. In this context, the re- represented by current defense conver- neously invests in sound, appropriate port contained some useful recommen- sion activities, multiple, coordinated worker training, assistance to business- dations: strategies are required. We no longer es to increase their performance, re- fear global nuclear war. With the end Deliver services more promptly. search and development assistance to of the Cold War we have the freedom advance technology, and help with eco- Target communities in greatest need. to work toward national goals other nomic development efforts. A recent than military security. Many experts in- report by the Office of Technology As- Supplement state programs. sist we must use this opportunity to sessment (OTA) of the U.S. Congress, Offer a range of technical assistance move forward as an economic competi- After the Cold War: Living With Lower to help companies grow rather than tor in an international marketplace Defense Spending, has recommended attempting to motivate them through and at the same time ameliorate the this kind of approach. tax subsidies. hardships experienced by defense The authors of the OTA report re- workers, defense-dependent communi- mind us that in 1991 national defense Another complication the report ties, and defense companies during this employed approximately six million gives attention to is the reluctance of transition period. people in the private sector, in the ac- major defense contractors to initiate 11 39 Evaluation Forum Issue 11 Summor 1995 4 7 -

Features Defense Converston and Distocation

Studying Defense Matt Kane, propose that in the larger that were highly dependent on the de- economic environment military cut- fense budget at that time employed Conversion backs will havePbeneficial effect on nearly 1.8 million workers. In industri- in Affected the national economy, releasing funds al sectors such as guided missiles, ship- for deficit reduction. However, the im- building, ammunition development, Communities pact on particular regions and indus- and tank construction, 75% of workers tries clearly varies. The variation is in- owed their jobs to defense spending. In this article we look at an interest- fluenced, they say, by the following ing 1993 study of seven sites in the factors: This defense workforce differs in U.S. that have been impacted signifi- important ways, the authors propose, cantly by defense cutbacks. The study The size of the cuts. from the average labor pool. It consists was sponsored and funded jointly by The speed with which they are largely of professionals, skilled and the Economic Development Adminis- made. semi-skilled craft workers, clerical tration (EDA) of the U.S. Department workers, and fairly skilled support of Commerce and the U.S. Department The mix of cuts vis-a-vis procure- staff. The Office of Technology As- of Defense's Office of Economic Ad- ment, operations and/or personnel. sessment of the U.S. Congress conclud- justment, and conducted by the North- ed in their 1992 study of the defense The overall strength of the U.S. east-Midwest Institute. The study re- workforce that it was more highly port, Defense Adjustment and economy at the time of the cuts. trained and skilled than workers in oth- Conversion: Lessons from Seven Sites, The authors give us an important er sectors of the economy. Its workers is the basis for the review. glimpse of defense expenditures, to were making high wages. These char- provide a background for understand- acteristics clarify what a substantial im- The Nature of the Study ing the implications of defense down- pact sudden job loss can have in terms of the reemployment challenge. The sites selected for the process sizing. About 75% of defense purchas- study were San Diego, Minneapolis, In- es are made from manufacturing The change in the fortunes of the de- dianapolis, Albuquerque, Bath (Maine), industries; 21% from the service sector. fense workforce must be understood in and Sunnyvale (California). The re- The rest is from construction, agricul- the context of the trend in manufactur- searchers used focus groups in each site ture and mining. More than 15% of the ing jobs since the late 1970s. The to explore how defense cutbacks were total output of 31 industrial sectors in- occupations most at risk have been in affecting workers, businesses and com- volves defense-related activities. At manufacturing areas engineers; tech- munities; what plans local leaders had least 29 of these industries manufacture nicians in engineering and science; developed for coping with the conse- durable goods. Some estimates indicate computer specialists; craft and similar- quences; and how these communities that as much as 57% of militaiy-related ly-skilled production workers; and less could promote effective economic ad- employment is in the manufacturing skilled operators and laborers. Overall, justment and diversification. sector. manufacturing jobs have not risen since Recognizing the effect of reduced the late 1970s and have fallen by 5% The additional purpose of the focus since 1990. groups was to support coordination purchasing in this sector, the U.S. De- among key actors and constituencies in partment of Defense (DOD) has re- Areas with a high concentration of these communities and some level of formed its regulatory procedures to as- advanced technology firms and military agreement about what action should be sist companies in defense-dependent bases have been the hardest-hit. Eight taken. To achieve this latter purpose, industries in order to reduce their states accounted for about one-fifth of focus group participants were provided costs, and to become more competitive. the defense spending in 1991. States with information obtained from case The DOD has also eliminated "recoup- with a large number of military bases studies of defense adjustment and con- ment fees," or reimbursements much and facilities had become the most de- version, and from programs designed to like royalties, that private companies pendent. The Congressional Budget Of- resolve problems associated with mili- on federal contracts had to contribute to fice reported in 1992 that a defense re- tary cutbacks. They were encouraged to the federal government when they de- duction of 20% between 1991 and 1995 discuss how the roles and responsibili- veloped products or technologies that could result in a significant output loss ties of the different kinds of organiza- became viable in the civilian market. in the District of Columbia and ten tions and groups participating in the fo- In terms of employment, although states. Commercial-sector jobs associ- cus groups should be defined, in the the portion of the labor force in de- ated with defense production are affect- process of developing strategies for fense-related businesses and industries ed in these areas as well. solving problems in their own commu- is small, it is nevertheless significant. nities. In 1989, the authors indicate that about Federal Assistance for 3.4 million private sector workers were Adjustment Purposes The Larger Context employed directly or indirectly in de- The Defense Conversion, Reinvest- The authors of the study report, fense-generated jobs about 5% of ment and Transition Assistance Act of Charles Bartsch, Paula Duggan and the total labor force, Eleven industries late 1992 authorized a set of new pro-

Evaluation Forum Issue 11 Summer 10P6 4 8 - _

Features Defense Conversion and Dislocation

grams responsive to the difficulties be- Small Business Innovative Research The Results of the Seven-Site ing experienced by certain businesses Program: this program increased Study and industries, and by whole communi- DOD's R&D funds directed to small Two major themes ran through the ties, as a result of defense cuts. Much businesses. recommendations of private sector par- of the funds for these programs flowed ticipants in the focus groups: to beneficiaries in fiscal 1993. Some of Looking at the funneling of DOD Remove regulatory and administra- the funds were channeled into existing dollars through multiple programs ben- economic development, worker train- efiting the American economy general- tive barriers that impede the adapta- ing and education programs, but a por- ly, one cannot avoid exploring issues of tion of dual-use technologies: corporate charity and the need for pro- tion went to new initiatives. Some of Accounting requirements. these were expected to enhance manu- gram consolidation. Congress provided facturing competitiveness by emphasiz- $694 million to assistance programs Proprietary rights limitations. ing "critical technologies" and "dual- targeted to the private sector in the 1992 legislation. Increase government support for di- use technologies:" versification and adjustment pro- Commercial-Military Integration Assistance to Workers grams designed to help defense-de- Partnerships: the DOD was to work While the issue of equity was raised pendent companies shift to civilian with high-tech firms to develop and by this large allocation to businesses, markets: commercialize high-payoff applica- $686 million was being directed to Technical assistance. tions for dual-use technologies. worker adjustment programs. Most of Advice on gaining access to the Regional Technology Alliances As- these programs assisted military per- technologies necessary for the sistance Program: funding was to sonnel separating from the service and civilian workers employed by DOD. shift. enable the development of alliances The programs are providing separation among defense firms (and prime counseling, incentives for early retire- Advice on marketing strategies. contractors) to develop and apply ment, extended benefits such as health Information about new markets. critical defense technologies for dual care, increased training for civilian oc- use defense and civilian with cupations, and the establishment of hir- Public sector participants wanted gov- an emphasis on regional diversifica- ing preferences for these workers in ernments at all levels to adopt a more flexible approach to worker training, tion in defense-dependent areas. government jobs. Seventy-five million of the total funding is to be used for a the financing of economic develop- National Defense Manufacturing new program for dislocated workers ment, and to other programs. Although Technology Program: the federal similar to DOL's EDWAA although the sites were quite diverse, the focus government was expected to engage DOD was given the option of simply groups yielded common proposals: small defense firms in using military adding this amount to JTPA Title III. Business leaders should identify al- technologies as a base for diversifi- DOD is to notify defense contractors of ternative operating scenarios and be cation. federal cuts affecting them, and will re- prepared to restructure their busi- quire them to post jobs with the Em- nesses so as to be competitive in ci- Technolov Transfer Initiatives: ployment Service. The legislation also vilian markets. these were to encourage private sec- requested educational institutions to tor businesses to transfer military provide training in environmental occu- Defense-related companies need a technology for civilian use. pations. range of information to increase their export potential: Defense Advanced Manufacturing Assistance to Communities Technology Partnerships: DOD Identification of overseas mar- The budget earmarked for communi- was to share the costs borne by firms kets. in developing and applying manu- ty assistance in the 1992 legislation facturing technologies for military was $132 million. Eighty million of The defming of prospective mar- that was to be transferred from DOD to ket "niches." and civilian production. EDA in the U.S. Department of Com- Manufacturing Extension Program: merce. In addition, DOD's Office of Information about mechanisms this program was to improve the Economic Adjustment received in- linking U.S. manufacturing capa- competitiveness of manufactur "ng creased funding to permit the distribu- bilities and technologies with prospective foreign purchasers. technology. tion of both planning grants and imple- mentation grants for assisting Information about how to target Dual-Use Extension Program: this communities, of the kind already dis- markets with growth potential. involved a number of programs tu tributed by the EDA. assist defense suppliers in making the transition.

41 Evaluation. Forum Issue 11 Summer 1995 4 9 Features Defense Conversion and Dislocation

Worker adjustment program rules There should be a study of 1) key The participants in the focus groups and regulations should be more flex- defense-related industries, 2) work- also felt that each community's defense ible, to allow defense-related dislo- forces and 3) skill needs by the fed- adjustment effort required a mechanism cated workers to be better served. eral government, and a national in- for initiating, organizing and maintain- dustrial agenda should be developed ing action, such as a person, group or Increased ability to launch pub- organization that commanded commu- licly-funded retraining programs or at the least defense manufac- turing policies should be coordinat- nity respect and support, and could prior to the loss of a job for serve as a focal point for policy devel- ed example, training for new tech- to ensure that necessary tech- opment, planning and oversight activi- nologies as part of a company nologies and production capabilities ties. conversion or product diversifica- can be maintained. tion project. Implications of the Study for Some Final Study Conclusions Different Levels of Government Better information on emerging The authors conclude that a success- The focus groups placed an unam- new skills and their mesh with ful adjustment strategy must "bring a existing work skills. biguous emphasis on 1) a national dual- high level of visible participation and use business and industrial policy, ac- More specialists in commercial man- commitment" to this effort by local po- companied by increased federal agement, productivity, marketing litical and business leaders. Simple in- technical and financial assistance to and purchasing need to be produced. terest in the problem will not suffice. state governments, defense-oriented Communities need to launch formal employers, and local communities af- A successful business retention needs assessments and planning pro- fected by downsizing, and 2) greater ef- strategy should be developed, target- cesses which speed adjustment efforts. forts on the part of community political ed to defense-oriented companies. Those sites that had established formal leaders, government agencies and the planning structures and processes were private sector to identify the array of Acquisition of capital should be making the greatest progress. They had needs due to defense cutbacks, and to more accessible and affordable, to used a series of meetings and surveys take well-organized action to reduce carry out defense adjustment poli- to identify the potential magnitude of the negative fall-out. cies and strategies. the problem, the nature and scope of existing workforce skills and manufac- The thrust of the stud, has important DOD should adopt a more quality/ turing capability, and other information implications for current federal efforts results-oriented approach to military needed to construct a workable adjust- to consolidate and otherwise coordinate procurement that places less stress ment strategy. This information base workforce development policies and on detailed specifications. was considered essential to the achieve- programs, and integrate their informa- ments realized. tion systems and for states' new There should be greater support for efforts to establish comprehensive the development of "critical technol- The following kinds of information state-wide councils responsible for ac- ogies" by small businesses, to in- were recommended for this information complishing the same goals. The major crease the number of high-quality bank: message would seem to be that defense employers and reliable specialized Numbers of workers employed and adjustment requires a multi-dimension- vendors for defense-related parts unemployed. al strategy involving assistance to dis- and components. located workers placed within the larg- Numbers of actual and likely plant er context of assistance to businesses DOD's proprietary rights regarding closings, and what industrial sectors and communities. the dissemination of information they represented. should be limited, to increase the in- The recommendations flowing from Numbers of workers on layoff from this study would suggest strongly that tegration of military and commercial the Departments of Defense, Com- production. defense-oriented companies, and their skills. merce, Labor, and Health and Human DOD policies and regulations that Services should increase their coordi- fit the Cold War but now interfere Kinds of emerging industries in the nation and collaboration on defense ad- with efforts to convert or scale back area, and the types of workers they justment issues, and that state work- production should be eliminated, were likely to hire. force development systems should incorporate economic development within the goal of maintaining criti- Information on the education, train- agencies within their formal structure cal technologies. ing and social service resources and functions. available. DOD stipulations regarding cost and pricing data, certifications and au- Job creation information. dits should be overhauled.

42 Evaluation Forum Issue 11 Summer 1995 50 mmol ,..., Features Defense Conversion and Dislocation

Si III Programs to Ease A number of states have their own former military personnel. programs covering services to dislocat- In 1994, rather than transferring de- the Adjustment to ed military and workers in defense- fense adjustment funds from the DOD related businesses and industries. Im- to DOL, DOL requested a general Defense Cutbacks portant examples are California's Em- funding increase for dislocated worker ployment and Training Panel, New In the early 1990s, the Defense Con- programs. Congress provided over one York's Skills Training Program, and version Commission in-the U.S. De- billion in FY 1994 for dislocated work- Washington State's Job Skills Program partment of Defense funded the Logis- er assistance in DOL, of which $125 and vocational education effort. tics Management Institute (LMI) to million was to be earmarked for de- fense workers. The Administration's analyze and classify information about Clearinghouse Activity the myriad programs providing some 1995 request includes approximately kind of assistance to individuals, busi- The DOD and Department of Com- $1.5 billion for Title III, with an esti- nesses and communities related to de- merce jointly maintain a useful clear- mated $195 million going to workers fense downsizing. LMI subsequently inghouse, the OECI Clearinghouse, dislocated from defense-related em- collected data on these programs and which tracks the assistance provided to ployment. interviewed a large number of federal, individuals, businesses and communi- state and local officials about them. ties in the plethora of programs created Evaluation Mandates Their effort produced the Compendium due to defense restructuring and down- Conspicuously absent from descrip- of Programs to Assist the Transition, sizing. For example, summaries are tions of DOD programs for dislocated published in Feburary 1993. available for the Defense Industry Ad- workers within, separated from, or out- justment Projects which down in- side but related to the military is any The Compendium includes pro- formation by type of award granted to grams in existence prior to the National mention ofprogram evaluation. While states state planning grants, ad- Defense Authorization Act for Fiscal Department of Education and DOL vanced planning grants, and defense in- programs have substantial evaluation 1993, and those initiated through this dustry adjustment grants and de- legislation and some of the legislation requirements, the DOD programs do scribe each project. that followed. Sixty-one of the 144 pro- not appear to have such requirements. Although the Defense Conversion grams were sponsored by state and lo- Recent Initiatives cal communities, even though the latter Commission has taken a strong position were sometimes utilizing federal funds. In March 1995, President Clinton on the need for performance measures, announced a new five-year package of no performance management system The Nature of Assistance programs under the Defense Transition comparable to that required for the Programs Program. These were mainly reinvest- DOL programs seems to be in place. If ment and transition programs devel- true, this would be a serious deterrent The major assistance programs in oped by the Congress but not imple- to sound policy development for this the Compendium are federally-funded mented by the Bush Administration. In portion of the dislocated worker popu- and emphasize training and economic the President's FY 1995 budget re- lation. development. They tend to serve as the quest, funding for four categories of ultimate safety net for individuals af- programs was requested: workforce, The Department of Education fected by defense cuts and drawdowns. community, dual-use technology, and Report Programs targeted to industries stress non-defense zachnology. Under work- In May 1991, the Office of Educa- development and technology transfer. force initiatives, the main interest was Almost all state programs involve job tional Research and Improvement in separation pay and retirement incen- (OERI) organized a conference to ex- training oriented to job retention or re- tives, and in education and training. employment. Industry programs also amine the role of education in restruc- Out of the seven education and training turing defense industries and ways to concentrate on worker training, as well programs recently established by Con- as industrial development. Service deliv- ameliorate the negative impact of such gress, the President chose to emphasize restructuring on workers. Much was al- ery for federally-funded programs are the two that trained active duty military for responsibility of states, while local pro- ready known. In 1990, over 200,000 public service-oriented jobs, the active and reserve service personnel grams are directed mainly to local em- "troops to teachers" and "troops to ployment and economic development. and veterans used educational benefits cops" programs. to help pay for their education. A study The Compendium is well organized Not included in the funding request, completed in 1976 had suggested that and detailed, reviewing separately pro- however, were three environmental ed- the earnings of veterans using Mont- grams for individuals vs. programs for ucation programs authorized in the FY gomery GI Bill benefits were 10% states and communities, according to 1994 Defense Authorization Act, and higher than they would have been oth- their funding sources. Figure I lists the funding for the Occupational Training erwise. A subsequent study indicated programs for individuals, summarizing Program that was to partially reimburse that educational benefits used by Viet- selected federal programs for workers firms for the cost of employer-provided nam-Era veterans and by servicemen in displaced from the military and/or de- training if these firms hired and trained fense-related employment.

Summer 1995 Evaluation Forum Issue 11 51 Features P Defense Conversion and Dislocation

Program and Funder Target Group Type of Assistance Involuntary Separation Pay Program Active members of military with 6-20 Lump sum separation payment. DOD years of service involuntarily separated from service. Montgomery GI Bill Active members of military and National- Educational stipends. DVA Guard. Post-Vietnam Era Assistance Same as above. Same as above. DVA

Vocational Rehabilitation and Disabled veterans. Counseling, education, training, and Counseling support services. OVA

Voluntary and Special Separation Selected active members of military Sedes of annual payments or lump Incentives officers, enlisted memberswho agree sum in return for voluntary separation DOD to voluntarily leave military. and continuation in reserves. Defense Automated Transition All members ot military separating from Systems Job placement assistance, (The sys- armed services. tem is in cooperation with DOD's Tran- DOD sition Assistance Offices, which are of- ten co-located with family and commu- nity support centers.) Armed Services Alumni All armed services personnel in process Program Career guidance, benefits counseling, of leaving services, and their families. job search assistance. (Members of DOD armed services leaving military receive certification of their job experience and skills while in military.) Interagency Placement Assistance Members of military leaving service. Program Job placement assistance, and the communication of the availability of DOD these individuals (or employment to the federal agencies. Reemployment Priority List Members of military leaving service due Usting of individuals on priority list for DOD to dislocation. federal jobs. Transition Assistance Program Separating service members. Three-day job search assistance semi- DVAIDOD nars through state Employment Ser- vice. Priority Placement Program Dislocated federal civilian employees. Job placement assistance. DOD

Displaced Employee Program All federal employees involuntarily sepa- Job placement assistance. Ali agencies. rated. JTPA Title III Dislocated workers. EDWAA Rapid response assistance, training, basic readjustment assistance, needs related payments. (The 1993 National Defense Authorization Act amended Title Ill to establish discretionary pro- gram designed to respond to reem- ployment needs of 1) those at risk of dislocation and 2) dislocated military, DOD civilian personnel, and defense workers.) Employment Service All workers. Job search and placement assistance. IDOL

Unemployment Insurance Eligible unemployed workers. Fiscal benefits. DOL

Figure 1 continues

Evaluation Forum Issue 11 52 Summer 195 Features Defense Conversion and Disiocation

Program and Funder Target Group Type of Assistance Job Opportunities and Basic Skills Adult recipients of AFDC. Educe+ fon, training, employment assis- Program (JOBS) tance, and support services. HHS Federal Work Study Higher education students needing Financial aid. DOE financial assistance. Federal Family Education Loans Same as above. Stafford Student Loans, PLUS loans, DOE and Supplementary Student Loans. Federal Pell Grants Same as above. Educational grants, up to five years. DOE Federal Supplemental Educational Same as above. Undergraduate/graduate nonrepayable Opportunity Grants grants. DOE Federal Perkins Loans Same as above. Loans with delayed repayment. DOE 1990 Carl D. Perkins Vocational and Same as above. Vocational education grants. Applied Technology Grants DOE Service Members Occupational Dislocated members of armed services. Six to eighteen months of occupational Conversion end Training Act of retraining. 1992 DOD

Ppg,krFOTated YA*199 Program and Funder Target Group Type of Assistance Continued Health Coverage for Former service members and their Temporary health benefits. Military and Dependents families. DOD Skill Training Programs for Dislocated DOD civilian employees. Occupational training. Civilian Personnel DOD Separation Pay Dislocated DOD civilian personnel. Lump sum payments. DOD Teacher and Teacher's Aid Separated members of armed forces, Education to acquire teacher or teach- Placement Programs terminated DOD civilian employees, er aide certification, job placement as- DOD displaced scientists/engineers working sistance, stipend during first two years for defense contractors. of teaching. Environmental Scholarship and DOD employees. Fellowship for training in resource Fellowship Programs management/restoration. (DOD also DOD funds grants to higher education insti- tutions to provide training in environ- mental restoration and hazardous waste management.) Defense Diversification Program Displaced military and employees of Transition adjustment and employment DOD/DOL defense contractors. services. (Expansion of Interstate Job Bank Programs and coordination with Defense Outplacement Referral Sys- tem.)

Acronyms: DOD - Department of Defense; DOL - Department of Labor, DOE - Department of Education; DVA Department of Veterans' Affairs; HHS - Department of Health end Human Services.

1111111M111. Evaluation Forum Issue 11i Summor 19P5 53 . . -:...i,;;;ata'..,,t

Features Defense Converskm and Disknation the all-volunteer army raised overall Findings from the a At the level of the individual work- earnings for veterans by an average of er, defense conversion is the process about 6%, if used for college or gradu- Defense Conversion by which defense industry workers ate school. These findings were consis- Adjustment and military personnel prepare tent with other studies in the 1980s and themselves for jobs in the non-mili- 1990s showing significant earnings Demonstration tary commercial sector. This con- gains for college graduates and those version process may involve learn- with graduate degrees. Project ing new technical or occupational Another set of analyses offered simi- skills or how to adjust to corporate lar insights. The conference benefited Mary G. %fisher culture in the commercial sector. from ten commissioned policy papers itorkeley Planning Associates relying on relevant research. "Lessons In May 1992 the U.S. Department of from the Past: Mitigating the Effects of Labor (DOL) became directly involved Military Cutbacks on Defense Work- Deborah Kogan with defense conversion when it an- ers," by Lois Lembo and Judith Phil- Social Policy Reeser% Associates nounced the availability of funds to lips, makes a substantial case for edu- support innovative strategies to assist cation as a vehicle for reducing the workers, businesses and communities dislocation effects of defense cutbacks, Background on Defense affected by downsizing.1 Twelve and for ensuring that defense conver- Conversion Round 1 DCA grants were awarded in -a sion contributes to workers and the November 1992, and another seven economy. Burt Barnow and Amy As a result of the end of the Cold Round 2 grants a year later. Grants Chasanov's paper on "Firm-Based Ed- War, outlays by the U.S. Department of were awarded under several different ucation and Training of Workers" il- Defense are projected to drop by 30% categories: community planning, dislo- lustates in a case study of the Xerox between 1987 and 1997. These reduc- cation aversion, and increased worker Corporation how companies can use tions in defense expenditures have af- mobility, reflecting the three different =-4 education and training to maintain their fected defense industry workers and levels of impact from the defense draw- economic health in the midst of unex- military personnel, private firms that down.2 pected changes in federal budget priori- are dependent on defense-related sales, ties. and communities with military installa- Overview of the DCA tions and/or concentrations of defense Demonstration Evaluation If we are to determine the actual im- firms. The term "defense conversion pact of the many DOD transition assis- adjustment" refers to the process by The Department of Labor contracted tance programs, the kind of research which communities, firms, and individ- with Berkeley Planning Associates (BPA) to conduct an evaluation of the expertise illustrated in OERI's report ual workers adjust to the defense draw- on the conference, Military Cutbacks down. DCA Demonstration. The objectives of and the Expanding Role of Education, this evaluation, performed by BPA and needs to be applied to these programs. a At the community level, defense its subcontractor, Social Policy Re- The Department of Labor and Depart- conversion requires decision-makers search Associates (SPR), are threefold: ment of Education research efforts to plan for and address the immedi- (1) to describe and document the im- would seem to offer useful models. ate impacts of reductions in defense plementation and short-term outcomes spending on affected firms and laid- of the demonstration projects, as they off workers, while building a foun- relate to the specific problems faced in dation for economic renewal by co- defense-related dislocations; (2) to identify exemplary approaches to the ordinating all available community specific problems faced in defense-re- resources. lated dislocations; and (3) to identify At the firm level, defense conversion the factors that contribute to or impede is the process by which defense-de- the success of various defense ponver- pendent firms develop new process- sion approaches. es and products that will enable them to compete in commercial mar- kets. Necessary changes may in- 1Section 325(d) of Title III of the Job Train- clude reorganization of the work- ing Partnership Act provides funding for demon- place and training workers and stration projects as part of the Defense Conver- managers in new technologies and sion Adjustment Program. high peiformance workplace skills 2Due to limited space, in this summary we only report findings from our observations of the (HPWO). projects using th c two latter approaches.

/11.1.1MINPMIIIIIMMAMIMINIMIMMER11111111 EvaluAtion ForumitIssue 11aSummer 1995

1 . Features Defense Conversion and Dislocation

performance workplaces. These skills The methodology for this evaluation contributed to the achievement of these include teamwork, communication, emphasizes the collection of qualitative goals. problem-solving, continuous improve- data using multiple site visits to each of This study has not attempted to col- ment skills, process analysis and statis- the 19 demonstration projects, supple- lect outcome data from comparison tical process control. While these skills mented by limited quantitative data on groups or sites to suggest what would were perceived in all projects asbeing the characteristics of participating have happened to participating workers key to successful firm performance in workers and firms, services provided, or firms in the absence of services.In commercial markets, projects were and outcomes achieved by participating assessing project effectiveness, we have most successful: workers and firms. This article, based compared each project's performance on mid-project findings reportedin the to its own stated goals, as well as to the If workforce training was closely evaluation's Phase I Final Report,3 outcomes achieved by other demonstra- linked to furthering specific conver- summarizes findings on how the tion projects testing similar approaches. sion goals at the firm level. projects have fared thus far, and lessons that can be learned from their experi- Averting Layoffs at Defense- If worker representati :es had a voice ences. Dependent Firms in the design and oversight of train- ing. Evaluation Issues Prhr.tjects If there were structured opportuni- The evaluation is based on detailed Nine projects are testing dislocation ties for workers to practice their new case studies of each projectdocument- aversion strategies. Figure 1 summa- skills in the workplace during and ing project goals and objectives, the in- rizes the dislocation aversion projects after training. volvement of different project partners and their key features. Projects vary in in developing and implementing the the number of firms they are assisting, If the firm had identified measurable demonstration design, and progress to the size of participating firms, and the indicators of success at the firm or date in achieving project goals. These stage of participating firms in the con- work-unit level case studies have provided valuablein- version process. Some projects target- If firms were given the autonomy formation on projects' successes in im- ed firms early in the adjustment process plementing their intended designs as and supported managers during the they needed to choose a service plan well as barriers to successful imple- process of planning for conversion. that was appropriate, as long as the mentation. Other projects targeted firms that had quality of the providers was moni- Particularly challenging for this already developed strategic plans for tored by the project administrator. evaluation has been the assessment of conversion and were ready for work- It is too early to draw conclusions project effectiveness using data on par- force training. about whether firms were able to avert ticipant-level and firm-level outcomes. layoffs as a result of the assistance feey Since most of the projects are still in Findings received. However, several of the par- operation, final participant outcomes A sequence of steps is necessary for ticipating firms were able to document are not yet available. For projects as- successful conversion, including identi- improvements in productivity and cost- sisting defense-dependent firms with fying firm strengths and weaknesses, effectiveness measures and/or avoid an- defense conversion, the intended out- developing a serategic plan for entering ticipated layoffs during the first phase comes (e.g., increased non-defense non-defense markets, and implement- of the demonstration. sales and workforce stability or growth) ing the changes necessary to compete may not be observable withinthe eval- successfully in these markets. Projects Increasirig Worker Mobility uation time-frame, particularly for that targeted firms early in the conver- firms at the beginning of the conver- sion process spent substantial amounts Projects sion process. Even where firm-level of staff time and project resourees as- Nine projects are attempting to pro- outcomes are available, it is not clear sisting management in conducting an mote increased worker mobility for in- whether and how demonstration-related objective assessment of their strengths dividuals dislocated as a result of lay- activities, such as workforce training and weaknesses and developing a stra- offs among defense industry workers, and strategic planning assistance, have tegic plan for conversion prior to work- military service personnel, and/or civil- force retraining. For these projects, ian Department of Defense employees workforce training sometimes grew out at military facilities. Figure 2 summa- 3This report was submitted in draft to DOL of the assessment process, when work- in March 1995. The report consists of prelimi- rizes the key features of these projects. nary findings on the three defense conversion ers were asked to participate in assess- approaches (worker mobility, defense conver- ing the strengths and weaknesses of The worker mobility projects fall sion, and community planning) in Volume I. their individual work units. into three rather distinct categories: (1) Volumes LI through IV contain detailed project those that seek to reemploy participants profiles for each demonstration project, orga- Projects that focused on improving quickly without intensive retraining; nized by approach. An executi it summary will worker skills often emphasized the irn- (2) those that seek to prepare partici- be available as a sepmrtely bcund volume. portanee of the skills needed for high

47 Evaluation Forum A 4E14 11 Sumiriet 1995 Features Defense Conversion and Dislocation

. . igure Oyoryieq.o.f.the Dislocation Aversion PrOjects-. .

International Association Assist defense-dependent :I mush to medium-sized Provide censuRing sarvic of Machinists and Aero- finns to become more firms in Los Angeles area es to assist in conversion space Workers (IAM) Con- competitive planning version Demonstration Assist in the reemploy- IAM Lodge 727 Provide HPWO and occu- ment of dislocated de- pational skills training Los Angeles, Callfom?a fense workers _Develop and provide training to build electric car components Develop job opportunities for participants

Long island Defense Assist defense-dependent 9 small to medlum-sized Diversification Project aAsess firms' strengths firms to become more firms in Long island area and weaknesses New York State Department competitive by promoting of Economic Development HPWC principles Provide HPWO skills New York, New York training

Management Assistance Faciii!ate development 20 smali to nodium-sized Support assessment of nd Technology Transfer anti Implementation of firms in St. Louis area Program workforce skills and pro- commercial conversion duction systems t. Louis County Economic plans for defense-depen- . curled dent meinufacturing firms Support strategic planning St, Louis, Missouri Support occupational and HPWO skills for workers

Massachusetts Strategic aAssist defense-dependent 12 small to medkirn-sized Skills Program Occupational skills firms to become more firms in Massachusetts training Massachusetts Industrial competiGva area Services Program Other skills training Boston, Massachusetts (HPWO, team and com- munication skills, TOM, process analysis)

San Diego County Defense Assess business needs Defense-dependent busi- Conversion Adjustment Survey of local business- Disseminate information to nesses exploring conver- es to assess and address Demonstration affected businesses sion in San Diego San Diego Consortium and affected business needs Private Industry Council Assist defense workers Conduct Business San Diego, California find new jobs Roundtabies to assist de- fense dependent firms consider conversion op- tions Deliver services to dislo- cated workers, Including high tech entrepreneurial training and TOM skills training

Figure 1 continues

48 56 Evaluation Forum Issue 11 Summer 1995 Features Defense Conversion and Dislocation

Assist in the conversion of Sargent Controls (employ- Provided support to as- Sargent Controls Project sess strengths and weak- Pima County Community Sargent Controls, produc- ing about 400) nesses Services Department er of nuclear submarine Pima County, Mzona valves Provided occupational, technical and other skills training to workers

Assist in Increasing the Alliant Techsystems (em- Occupational skills train- Minnesota Defense ing for machinists Conversion Adjustment competitiveness of Aliiant ploying about 4,000) Demonstration Techsystems, a producer Workplace basic skills Department of Jobs and of explosive devices training Training Basic skills enhancement St. Paul, Minnesota Consulting services to The Hummer Project: A Assist in increasing the AM General (employing conversion, stabilization of 1,500 workers) and sever- support HPWO principles Dislocation Aversion (AM General only) Project AM General, producer of al supplier firms Workforce Development an all-terrain military vehi- Workforce training on Services cle. communications, team- South Bend, Indiana building and process analysis (AM General and nine suppliers)

Assist in increasing the 20 small to medium-sized Occupational and techni- Workforce Protection cal skills training Program competitiveness of small defense firms in Rhode Is- Rhode island Port Authority defense-dependent firms land and Economic Development Corporation Providence, Rhode island

49 Ey* !cation Forum aIstu1111 Summer 1995 57 Features Defense Conversion and Dislocation

. , Figtae 2 n Overview 6f the WorkeitiViobitityzPtplects

Philadelphia Naval Base Plan and coordinate a re- Several thousand at-risk Identify and review strate- and Shipyard Complex sponse to the drawdown and dislocated base work- gies to assist affected Planning Project of th...% Philadelphia Naval ers workers Pennsylvania Department of Base, soheduled to be Create a database con- Labor and Industry complete ro early 1996 Philadelphia, Pennsylvania taining skills and charac- tedstim of workers and needs of businesses Develop plans for an on- base transition center

CharlestonNaval Complex Plan and coordinate a re- A small number of dislo- SI Produce an economic Community Planning sponse to the dislocations cated base workers to Project profile of the Charleston caused by the drawdown participate in pilot training metropolitan area Charleston County of the Charleston Naval programs Develop a coalition be- Employment and Complex, scheduled to be Training Administration complete in 1996. tween economic develop- Charleston, South Carolina ment entities, educational institutions, and the job training and placement community Develop a plan for a "one- stop shop"

International Association Assist defense-dependent Dislocated defense indus- of Machinists and Aero- Provide consulting servic- firms to become more try production workers with es to assist in conversion space Worker* (IAM) Con- competitive experience with composite version Demonstration planning Assist in the reemploy- materials Provide HPWO and occu- !AM Lodge 727 ment of dislocated de- Los Angeles, California pational skills training to fense workers workers in participating firms aDevelop and provide training to build electric car components Develop job opportunities for participants San Diego County Defense Assess business needs Dislocated defense Survey of local business- Conversion Adjustment workers Demonstration Disseminate information to es to assess and address affected businesses affected business needs San Diego Consortium and Private Industry Council Assist defense workers aConduct Business San Diego, California find new Jobs Roundlables to assist de- fense dependent firms consider conversion op- tions Deliver services to dislo- cated workers, Including high tech entrepreneurial training and TOM skills training

Figure 2 continues

Evaluation Forum Issue11 Summer 1995 -

Features Defense Conversion and Dislocation

, ' Figure 2i 'COptinued ., ,,---... ,., e,.7 ZA , *AA% .A.4., '1.,'.., > ,,..,., . Dislocated defense work- Classroom training in self- Center for Commercial aAssist defense workers to a start up new businesses ers with professional or directed teamwork and Competitiveness commercial competitive- State University of New York technical skills ness Binghampton, New York Support for project teams to develop new business- es or contract with local .c. firms

Assistance In completing Military Certification Assist military personnel to Separating military officers become teachers in South prerequisites for Clemson Project: A Worker Mobility University's teacher cert. Carolina Project ficatIon program University of Clemson Clemson, South Carolina Assist In placing partici- pants in paid

Assist laid-off employees a Production workers and Basic readjustment ser- Project Earn: A Worker vices, including assess- Mobility Project from McDonnell Douglas managers laid off from Mc- to find replacement jobs Donnell Douglas's Titus- ment, career counseling McDonnell Douglas and job search assistance Aerospace East Alio facility after canoes- Short-term training to up- Titusville, Florida tion of Advanced Cruise Missile Program grade skills

Assessment, career coun- Operation StepOut: A Assist participants to tran- Women dislocated or at sition to non-defense em- risk of dislocation from de- soling, job search assis- Worker Mobility Project tance services Arizona Governor's Office ployers tense sector for Women Seminar on gender Is- Tempe, Arizona sues Access to career network- ing group

Support development of Alternative Fuels Training Train and place dislocated Workers dislocated from curriculum Project: A Worker Mobility defense workers In emerg- Dallas-Fort Worth area de- Project ing high technology occu- tense contractors aProvide skills training Texas Railroad Commission pations Assist in certification pro- Dallas-Fort Worth, Texas Ms Assist in placement of participants

_41

51 Evaluation Ponan issue 11 Summer 1095 59 Features Defense Convorsion and Dislocation

pants for new in "niche" occu- projects have found it very difficult and pations (e.g., as teachers, alternative Mary G. Visher is principal Ana- time consuming to recruit appropriate lyst at Berkeley Planning Associates fuel technicians for public vehicle participants to match to the available (BPA) and project director for the fleets, or production workers using services. In addition, projects trying to Evaluation of the Defense Conversion composite materials); and (3) those that place participants in emerging indus- seek to support would-be entrepreneurs Adjustment Demonstration. Since re- tries have found that the new jobs have ceiving her doctorate in labor market in starting new businesses or joint ven- not always been available when they sociology in 1984 from the University tures in high-technology fields using were needed. It is too early to say of Wisconsin-Madison, she has been skills and_processes developed in the whether these projects are helping par- active in policy analysis and defense industry. program ticipants find high quality jobs at the evaluation. Her area of specialization conclusion of training. Findings over the past ten years has been in the The projectssupporting new busi- field of employment and training. She The projectsproviding basic read- ness start-ups or joint ventures with ex- has directed or been involved withnu- justment services and short-term skills isting firmsare testing how high tech- merous U.S. Department of Labor stud- enhancement training arerather disap- nology start-up firms can be used to ies. pointing. Few of these projects identi- stimulate economic growth by creating Deborah Kogan is Senior Social fied new or innovative models of basic new jobs in the local economy. Both readjustment services tailored to the Scientist at Social Policy Research As- Round 1 projects in this category de- sociates (SPR) and co-principal investi- needs of dislocated defense workers. veloped highly innovative and ambi- Because project planners in several gator of the Evaluation of the Defense tious designs using linkages with exist- Conversion Adjustment Demonstration. projects had misjudged local labor ing businesses in innovative ways. One market conditions, the limited services She has played a key role in a number project appears to be particularly effec- of national evaluations of federalem- offered by these projects were often in- tive in achieving positive outcomes for sufficient to meet the reemployment ployment and training programsspcn- participants, perhaps because it selected sored by the U.S. Department of Labor, needs of project participants. Projects participants who already had strong struggled to adapt or refme services af- including a recent evaluation of the business concepts and emphasized the EDWAA program. She is also direct- ter project start-up to correct these "nuts and bolts" skills needed to shortcomings. In addition, projects in ing a study for the U.S. Department of achieve a successful-business startup. Agriculture of a five-state demonstra- this category that were not closelyco- tion project to test the feasibility of ordinated with local JTPA systemsran Conclusion ap- the risk of duplicating the services plying Food Stamp Employment and available from EDWAA or competing Like any major experiment, the Training Program regulations to the JOBS program for AFDC recipients. with the EDWAA system for clientsor DCA Demonstration haseven at this continuation funds. early stageshown some successes and some failures. When the Department The outcomes achieved by the of Labor requested proposals for these projects emphasizing basic readjust- grants three years ago, it deliberately ment services were mixed. Although invited applicants to "break the mold." most projects appear to be meeting the Project designers responded by taking placement rates specified in their pro- risks in setting goals, forming partner- posals, the placements sometimes took ships, selecting target groups, design- longer than expected, were not in a ing interventions, administering servic- training-related field, or were not at the es, and monitoring progress. desired wage level. Risk-taking leads to mistakes, and The projectspreparing dislocated this demonstration includes its share of defense workers for new careersidenti- mistakes. But risk-taking also leads to fied careers that they thought would be new knowledge, new models, new les- attractive to and appropriate forgroups sons. The DCA Demonstration is suc- of dislocated defense workers. Using ceeding in providing useful lessonson project partners with close organiza- how to support economic conversion, tional linkages to the targetedoccupa- whether or not the individual projects tions or industries, they succeeded in succeed in accomplishing each of their developing high quality trainingpro- objectives. grams in those fields. However, these

NW' 52 Evaluation Forum *Issue 11 60 Summer 1095 Features Editorial Commentary

Editörial COmnientary puters companies have been buying In Our share of world imports climbed; our record numbers, but not all of these share of world exports declined. At the Most economists agree with the workers were permanently retained. same time, over the past two decades, Clinton Administration's assessment Expansion has been less profitable in the living standards of more than 80% that economic growth has resumed in the context of international competition. of American families had dropped. the 1990s but at a slower pace than in On the side of a generally positive An important lesson about the new previous upswings. The International economic vision for the U.S. is the U.S. economic environment has been that Monetary Fund supports this conclu- Census Bureau's 1994 report indicating industries focusing on intense knowl- sion in terms of global economic condi- that Americans were living longer, edge bases, and industries achieving tions, predicting a gradual expansion smoking less, spending more money on significant "economies of scale," have from 3.8% in 1994 to 4.2% in 1996. The educational materials, owning fewer had a longer-term impact than others Mexican peso crisis and the falling U.S. guns, divorcing less, and staying mar- on technology performance In other dollar have led to cautions, however. ried longer. Infant mortality had economic sectors. Governments that But new trade agreements such as dropped. Per capita income measured have targeted such high-technology GATT have been considered positives in constant 1987 dollars had increased sectors for special support and devel- in this picture of the future. Disposable income had risen. The level opment assistance have encouraged In fact moderate growth is viewed by of educational attainment had in- economic progress. Governments that some economists as having real virtue. creased. But there was a dark side as have over-shielded major corporations Inflation is remaining low and forecast- well. Average hourly earnings mea- from competition have created econom- ers propose that a 3% growth rate will sured in constant 1982 dollars had de- ic disincentives. In fast-growing econo- lift Americans' after-tax income in real clined between 1980 to 1993. Median mies, government and industry have terms and reduce unemployment to an scores on Scholastic Assessment Tests worked as collaborative partners in de- acceptable 6%. However, they ac- had plummeted. The increase in Ameri- veloping critical industries facing for- knowledge that military downsizing and cans' expenditures on video and audio eign domination. In nations with the continued recessions abroad could equipment and personal computers had best records of industrial development, change these predictions. And it is sig- dwarfed their expenditures on educa- government, labor and management nificant that although corporate Ameri- tion. pursue manufacturing excellence to- ca spent an enormous amount on new gether, using a variety of policies and Meanwhile, international competition actions. They ensure that a high quality equipment in the 1990s in order to be- had grown dramatically, particularly be- workforce is available through a com- come more efficient and profitable, little tween the U.S., Europe and the Pacific petent, accountable education and job growth resulted. Only 2.3% was Rim. Studies of economic performance training system. And they give serious spent on expanding businesses. A in the early 1990s showed clearly that large portion of the increase in employ- American manufacturers in general and attention and energy to the readjust- ment and reemployment of workers ad- ment in the 1990s has been related to in a wide range of industries, including versely affected by economic change. the manufacturing, installing or pro- many high technology industries, were gramming of the machinery and corn- losing ground to foreign competitors.

_

Issue 11* Summar 1095 53 Evaluation FOrm "." Cornmentar

Robert B. Reich Basic Causes (1983 to 1993) the number of jobs in Secretary of Labor The fundamental fault line running sectors that typically require higher lev- through today's workforce is based on els of education has grown by an annu- education and skills. Well-educated al average of 2.8% while job growth in ==.1 Even though it has been a long time and skilled workers are prospering. sectors requiring less education has since the picture has been as bright as it Those whose skills are out of date or been only 1%. This shift in favor of is now for the average American work- out of sync with industrial change anx- skills shows up not just in the U.S. but er, the experience of the "average" iously contemplate their prospects. in other advanced countries. worker is becoming less and less rele- Those without education or skills drift In a seeming paradox of today's vant. Some workers are surging ahead, further and further away from the eco- economic news, financial markets fret others are treading water, and still oth- nomic mainstream. The notion that we that unemployment is too low to con- ers are sinking fast in the same are creating a bounty of bad jobs is a tain inflation, even while eight million economy, at the same time. The state of myth. Most new jobs are good jobs. willing American workers remain job- the American workforce, in short, is di- The problem is that tiv, jobs that remain less. Part of the answer to this apparent vided. for workers without shills or with the contradiction is that markets for highly In the late 1960s, as my generation wrong skills are becoming grimmer. skilled labor are becoming tight in was joining the workforce, the vast ma- Skills have always mauered, but many parts of the economy, creating jority of Americans were middle class, they have not always Or.,1 such potent the conditions that can kindle inflation in reality as well as in perception. Their determinants of economic destiny. As worries. But millions of less-skilled status and prospects differed only mod- recently as 1979, a male college gradu- workers remain idle or underemployed. erately from the average. In an aston- ate earned 49% more than a similar This wasted workforce is walled off by ishingly short time, the old middle class man with only a high school diplo- skill barriers from the leading edges of has splintered. The erosion of a sense ma a sizeable difference, to be sure, the economy. The best way to expand of shared prospects poses what may be but not too large for the two to share the economy's capacity and lower the our nation's most critical challenge of the label "middle class." By 1992, level of unemployment needed to bridle the post-Cold War era. Reversing this however, the average male college inflation is to dismantle these walls by erosion, giving all Americans a reason graduate was earning 83% more than preparing underutilized workers for to believe once more that hard work his high school graduate counterpart, more productive work. will lead to a better life, must become a and the notion of common prospects What lies behind these widening central policy objective. had faded considerably. The picture for disparities within the workforce? Man- Broad trends that have converged women is similar, if slightly less stark. ufacturing jobs were once the gateway and accelerated since the middle 1970s At every level of education and training to the middle class, even for workers have split the old middle class into the pattern holds: the higher the skill who started off without high-level three groups: an underclass largely level, the higher the earnings. And the skills. While it is a myth thateinterna- trapped in central cities, increasingtr gap ha:i been growing. tional trade has robbed America of its isolated from the core economy; an Not only the wages and benefits that manufacturing industries, technological overclass of those who are positioned workers can command, but also their changes have diminished the role of la- to profitably ride the waves of change; chances for holding a job at all, are di- bor, especially unskilled labor, in the and in between these the largest group, vided along lines linked to skills. This modern factory. And global trade and an anxious class, most of whom hold gap, too is widening over time. In the investment surely have hastened these jobs but who are justifiably uneasy 1970s, the average unemployment rate changes. Even within manufacturing about their own standing and fearful for for people who had not completed high industries, a rising share of value is their children's futures. Despite the school was 7%. By the 1980s, this rate added before the assembly line begins, progress of the past year, this division averaged 11%, and in 1993 it was over and after goods are produced. still casts a shadow over the state of the 12%. By contrast, the unemployment Labor unions have long helped American workforce. We can describe rate for workers with a college degree shore up wages and benefits even for this picture in greater detail with the or better has held fairly steady at workers without high-level skills. But help of new data, much of it recently around 3%. Over the most recent ten- today only 11% of the private sector compiled by the Bureau of Labor Sta- year period for which data are available workforce is represented by a union. A tistics.

4wwlsmaNdliwkik 'amImmEme Evaluation Forum Issue 11 Summar 105 commentary

en who have graduatedfrom communi- revitalization of the labor movement However, the road to opportunity is often much rockier for people of color. ty colleges earn 33% morethan women would help reverse the erosion of the with only a high school diploma. middle class. Last month the national unemployment rate for black Americans was over The most striking change in the 11%, more than double the rate for Middle Class Revitalization workplace has been the brash arrival of white Americans. Today black men Skills clearly matter, and they can be the computer. In 1984, about 25%of hold only three out of every one hun- learned. We have to combat the forces American workers used computers on dred managerial, professional, and dividing America's workers with an ar- the job. Last year almost 47% did so. technical jobs. The median income for mory of new approaches tolifelong And contrary to the belief that comput- black males working full.time and full- learning. An additional 130,000 chil- er-literate youth are running circles year in 1992 was $22,400,just 72% of dren each year can now be made ready around their technophobic elders, the median income for comparable to learn at school through HeadStart. workers between ages 40 and 54 com- white males. Hispanic Americans seek- School systems throughout America prise the group most likely to work ing to join the middle class have con- have millions of dollars in new incen- with computers. Even workplace com- fronted similar obstacles. Their unem- tives to improve their performance. munication has been transformed, as ployment rate tops 10%, and their During the summer of 1994, some one in ten workers now useselectronic earnings have been steadilyeroding. 120,000 disadvantaged youth combined mail to communicate with colleagues And Hispanic workers are significantly jobs with classroom instruction. Over around the corner or around the world. less likely than their white counterparts the next six years, almost half amillion But the computer revolution has to receive health insurance onthe job. young Americans will beentering youth during the last decpened the division of the American Closing these gaps requires a long, of them workforce. Two-thirds of college grad- painful process of national growth and two years of high school, many uates use computers at work, butonly healing to which we must remain com- receiving special skill training beyond one-third of high school graduates, and mitted. Virtually every willing worker high school. fewer than one in ten high school drop- can expand his or her skills.Informa- Starting in the fall of 1994, 20,000 outs. The vast majority ofmanagerial, tion technology itself promises to vast- young people will enter NationalSer- technical, and professional workers use ly increase the productivity of learning vice, earning money that they can apply computers. But people in lower-pay- as it makes top-flightteaching tools to a college education. The threeand ing, lower-skill occupations use them more widely available. Wealready see one-half million people who take out far less frequently. The information emerging around us examples of so- education loans each year now have the highway promises to speed some peo- phisticated yet accessible work that can option of repaying their loans as a per- ple ye desirable destinations, but it may form the platform for a new middle centage of future income. Fifteenmil- be leaving others stranded in the high- class. Some of the most notable job lion working families with modest in- tech version of inner-city ghettos. A re- growth has occurred among technicians comes are receiving taxrelief. We have cent study of census data found that who defy the traditional categories of begun transforming the old unemploy- fully 5% of America's households lack the old economy. Their ranks are pre- ment insurance system into areemploy- access to a telephone, thebasic ticket dicted to grow by nearly 40% over the ment system, insuring that in1994 of entry to the information web. Other next decade. Data released recently alone an additional 150,000 Americans data show that minority children are show that the most rapid job growth is who have lost their jobs will get the less likely to use computers at school in high-paying occupations, even with- skills or the job search help they need be- or at home. As personal computers in traditionally low-paying industries. to find new ones. We must alsodeliver come standard equipment forlearning This hints at the proliferation of new on health security, or ourmission of at home, children who aredisconnected kinds of middle-class jobs throughout turning the anxious class into a new, from the system will fall further and the economy. more secure, more productivemiddle further behind. The technician jobs that will sustain class cannot be completed. Ethnic Divisions in Rebuilding the core of the new middle class usual- American business has a crucial role the Middle Class ly require some education beyond high to play in building a new middleclass. school, but they do not always demand Skills learned on the job, or in a work- Despite the issues cited, long-term a four-year college degree.Degrees related setting, tend to be especially data affirm the potential for building a from community colleges are paying well-tailored to the requirements of the skill-based divi- new middle class. The off, and their enrollments are rising. workplace. New data on work-related sions of today's workforce are in some Men with community college degrees training show that the impact of such than the ways more readily overcome earn 26% more, on average,than men training is of the same magnitude as divisions based on race and gender that with only a high school diploma. Worn- more traditional schooling.Men and have haunted America throughout our history.

5$ Evaluation Forum Issue II Summer 1995 63 maL...... ,1!ffWO.,2 _

Comment

women who graduate only from high all workers along on the routeto pros- If unchecked, these divisionscan school, but who have received work-re- perity, investing aggressively in their lated training, earn more than people corrode our society. Unlike the citizens skills and making them partners inpro- of most other nations, Americans have who have attended some college but ductivity. This new compact isan im- who have not received any additional always been bound together less bya perative not only of corporate citizen- shared past than by shared dreams of training, and almost as muchas college ship but of firms' own long-term a better fiture. If we lose thatcommon graduates who lack additional work- interests. based training. future, we lose the glue that holdsour nation together. Nothing ismore vital Yet most companies are not yet do- Achieving A Better Future to fulfilling our nation's defining ing enough training, and what training A constant theme of American histo- promise than preparing all Americans they do provide is not beii.g directedto ry has been the challenge of forging for meaningful, productive working the workers who most need it. Among unity out of diversity. Weare at an im- lives. In this century's waningyears, in young college graduates, about 35% portant point once again in meeting this the hopeful, fearful confusion of the now receive work-related training. That challenge. This time the deepest divi- Cold War's end, our most important is nearly douLle the rate for high school sions are not based as muchon race, mission is to restore hope to theanx- graduates and more than four times the national origin, or geography. Theyare ious class and lift despair from theun- rate for high school dropouts. The based on the ability of individualsto derclass, and to affirm the conviction workers most likely to receive training make their way in an increasinglytur- that the American dream of broadly are white, male, well-educated, and bulent economy. The overclass is doing shared middle class prosperity stillen- working in high-paying occupations. fine, but questioning its connectionto dures. This imbalance serves only to harden the rest of America. The underclass is the divisions within the workforce. isolated in marginal enclaves walled off The payoff to work-based training, from hope. The anxious middle classis and the inadequacy and being pulled and stretched by the need uneven distri- Thecommentary above bution of such training, highlights the to work two or more jobs to keepa is excerpted family solvent, by uneasiness about from Secretary Reich's 1994 Labor Day need for a new social compactamong speech delivered at the Center for Na- American workers, business, andgov- health care, by the specter that today's tional Policy in Washington, D.C. ernment. Businesses need to do their job will disappear tomorrow, and by part in this new compact by bringing fears that their children will be denied the opportunity for a better life.

Evaluation Forum.Issue 11s Summar 1995 6 4 interviews

Ertitorial IntroductiOn mocracy and a market economy. This is Faced with the necessity of unexpected a uniquely informative macro-perspec- layoffs in the 1990s, the company and In this issue of Evaluation Forum we tive on economic restructuring and its its unions, working with federal and are privileged to feature our first inter- Implications. state agencies, have developed a mod- el reemployment program to assist dis- view with a respected European expert. In addition we are pleased to include This extensive interview addresses located Boeing workers in a period of our first interview with representatives adjustment to economic change and some of the fundamental economic of management and labor In a major challenges facing all industrialized na- U.S. multinational corporation the the internationalization of markets. tions, and highlights the situation for aerospace giant, the Boeing Company. countries in transition to political de- An Interview with Salvatore Zecchini Treasury, the Minister for Agriculture, Editor s Introduction" ropean nations, and the independent republics of the former Soviet Union, in and the Minister for the Coordination of Given the emphasis in this issue of their transition to market-based econo- Community Policies. He represented the Benca d'Italia in international bod- Evaluation Forum on industrial econo- mies. mies and their efforts to survive in a ies addressing economic issues in Eu- new competitive global marketplace, The Assistant Secretary-General re- rope, and participated in the technical we are pleased to have an interview ceived a Ph.D. in economics from the meetings of the Deputies of the Group with Salvatore Zecchini, the Assistant Wharton School of Finance in the Unit- of Ten. He was also a member of Italy's Secretary-General of a strategically im- ed States, and is a former professor of delegation to the two technical negotia- portant international organization, the economics in Italy. He subsequently tions for the establishment of the Euro- Organization for Economic Cooperation served as economist for the Economic pean Monetary System and European and Development (OECD) in Paris, Research Department of Bence d'italia. budget reform. France. Established in 1960, the From 1984 through 1989 he was Exec- utive Director of the international Mone- In addition, Mr. Zecchini has pub- OECD's mission has been to promote lished numerous studies on corporate economic growth, sustainable employ- tary Fund. In 1989 he was named Spe- cial Counsellor for Economic Affairs at finance, European financial policies, ment, and a rising standard of living in the consequences of Italy's member- member nations. Its membership in- OECD, and began his tenure as Assis- tant Secretary-General and Director of ship in the European Union, interna- cludes all the major advanced industrial the Centre in 1990. tional economics, monetary and finance nations, currently twenty-four countries. issues, and the transition from central Mr. Zecchini is also Director of Mr. Zecchini has had an Impressive planning to market economies. OECD's Centre for Cooperation with career, serving as economic advisor to Economies in Transition, which offers the Prime Minister of Italy, the Minister assistance to Central and Eastern Eu- of Foreign Affairs, the Minister of the

Question: For Central and Eastern agea wide-ranging program of dia- relatedto the transformation of the eco- Europe, the transition to a market econ- logue and cooperation with the transi- nomic system, and the integration of omy clearly represents a difficult his- tion countries. The goal of the OECD's these reforming countries in the world torical passage. This is not unexpected cooperation has been to contribute to- economy, and 3) training activities. wards the development of pluralist de- given the magnitude of the economic, With regard to the subject of this is- mocracy and market-oriented econo- social and cultural changes occurring in sue of Evaluation Forum, itisimpor- these societies. How has theOrganiza- mies by mobilizing the experience and tant to note that from its inception the tion for Economic Cooperation and expertise of the OECD secretariat and CC751' has given high priority to the Development(OECD) addressed the Member Countries. The cooperation human resource issues related to struc: challenges involvedin this transition? takes a variety of forms, including tural change in the transition countries. 1) provision of advice on the design, In particular, I am referring to educa- Mr. Zecohini:In 1990, the OECD implementation and monitoring of ap- Centre tion, social and labor market issues. In Council decided to create the propriate policies, 2) policy dialogue the framework of the CCET's work for Co-operation with Economies in with the Organization and its Member to launch and man- program for these areas, the OECD's Transition (CCU) Countries on the maineconomic issues

Evaluation Forum Issue 11 Summer1925 65 57 Interviews

Directorate for Education, Employ- Conference on Labour and Social Im- some cases we are still struggling to ment, Labour and Social Affairs holds plications of Structural Change in Cen- solve them. Fortunately, there are many responsibility for provision of technical tral and Eastern Europe. The proceed- encouraging developments in terms of assistance. The work program is fo- ings were subsequently published. Did institutional and economic reform. The cused primarily on 1) improved moni- this contribute a foundation for future leading economies in transition have toring of educational, labor market and OECD technical cooperation? turned the corner and are once more ex- social developments, and 2) design and Mr. Zecchini:The High-Level Con- periencing economic growth. implementation of an appropriate poli- ference was a concrete and early mani- cy strategy that aims at helping to es- Question:High unemployment has festation of our concern for labor mar- tablish well-functioning labor markets, become a troubling feature of the labor social safety nets and educational sys- ket and social issues in the transition market in most transition countries. tems. countries. The main conclusions of this Could you comment on unemployment conference remain relevant to the situa- trends and the underlying dynamics? Specific reviews of education, labor tion today. Broadly speaking, three Mr. Zecehini: market and social policy are the main points are of particular relevance: The first phases of transition to a market economy in the instruments used to pursue these goals. s' First, during the transition an appro- In cooperation with the corresponding Central and Eastern European countries host country Ministries, these reviews priate set of human resource policies (CEECs) and NIS have brought about survey and analyze the full range of can work to help address major major changes in the economic and so- available information on the recent de- social and labor market problems re- cial fabric of each country, including velopments in a sector. Drawing on this lated to structural change these widespread employment adjustments information, the current policy frame- policies can facilitate a smooth func- and, in most, a rapid increase in open work and institutions are examined tioning of the labor market while unemployment. Although these adjust- ments are an inevitable consequence of with a view to offering a critique of the providing the .foundation for the so- situation, highlighting positive develop- the process of economic transforma- cial safety net, and the promotion of tion, they are producing growing hard- ments while offering suggestions where the social consensus for reform. performance falls short. To date, these ship for the population as a whole, and reviews have focused on the four transi- IISecondly, these policies can promote for certain vulnerable social groups in tion countries participating in the longer-term economic development particular. Furthermore, unemployment and poverty have created pressures on OECD's Partners in Transition Pro- through enhancement of human cap- state budgets due to increased outlays gram: the Czech Republic, Hungary, ital, and promotion of cooperative for and social Poland, and the Slovak Republic. This industrial systems. year we will add to the list, as the first assistance. They are thus creating new education review of the Russian Feder- Thirdly, they can strengthen plural- constraints for the disinflationary poli- ation is undertaken. istic democracy by improving the cy strategy as well as for the reform of public expenditures. Besides the review process, the on- representation of the interests of the going series of activities targets a range social partners. In most of the CEECs, unemploy- ment materialized in a wave, following of technical and policy issues using a The conference proceedings were pub- variety of approaches, including coun- the institution of market-oriented re- lished by the OECD in a volume enti- forms. Already at the end of 1991, un- try-specific case studies, expert mis- tled Structural Change in Central and sions and seminars, and international employment rates were close to 10% Eastern Europe: Labour Market and (or even above) in Bulgaria, Hungary, conferences and workshops. The imple- Social Policy Implications. The first Poland and Slovakia. In these coun- mentation of the market-oriented chapter of this book addresses key is- tries, unemployment rates continued to recommendations drawn from these ini- sues in the transition that remain cen- grow at a fast pace during 1992. The tiatives is supported through training tral even as the transition has advanced. activities designed to reinforce the Czech Republic, which was the subject These include 1) sluggish growth in of a recent OECD labor market review, working capacity of host country insti- these economies, 2) privatization and tutions. The OECD is also contributing is a significant exception to this pat- enterprise restructuring, 3) a rise in un- tern. After initially experiencing a rise important statistical tools through the employment and labor marginalization, development of education and labor in unemployment (although at a lower 4) increased poverty, 5) widening eco- pace than most CEECs), there was a statistical databases that provide inter- nomic disparities between social significant decline in the number of job nationally comparable data series for groups and between regions, and 6) in- several transition countries. seekers in 1992, and a stabilization in ternational migration. the unemployment rate thereafter Question: In 1991, the OECD orga- Although much has happened since (around 3%). nized jointly with the International La- the time of the conference, the issues it bor Organization (ILO) a High-Level The Russian Federation too has raised proved to be right on target. In shown distinctive labor market pat-

66 Evaluation Forum Issue 11 Summer 1995 e

Interviews

inflows into unemployment have been ily overcome by geographical labor terns. In general, employment in Russia mobility. Rather they require, at least in has been declining much more slowly relatively contained,evenwhen com- pared to those prevailing in low-unem- the short term, specific efforts in the than output. In the three years since backward areas, and a redistribution of 1991, employment is estimated to have ployment OECD countries. Also, there resources to them from wealthy areas. dropped by roughly 4 million to under have been significant flows that com- 70 million. Compared to most of the pletely by-pass unemployment. Direct Unemployment has risen and contin- CEECs, unemployment remains fairly job-to-job shifts and flows out of the ues to grow in some areas. However, low, with estimates of the unemploy- labor market have played an important this must be contrasted withongoing ment rate ranging from roughly 2% role in the reallocation of labor. reallocation oflabor and someeco- (based on administrative data on the Put another way, unemployment has nomic restructuring.Many new jobs registered unemployed) to about 6.5% been a rather stagnant pool, especially have been created although they are (based on estimated Labour Force Sur- in the earlystages of the transition. In somewhat precarious, particularly those vey updates) in the fourth quarterof most of the transition countries, once a in small firms and in self-employment. 1994. As of October 1994, the popula- worker has lost his or her job it is very Some of the newly created businesses tion officially registered as being "out difficult to obtain another position. Un- are yielding good wages andbenefits, of employment" was 1.75 million. Re- der these circumstances, unemployment but many do not and will not. cently unemployment has begun to rise ishighly persistent,and in many cases Question: What have been the main and is likely to continue to do so, espe- the lack of job opportunities results in elements of CEEC and NIS human re- cially if the transition process gains the job seeker leaving the labor market source policy in the face ofeconomic momenttua. altogether instead of being reintegrated restructuring? into a job. Unemployment rates by themselves, Mr. Zecohini: The transition countries however, offer only a partial picture of As is the case in most high-unem- have been quick to initiate develop- the labor market slack during the tran- ployment OECD countries, the burden ment of human resource policies and sition process. Sizeable flows from em- of unemployment is unevenly spread programs corresponding to the require- ployment to out-of-the-labor force have across the different social groupsin ments of a market economy. There occurred. This is especially true where transition countries. It has fallen dis- have been significant successes in this extensive use of soft dis-employment proportunately on theyoung, unskilled regard, although further adjustments in measures such as early retirement and older workers,and someethnic mi- the new policy framework or policy and "easier" access to various kinds of norities.Furthermore, even in a context implementation processes will be re- was made in the disability benefits of stable or declining numbers of job quired. Moreover, the task of restruc- early stages of the transition, and where seekers, the average duration of unem- turing elements carried over from the most working pensioners were pushed ployment has increased dramatically. previous system has not been complet- out of the labor market. Flows to inac- Many long-term unemployed have left ed. tive status (rather than into unemploy- the labor market.These phenomena are ment) have also occurred among indi- signaling the difficulties now encoun- Labor market policyhas been at the viduals of working age, especially in tered by certain social groups, which center of the reform efforts. Early in the Czech Republic and Hungary. In should be the focus of more attention the reform process, transition countries the latter, for example, the participation by labor market and social policy insti- enactednew comprehensive employ- rate for the working age population tutions. ment lawswhich are serving as the ba- dropped by some three percentage sis for labor market policy and pro- points during 1993. In Russia, there has The economic and social hardship of grams. These laws generally affirm the been a substantial growth in u.nderem- reforms also hasa geographical dimen- rights and responsibilities of people in ployment and wage arrears, as well as a sion,which was the subject of a Tech- employment matters, including the continuing decline in labor productivi- »ical Workshop organized by the choice of whether to work and of occu- OECD and the Institute for Advanced tY. pation. In many cases such laws bor- Studies in November 1994. In most of rowed from similar statutes of OECD In September 1993, the OECD orga- the transition countries, large urban Member Countries. Building on these nized a Technical Workshop on "The centers have unemployment rates well employment laws, transition country la- Persistence of Unemployment in Cen- below 10% and rather favorable ratios bor market policies and programs have tral and Eastern Europe." Evidence pre- betweenunemployment and job offers, generally sought to achieve: sented there suggests thatunemploy- while rural areas and many mono-cul- ment has grown in many cases more as tural industrial sites are suffering from Reduction in the level of unemploy- a consequence of low rates ofoutflow unemployment rates above 20% and ment. into jobs, than of unusually high rates few job openings. Given the rigidity Decreased labor market segmenta- of inflow into unemployment.Despite and tight supply in the housing market, tion and improved labor market the massive employment adjustments in these regional disparities cannot be eas- most countries of the region, monthly

1INIMINNIIIIIM~ 59 Summer 1995 Evaluation Forum A Issue11 67 ..r;)

= _

Interviews

prospects for vulnerable groups in pension eligibility while increasing the budgetary pressures is constraining the society. average payment size. The implementa- modernization of education. e Promotion of occupational and in- tion of such strategies is fraught with political risks end consequently does Generally, the s) nem of training in dustrial mobility, and delivery of not seem feasible over a short period. the transidon countries placed empha- basic income support for the unem- Family allowances vary, and in some sis on enterprise-based programs. The ployed during limited periods of job cases are based on inappropriate per transition has led to a substantial reduc- search. capita measures that do not take into tion in the number of training places 6,r4) account household economies of scale. available in state-owned enterprises, Employment laws have provided for the and the private sector has been unable establishment ofpublic employment Approaches to tbe provision of further social assistance have varied, depend- so far to provide sufficient opportuni- senfices charged with the administra- ties for initial and tion of unemployment compensation, ing in part on the level of decentraliza- tion in the system. In some cases, the training. This comes at a time when placement assistance and additionalac- there is a major need for retraining tive labor market programs such lack of national norms has causeda as workers who have lost their jobs,or for training and public works. In some cas- wide variation in the availability and level of benefits. In the Russian Federa- training young entrants into the labor es, the public employment services market. were established as independent agen- tion, for example, such assistance is cies. The development of these services primarily the responsibility of the local Question: During the 1991 OECD- has been the focus of substantial and regional governments. ILO conference, migration issueswere amounts of international aisistance, in- The education policies of the transi- raised as an area of possible futurecon- cluding the technical cooperation activ- tion countries need substantial andur- cern. How has this issue developed, ities of the OECD. gent reform as well. As stressed in sev- and how do you see it evolving? Across the transition countries, the eral studies conducted by the OECD Mr. Zecchini: Despite thesevere limi- structure of social policy is still under- during the last three years, there isan tations imposed by the centrally- going substantial restructuring and/or urgent need for greater coherence be- planned governments, considerableex- development. Witness the fact thatsev- tween labor market and education poli- ternal migration flows have beenan eral countries have recently introduced cies. Education reviews carried out by aspect of the socio-economic develop- new social laws or policies for which it the OECD in the former Czech and ments of central and eastern European is still too early to assess the effects. Slovak Republics, and Hungary and countries during the past decades. The Given the wide range of issues, I will Poland, have identified the following ongoing transition process has radically focus on three aspects of social policy common problems: changed the nature, if not the magni- that are closely tied to poverty issues: The lack of long-term plans for the tude, of migration flows. , family allowances and social develnpment of higher education, Continuing emigration flows are assistance. and the need to render the system likely to result from the labor market In the old system, inefficienciesen- more flexible in responding to the slack, especially in certain backward sured high demand for labor. Employ- needs of a market economy. regions. Cross-border commuting has ment was virtually guaranteed for those also emerged in regions on the border who could work, thereby ensuring in- Problems relating to the capacity of with the West. Evidence presentedat a come during one's working life, and a educational institutions. recent OECD conference: on Regional pension in the event of disabilityor re- The growing social differentiatibn, Unemployment in Transition Countries tirement. As a result, social assistance suggests that this is directly related to and the problems of identifyingso- targeted only a small minority of the cial needs. the labor market conditions of there- population, such as the handicappedor gion of origin, and the existence of for- orphans. The quality of the teaching staff, and mal agreements between the counties of origin and destination, Today, pensions and family allow- "brain drain" problems. as is the case between the Czech Republic and Ger- ances serve as a primary means for de- The autonomy of institutions, the many. livering social protection. In reforming decision process, the legislative countries, restructuring of the pension Official in-migration data suggest foundation, and the division ofcom- system is required to ensure long-term that there are ;ncreasing flows toward petencies within institutions. viability, particularly in the face of low transition countries, due also to there- retirement ages and an erosion of the In all cases, financial resourcecon- turn of former residents. However, offi- value of benefits due to inflation. straints are proving to bea major con- cial data may under-report inflows of foreigners in transit through To combat low benefit levels, cern. Competition with other govern- some tran- many sition countries toward Western Euro- strategies for reform aim to restrict ment priorities in the face of heavy

14119.M.1101!

Evaluation Forum issue 11 A, Summar 105 68 True

Interviews

particularly during the economic transi- introduce flat rate systems on a tempo- pean countries, as as the casefor Po- Another option tion. Economic policy must address a rary basis, as in Poland. land. The cooperation between transi- of un- broad range of structural impediments for improving the effectiveness tion countries and the OECD in the employment benefit systems mightbe migration had al- to growth, including thoserelated to arta of international human =sources. Let me give a few ex- to improve the targetingof support ready started in 1991 in theframework measures by reducingbenefits for those amples. of the Continuous Reporting Systemof who voluntarily quit their jobs,and for Migration. This system is planned to Problems in policy coherence are a new entrants to thejob market. further contribute to the improvement conunen issue in transitioncountries, Improved systems to monitor per- of information on the dimensionof the as in OECD MemberCountries. In in-migration phenomenon, on one formance, and provide feedback to em- transition countries, OECD surveys aid hand, and the definition of suitablepol- have foand particular problems in the ployment service offices, may also effective- icies to cope with it, on the other. linkages between unemployment bene- in raising the efficiency and ness of labor market programs.Anec- Inter-regional migration is also an fits and the social assistance system, particularly concerning the long-term dotal evidence from some transition issue of great importance in al/transi- countries suggests that well-designed tion countries. As stressed before, re- unemployed. As a consequence, in many countries a significantnumber of management information systems can forms have brought about major re- contribute toward needed performance and the long-term unemployed do not bene- gional disparities in the economic enhancements. labor market conditions of different re- fit from the placement andcounseling gions. indeed, labor migration flows services of the employment services. As regards social policy, pensions Many lose contact with the labor mar- seem to respond to differencesin the are of the greatestpressing financial ket. Other gaps in policy coherence can concern. As mentionedearlier, the economic conditions of regions. The of major urban centers, and other regions be found in the relationship between long-term viability and effectiveness education and labor market policy,with with a diversified production base, pension systems in most transition in-migra- the result that graduates of educational countries will require a tighteningof el- have experienced consistent education that tion of labor in the last period.Howev- programs risk having an igibility criteria. fits poorly with the skills sought inthe er, inter-regionalmigation flows have In the area of education, theeffec- not increased in aggregate termsin all labor market. tiveness of the system should be en- transition countries, due to the majorri- In terms of labor market policy, hanced by stressing the linkdgeswith gidities in the housing market, and the there is some evidence that many tran- the labor market. One means for lack of information regarding job op- sition countries could benefit from im- achieving this is to increase the in- portunities. proved targeting of public programs. volvement of the various stakeholders of ed- Question:Each of the transition Given the limited amount of resources in the design and implementation counties has entered the reform pro- available for these programs, and the ucational programs that is, govern- unemployment, cess in the context ofunique exper- uneven distribution of ment representatives fromnon-educa- iences and conahions which will re- it is most appropriate to focus on the tion agencies, business representatives, the quire varied policy responses. Beyond most vulnerable groups, such as and parents. More generally,education in this, however, are there common long-term unemployed. Targeting would benefit from improvements themes that you have identified for en- might also take into account the geo- national standards. graphic distribution of labor market hancing human resource policy? These examples do not constitute a hardship. Also, evidence from the is important to un- comprehensive policy framework. Mr, Zecchink It OECD's review of the Czech labor poli- derline the importance of a lesson that market indicates that investing more re- However, they signal some of the cy components that mosttransition we have learned in theOECD Member sources in assisting theunemployed in countries need to consider within the Countries. Experience from a wide their job searehes, and in monitoring range of -countries hasshown that the job search efforts of the unem- ongoing transition process. sound economic policy is the bestbase ployed, leads to better results than oth- from which to promote employment.As er methods in term of jobplacements. the recent OECD Jobs Study points out, .Editprs"fOlo'. of such a policy Many transition countries might a primary component The interview with Mr. Zecchinl was consists of an appropriate macroeco- benefit from the simplification of the unemployment benefits system. Men completed In February, 1995. The fol- nomic policy that includes soundpub- lowing OECD publications may be ofIn- these systems involve complicated pro- lic finances, price stability, and promo- terest to readers wishing to pursuefur- tion of a =stainable balance of cedures and calculations that strain the ther the issues covered In the Interview, administration, while at the same time payments position. However, asound Again, CCET stands for the Centre for macroeconomic policy by itself cannot delivering insufficient benefits. One Co-operation with Economies in Transl- option for simplification might be to guarantee sustained economicgrowth, ton.

IMINN14.110.1 1005 EVaIumion room Issuo ft* Summer 69 Interviews

Labor Market and Social Policy Issues: Review of Labour Market and Social Education Issues Policies of Hungary. Paperpresent- Employment and Unemployment in ed at the ELSA Committee Meeting Review of Higher Education in the Economies in Transition: Conceptual In Budapest, Oe:ober 1994. Czech and Slovak Federal Repub- lics: Examiners' Report and Ques- and Measurement Imes. Eurestat. Short-Term Economic Indicators CCET. Paris, 1993. tions, 1992. Transition Economies. Labour Mar- The Labour Market in Poland. CCET. ket Indiattors Annex, No. 4, 1994. Review of Education Policy in Hungary: Paris, 1903. Examiners' Report and Questions, Unemployment in Transition Countries: 1993. Structural Changes in Central and Transient or Persistent. CCET, Eastern Europe: Labour Market and 1994. Follow-Up Review of Higher Education Social Policy Implications. A Joint in the Czech and Slovak Republics. publication of the OECD and the The Regional Dimension of Unemploy- Report to OECD's Education Com- ILO. Paris, 1993. ment in Transition Countries: A mittee, 1994. Challenge for Labour Market and Current Labour Market Problems in Social Policies. Proceedings of the SeminarQuality Assurance and Ac- Russia. General Distribution Docu- Technical Workshop on "Regional creditation in Higher Education. ment. CCET, 1989. Unemployment in Central and East- General Distribution Document. ccEr,1994. Review of the Labour Market in the ern Europe" jointly sponsored by the Czech Republic. Paper presented at OECD's CCET in Paris and the than- Seminar it: Mobility in Higher Educa- tute for Advanced Studies in Vienna, the ELSA Committee Meeting in tion. General Distribution Document. November 1994. Prague, December 1994. CCET, 1994. Seminar Research in Pedagogy. General Distribution Document. CCET, 1995.

An Interrview withGary K. Jacksonand John E. Dryer ..tntialieti'' aevelopment of 1AM hourly employees petitive market In which aerospace at Boeing through a number of unique On April 17, 1995, the United States services. companies operate, and the volatility of Secretary of Labor, Robert Reich,came orders under government contracts, to Washington State to meet with the The Quality Through Training Pro- The Boeing Company has alwaysex- co-directors of the IAM/Boeing Quality gram (QTTP) has piayed an integral perienced changes in the volume and Through Training Program which Sec- role in planning and obtaining thefund- characteristics of its workforceover retary Reich called a model of labor- ing for the Boeing Reemploymentpro- time. This has been related to thegen- management cooperation. 'Weare be- gram, which has two service centers of- eral health of the economy, to tradere- ginning to use this model around the fering a "one-stop shopping" approach lations, and to the state of technology. country, and it's I/Asking? Secretary to retraining and job placement servic- However, by the late 1980s Boeingwas Reich said. "Labor turd Management es for drspiateed Boeing workers. The facing a new competrive enrironment can work together, even when thenews Boeing Reemployment Programis ate that was considerably internationalized. for workers is bad? The Secretaiyalso nimstered by the State Employment The company was competing withsub- brought a check for $4.5 millionin sup- Security Department (ESD), andbrings sidized aerospace designers andman- port of the Boeing Reemployment Pro- together The Boeing Company, the ufacturess In other countries. There gram. company's two major unions the In- were recessions in the Industrialized ternational Acsociation of Machinists nations. And defense contractors, Leading the Quality Thrnugh Train- In- and Aerospace Workers (IAM &AW) cluding Boeing, were coping wichthe ing Program, John Dryer, Boeing Co- and Seattle Professional Engineering military downsizing that accompanied Director, and Gary Jackson, IAM Co-Di- Fmployeoe Association (SPEEA),local the end of the Cold War. rector, work together with the supportof Private Industry Counclis (PiCs),a con- The speciler of business losses a team of Administrators from both the sonium of seventeen Community that company and the union Ic nevelop and could result front the reducednumbers Techmoal Colleges, and theOTTP in a of ordere and the cancellation training opportunities that assist 1AM- broad-based drive to help dislocated or post- represented Boeing emsloyeesimpact- ponement of previoesiy-committedor- workers and mitigate the effects ofjob dors from airlines, who ed by technology change, jobccmbina- loss. were them- tons, and workforce reduction. The selves experiencing a business program is also proactive in developing The aerospace undustry byIts very downturn, brought about the needto programs for the career and personal nature experiences peeks and rilleys reassign, re:ooate, or lay offunprece- of employment. Because of thecorn- dented numbers of employees.In addl- 'We would like to convoy our special thanks. to Ellen I !swill, a slatperson ia IOTTP, who contribtatte substentiallyto this introduetion,

62 1401#101MIONWIAM1StallOsaINNINOC. 70 Svaluatton Forum 1. !raw* 11 Summer MO . eaSerea --4 '

Interviews

center" concept was included in an ap- 4. tion, because of the intensifying inter- sea, to assist them in preparing for new national competition from state-subsi- career options. Funded under the 1989 plication for federal funding of two ma- dized aerospace companies, Boeing and 1992 collective bargaining agree- jor Boeing Reemployment Centers in has had to drastically, pare production ments by an amount equivalent to 10 the Puget Sound area. costs, exacerbating problems for work- cents per hour for each IAM-represent- John Dryer, Gary Jackson, and ers. ed worker employed, with a guaranteed members of the Quality Through Train- $10 million base, the Quality Through Although employment at the compa- ing Program staff, along with represen- Training Program did much to help tatives of the company, the IAM and ny was at its peak in 1989, the IAM and IAM-represented employees beoome the company were well aware of the SPEEA unions, the Employment Secu- reemployed by offering an education rity Department, Private industry Coun- A forces that could affect employment assistance program and opening "Hori- conditions, If not the exact events that cils, and the State Board of Community zons" learning centers offering advis- and Technical Colleges formed a Labor were to follow. Consequently, negotiat- ing, counseling, career exploration, and ed into the collective bargaining agree- Management Committee and started an skills training. QTTP also saw the need intensive effort to apply for funding un- ment that year was the new jointly-run to provide a placement function for em- Quality Through Training Program, es- der the Job Training Partnership Act. ployees. tablished to develop training programs Charles Wetmore and Delia Curry of for IAM-represented employees impact- In 1993, QT112 contracted with Main- the QTTP staff led the team that put to- ed by the above-mentioned conditions stream Access and worked with Boeing gether the funding package. The appli- in the workplace. Outplacement and the Washington cation for funding featured the ESD as State Employment Security Department administrator for the program. This ap- There were almost no layoffs during plication was granted and an unprece- a (ESD) in a Job Opportunities Program the period between 1983 and 1990 to deliver services to IAM workers. The dented $5 million was awarded to the period of expanding business and high focus of the Job Opportunities Program project the highest up to that date for production for The Boeing Company. was to provide the supportive environ- any Washington State program. The problem then was building an ade- ment and services needed to help lAM 4 quate workforce to produce the compa- The Boeing Reemployment Pro- employees become reemployed as ny's airplanes. gram, a comprehensive program deliv- quickly as possible. Job clubs were ering a full range of reemployment ser- Reductions in defense-contract work formed, where employees with similar vices at two one-stop, multi-service brought the first cloud on the horizon, interests and goals met together with a centers a remarkable private/public resulting from the easing of the Cold job counselor on a regular basis. The partnership between The Boeing Com- War and the federal government's job clubs also afforded the opportunity pany, its unions, and government agen- downsizing of military programs. This to learn interviewing and resumé writing cies and institutions was born. affected the Boeing Defense & Space skills. Job fairs, job and skill matching, Group. Then came substantial aircraft and job placement services were in- a aa productien cutbacks directly related to cluded in the mix. The interviewer had the opportunity the decline in the commercial aircraft Services provided by ESD's local to talk with the two OT1-12 co-directors market worldwide. Boeing Commercial centers were alsO inciuded, such as as- in March 1995. The following interview Aircraft Group had to resize as produc- sistance with the Unemployment Insur- is the result of this meeting. The inter- tion schedules shrank. Since Washing- ance Claim process, and job search viewees are John Dryer and Gary Jack- ton State's economy was to a signifi- and job placement on-line through the son. cant extent dependent on Boeing, the Employment Service. ESD representa- Gary Jackson Is Co-Director of the downsizing made necessary by these tives were located on-site at the OTT? IAM/Boeing Quality Through Training cutbacks also had important ripple ef- offices. Since Q17P was certified by Program, which is jointly sponsored by fects on the hundreds of firms in the re- the ESD to gran Commissloner-Ap- the International Association of Machin- gion that constituted Boeing's extensive proved Trainingnder Title Ill of the ists and Aerospace Workers and The supply and service chain. Job Training Pa nership Act in fact it Boeing Company to address the train- By the end of 1994, Boeing employ- is the only private organization to enjoy ing and employment needs of active ment had dropped 20,000 from the that status this program also provid- and laid-off Boeing workers who have 1989 peak of 107,000 employees, as ed the benefit of being able to draw un- been affected by changes in the aero- reported by Secretary Reich during his employment insurance while going to space industry. visit here. This included attrition due to school through OTT?. and , as well as Mr. Jackson, an experimental elec- The Job Opportunity Program gave tronics technician while at Boeing, held layoffs. According to Washington Em- rise to a new, expanded concept one numerous Union positions as a Local ployment Security Department figures, which brought wider participation from Lodge officer prior to becoming a full- 8,000 employees were laid off during the community and provided a full time staff person with District 751 of the 1993 the largest during any one year range of benefits to all dislocated em- International Association of Machinists in the early 1990s. ployees of The Boeing Company. IAM- and Aerospace Workers. He has The Quality Through Training Pro- represented employees still took the served in the Union's Health and Bene- grarn built up its Career and Personal major "hit" from downsizing the work- fits and Wage Determination offices, as Development Services for both active force, but other areas of the workforce well as Contract Negotiations. He has and laid-off 1AM-represented employ- were also being affected. The "one stop

11011111011MMIN.MMINI Evaluation Forum issu 11 Summer 1995 63 Interviews

been involved with the Quality Through this philosophy. /n addition, the "one velop their own individualized career Training Program since its inception in stop center" concept is a key feature of education plan, utilize career assess- 1989, first as a Joint Policy Board previous QTTP job placement activi- ment tools, participate in on-site com- member and now as Co-Director. ties, arid provided the organizational puter-based training, or be referred to John Dryeris the Chairman of the framework for the Reemployment Pro- other training/educational opportunities Labor-Management Committee in- gram. which match their skills and pace of volved with the Boeing Reemployment learning. In addition, the program of- Program, and Co-Director of the IAM/ Our major issue at this time is the Boeing Quality Through Training Pro- impact of company downsizing on our fers on-site courses in math, reading, gram. A design engineer, Mr. Dryer was employees. Q1TP offers Career and writing, creative thinking, problem Director of Labor Relations at Boeing in Personal Development programs for solving, presentation skills, and study Kansas prior to coming to the QTTP both active and laid-off IAM-represent- skills. Tutors are available on-site. He has served as a consultant on ne- ed employees of The Boeing Company. Community and technical colleges pro- gotiation strategies for contract negotia- vide the professional advisors and adult tions in both the public and prkate sec- QTTP's support and leadership in re- gard to the Labor Management Com- basic educational instructors. English tor, and has lectured on labol ,elations as a Second Language (ESL), and a re- at the University of Puget Soun.i. mittee and the Boeing Reemployment Centers has helped a great many laid- cent addition, American Sign Language Mr. Dryer has been Intensely In- off employees "land on their feet" and, classes are available for IAM employ- volved in the evolution of the Quality as Secretary Reich put it, "prove that ees. ESL classes, offered in partnership Through Training Program, providing with organizations throughout the com- guidance to the program in addressing there can be new work and new life af- ter the pink slip." pany, are set up for both active and the personal, training and career needs laid-off employees in several locations of workers. His emphasis on Joint Career and Personal Development convenient to either their workplace or union/management sponsorship has re- programs include four full-scale advis- sulted In a high level of acceptance of home. ing and learning centers at this time: these programs by both Boeing and QTTP's other two initiatives pro- union management. the Advising and Learning Center at our main offices in Tukwila, Washing- viding assistance to IAM employees ENO. ton; the North Site Advising and Learn- impacted by technology change and job Question: As the initial effort to as- ing Center in Everett, Washington; the combinations address the training sist the major segment of the Boeing Wichita, Kansas, Advising and Learn- needs brought on by change in the workforce, both those needing to be as- ing Center; and a new QTTP/Boeing workplace. This inclndes IAM-repre- signed to new jobs within the company Advising and Learning Center in Port- sented employees whose jobs are af- and those ultimately dislocated is land, Oregon. A fifth is due to open fected by technology change (defined the model established by the Quality soon. Two of these locations also ad- in the collective bargaining agreement Through Training Program the core of minister the QTTP Education Assis- as technological improvement in the Boeing Reemployment Program? tance program which offers an an- "tools, methods, processes, equipment And what are the main elements of nual benefit to active or laid-off or materials") such as employees at the QTIT's program design? IAM-represented employees of the new Chemical Pm cess Line and the company. Active IAM employees can Calibration Lab. For the new Chemical Gary Jackson: The Quality Through Process Line, self-paced video and Training Program came into being to receive up to $2,000 per calendar year for education and training at state-rec- computer instruction developed by provide IAM-represented employees of QTTP in cooperation with Boeing The Boeing Company with learning op- ognized institutions, and laid-off em- ployees whose training needs do not fit Training Organization is located right portunities that can lead to career at the workplace and a peer mentor growth, provide job security, and re- the Boeing Reemployment Program guidelines for retraining can receive up training syt.tem is in place. Calibration train employees impacted by changes Laboratory employees can access a so- in the workplace due to technology to $2,500 per year for three years after layoff, beginning on the date of their phisticated interactive electronics change or job combinations. This con- course cooperatively developed by cept of cooperation between company layoff from the Company. These bene- fits are available to all IAM-represent- QTTP and Boeing Training that incor- and union in order to provide important porates the elements of an AA degree career and education opportunities is at ed hourly employees, regardless of their location. in electronics. Technology change re- the heart of all QTTP activities. Bring- training has been developed and deliv- ing the company, unions, educational QTTP's Career and Personal Devel- ered in a number of other locations in institutions, Private Industry Councils, opment initiatives include a broad spec- Washington, in Portland, Oregon, and the Employment Security Department, trum of career development and skills in Wichita, Kansas. and other federal and state agencies to- enhancement programs for employees. gether in a cooperative effort to benefit Through this program employees can Job combinations occur when the employees is a natural outgrowth of meet with professional advisors to de- Corporate Jobs Committee agrees to combine two or more bargaining unit

72 Evaluatlon Porum issue 11 Summer 1906 Interviews

diately set to work on opening two one- jobs. QTTP has a responsibility to iden- The Region X office of the U.S. De- partment of Labor. stop, multi-service Boeing Reemploy- tify what additional training needs may ment Centers, one in each of the major be required for the affected employees In early 1993, QTTP, leading the ef- in- counties where Boeing dislocated to do the new combined job. QTTP fort to obtain federal funding, contract- workers live and work, and a satellite vites employees as subject matter ex- ed with the Human Resources Develop- center in a third county. These centers perts and line managers in the areas ment Institute (HRDI) in Washington, involve the out-stationing of staff from where jobs were combined to partici- D.C., an arm of the AFL-CIO, to assist Boeing Outplacement, both unions, the pate in a training needs analysis. The with the program design and applica- Washington Employment Security De- requirements of the new job are scoped tion. HRDI worked with the Labor- partment, Private Industry Councils, out through brainstorming techniques, Management Conunittee as a facilitator and the State Board for Community then validated and verified by addition- to develop and submit a grant proposal and Technical Colleges. al groups of subject matter experts. to the U.S. Department of Labor for a Finally, the skills list is given to Boeing National Reserve Grant under Job Secretary of Labor Reich pro- Training for identification of the train- Training Partnership Act (JTPA) Title nounced the Boeing Reemployment Program a model program that he ing courses required for each job level. III, for services to Boeing workers The purpose of using this job task anal- meeting the eligibility criteria under the wanted to replicate elsewhere through- ysis is to gather data from the employ- Economic Dislocation and Worker Ad- out the United States. On June 20, 1994, Vice President Al Gore and Sec- ees who actually do the work. justment Assistance Act. retary Reich presented the "Hammer The Quality Through Training Pro- The initial request wa for $10.2 the Award" to the Boeing Re-employment gram has shown that the union and million for a new Boeing Reemploy- help to Centers for "Putting Customers First." company, working together, can ment Program for approximately 3,800 increase opportunities for Boeing dislocated Boeing workers. The request The success of the Boeing Reem- workers and provide Boeing with a was urgent, as existing budgets ac- ployment project was acknowledged in September 1994 when the U.S. Depart- highly skilled workforce for a changing quired through unprecedented leverag- work environment. ing of funds and in-kind contributions ment of Labor awarded an additional $5.2 million to assist laid-off Boeing How was the Boeing Re- across a number of private andpublic Question: workers in Washington State. This en- employment Program established and sources were insufficient to meet the organized and how is it being funded? needs of the large numbers of workers sured that the centers could continue to provide dislocated worker services to dislocated in the 1990s. The new pro- John Dryer: The new Labor-Manage- gram redefined the unemployment sys- all laid-off Boeing workers, including ment Committee brought together rep- Of this tem to a reemployment system and in- an additional 1,000 employees. resentatives of a wider array of constit- troduced a number of new concepts requested amount, the largest propor- uencies, and charged them with that met the current thinking of the De- tion was for longer-term retraining, responsibility for a new program spe- partment of Labor: a public/private along with money earmarked for sup- cifically targeted to dislocated Boeing partnership; one-stop shopping for ser- portive services such as child care and workers across all divisions of the com- transportation, additional case man- vices; a consolidation of customer-fo- pany. These representatives were to cused services; market-driven training, agement for work-based learning and speak for an impressive group of stake- job development staff, and a full-time and worker empowerment and account- holders: project director for the program. ability. The Boeing Company. In reviewing the request, the U.S. The additional funding received on April 17, 1995, adds another $4.45 mil- The International Association of Department of Labor awarded an im- mediate grant of $5 million and an op- lion just in time to meet the needs of Machinists and Aerospace Workers. employees who will be impacted by ap- tion for requesting additional funding The IAM/Boeing Quality Through should the award prove inadequate to proximately 6,000 more layoffs an- nounced by Boeing for 1995, and Training Program. the need. Experienced in operating JTPA Title III and other employment bringing the total grant to $ 14.65 mil- The Seattle Professional Engineer- lion. ing Employees Association. and training programs, the Washington State Employment Security Department Looking at all the resources support- The Washington State Employment was to receive the National Reserve ing the Boeing Reemployment Pro- Security Department. Grant on behalf of the Labor-Manage- gram, we see an inventive combination ment Committee, and was ass:gned the of public/private funds and programs to The State Board for Community and role of administrator and operator of service dislocated workers. In terms of Technical Colleges. the program. funding, the program is utilizing spe- Several Private Industry Councils: With the federal grant in hand, the cial federal funds, regular federal pro- Seattle/King County, Pierce Lounty, Labor-Management Committee imme- gram dollars, and funds associated with and Snohomish County.

11100MIlmommemsallmblallatt 65 Evaluation Foruma ssue 11 Summer 1096 73 =

Interviews

a new state legislative initiative that has workers. Accommodating the various ing and evaluation requirements. Al- expanded training slots in the state's cultures, policies, and practices of these though no net impact study of EDWAA community and technical college sys- groups has required a great deal of has been conducted at the national lev- tem. Regarding programs and related trust, commitment, communication, el, the Department of Labor did con- support, the Program is bringing to- and sometimes negotiation. tract with a competent research firm to gether federal, state, and sub-state pro- The partnership involved accommo- study EDWAA implementation. The grams; personnel in community and latter may serve as a useful model for technical colleges; facilities, equipment dation and a willingness to cooperate in areas where organizations might be ac- studying coordination and service de- and staff contributed by The Boeing livery in the Boeing program Company; the IAM/Boeing Quality customed to having full discretion and Through Training Program, IAM and authority. Many strong organizations The use of state funds for increased SPEEA volunteer representatives, and have been involved. In my opinion, the community and technical college op- other Boeing programs. partnership has been an outstanding portunities for dislocated workers, success. which involves a set-aside for Boeing To date, over 12,000 employees workers, will require evaluation activi- have utilized Reemployment Center The Boeing Reemployment Program represents a significant human resource ties consistent with the evaluation plan outplacement services. Approximately developed by the State's Workforce Ed- 5,580 Boeing employees have obtained and reemployment effort involving a large multinational corporation and ucation and Training Coordinating employment, and 3,365 employees are Board, which calls for a study of both enrolled in JTPA retraining programs. key labor unions and three levels of government. The program is also the implementation and the legislative Question: We have been hearing a unusual in its commitment to keep intent behind this funding, and a study great deal atrout European models of communication open with employees, of outcomes for participants, employers collaboration between government, elected officials, business and labor and the community college system. business, and labor, related to the cur- communities, and others invested in the The Labor-Management Committee rent U.S. efforts to increase cooperation program, concerning the program's has selected a set of outcome measures among these different constituencies. progress. and has established quantitative goals While this kind of partnership has been for monitoring the Reemployment Pro- more traditional in Western Europe, Question: Who has oversight respon- sibilities regarding the Boeing Reem- gram, such as "entered employment these three groups have often behaved rate," average wage at entered employ- in an adversarial manner in the United ployment Program, and how are its re- sults to be judged? ment," "cost per participant," and "cost States. How difficult has it been to de- per entered employment." In addition velop this kind of partnership for Boe- John Dryer: Because of the multiple the Committee is interested in request- ing workers? funding streams and programs for this ing private/publk 'funds to conduct a new effort and the fact that differ- Gary Jackson: Clearly it has beena study of the program's overall efficien- greater challenge to develop a govern- ing expectations and formal regulations cy and effectiveness. exist among them regarding the evalua- ment/business/labor partnership in the On a human level, the fact that so Reemployment Program than it was to tion of performance there is no one source or method of assessment for the many former Boeing employees are re- coordinate resources in the IAM/Boe- ceiving retraining and have found em- ing Quality Through Training Program. entire program. Nevertheless, oversight responsibilities are given strong atten- ployment through the program, at a rate The Reemployment Program brings to- of pay equivalent to what they earned gether many more diverse, but well-es- tion by the Labor-Management Com- mittee and the program's administrator. at Boeing, constitutes the highest mea- tablished organizational entities. Asso- sure of success. This program has dem- ciated clusters of stakeholders had to be The use of Title III funds and a Na- onstrated that a cooperative partnership brought together around reasonably tional Reserve Grant require compli- approach to dealing with dislocated common purposes and goals for our ance with federally-mandated monitor- workers can and does work.

'01~1111 74 Valuation ForumIs IMO 11 Summer 1095 EvaluationIssues andActivi ies

.1

At the InternationalLevel

Union and the ternational economy. They are issues sues in the European and Editorial Introduction former Soviet Union. These are issues about which American employment training professionals need toremain facing all industrialized nations, even This portion of Issue #11 emphasizes informed. economic reform and restructuring is- though to varying extent, in the new in-

: Economic Reform inthe European Union that the EU must first consolidate its To achieve the highest sustainable Editprial IntrOduction economic growth and employment, own internal development sothat en- largement does not have a destabiliz- and a rising standard of living in In the midst of negotiations to form a member countries, while maintaining ing effect on European unity. 12-member European Union (EU), the financial stability, and thus to con- definition of "Europe" has grown in- The issue of EU enlargement is only tribute to the development of the faced by creasingly elusive. The eligibility re- one of many difficult issues world economy. quirements for membership in the Euro- the leaders of this important experiment pean Community (EC)have reflected In transnational integration.However, To contribute to sound economic ex- that confusion. The MaastrichtTreaty of the increased consolidation ofEurope pansion in member as well as non- 1991 proposed that "any European has substantial economic implications member countries in the process of State whose systems of Governance for the rest of the world, since it will economic development. are founded on theprinciple of democ- vastly expand intra-European tradeand To contribute to the expansion of racy may apply to becomemembers of expand the EU's access to global mar- the Union." But the concept democracy kets. And the EU's growing regional co- world trade on a multilateral, non- discriminatory basis in accordance was not spelled out precisely.The or- operation agreements with surrounding ganization of free elections, the prac- non-EU nations will play a significant with international obligations. for tice of the rule of law, and respect role in creating a larger European eco- The original member countries ofthe civil and human rights wereassumed to nomic structure. Already European vot- OECD were the world's major industri- election of the Eu- underly this concept. But member ers participate in the alized nations eighteen nations in countries were also expected todevel- ropean Parliament. There isalready a Western Europe, and the U.S.and Can- op stable marketeconomies based on European Council of Ministers, a Euro- Tur- in- ada. Although less industrialized, private entrepreneurship. These twin pean Council, a EuropeanCommission, key was also a member. Later Japan, tentions complicated the issue ofeligi- and a European Court of Justice.The Finland, Australia, New Zealand and bility since countries varied consider- Single European Act of 1986 supported Mexico were added. The Commission ably in the extent to which they had the development of a common market of the European Communities now par- applied these abstract concepts. for goods, labor, capital and services. ticipates in the OECD's studies and Beyond that, the gradual enlarge- The Treaty on European Union in 1991 other projects, and OECD hasextend- ment of the EU, however inevitable, strengthened political and monetary ed its work to encompass economicis- has remained controversial politically ties. sues in Central and EasternEurope, and economically. A significant memo- Within this larger framework, the Or- and the republics of the former Soviet randum submitted to the European ganization for Economic Cooperation Union. Council in the early 1990s by the three and Development (OECD) has played a In this series of xticles we look at Benelux countries stressed that mem- critical advisory role In providing cross- labor market issues in the European bership for the smaller countries could national data and policy analyses on a Union, and again focus the microscope Eu- by the most change the nature and form of the range of issues confronted on the new employmentand training ropean Community, mightdistort the advanced Industrial nations, including initiative in Great Britain. In the pro- In basic objectives of the founder nations, the EU. The OEOD was established cess, we rely on a numberof analyses and would likely erode the positionand Paris in 1961 tt promote the following such an those carried out by the OECD. role of the smaller members. Franca Impressive goals: has taken the position, for example,

67 Evaluation Forum lam 11 Summar 1998 Evaluation Issues and ActIvitlesv ...EconomicReform In the Euroan Union Bl The OECD's Only a portion of this unemployment Kingdom. In most other OECD nations, Employment Study was cyclical.Much of it was attributed unemployment rates were higher for to structuralfactors. Most of thisstruc- men. Regional differencesoccurred In 1992 the ministers of the tural unemployment was considered by more in Europe than in the U.S. Age coun- the OECD analysts to be the result of tries with membership in the Organiza- was another factor. dissonance between pressureson eco- tion for Economic Cooperation and De- nomic systems to adapt to global There was a striking differencein velopment requested a literature search eco- nomic change, and their willingness long-tenn unemployment rates. Inthe and major study addressing key issues and ability to do so. European Community, 40% of theun- in several broad areas: macroeconomic employed were unemployed long-term policy, trade, technology, wage forma- These analysts made the assumption compared with 15% in EFTA and 11% tion, labor adjustment, education and that the OECD economieswerenot in North America (U.S. and Canada). flexible enough training, and benefit systems and taxa- in coping with new In several EC countriesmore than two tion. OECD's response to this mandate technologies and trade challenges, and out of every threeunemployed older is reported in two volumes: 1)The therefore were relying too heavilyon workerswere unemployed long-term. OECD Jobs Study: Facts, Analysis, methods that were protectionist, that In general, the OECD statisticssuggest and Strategies,and a background vol- sought to restrict competition. And they that EC countries havea greater rehir- ume believed that these nations 2) The OECD Jobs Study: Evi- had placed ing/reemployment problem than other dence and Explanations.In this arm-1p too little emphasis on developingnew member nations, given the correlation we review the first of these companion products and new processes, new kinds between the low rates of flow backinto reports. of workforces and workplaces, andnew the labor force and the high incidence ways of creating jobs. The emphasis in this international of long-term unemployment.In the U.S. and Canada, workers have study is onunemployment,the most The Nature of Unemployment a great- troublesome aspect of capitalist econo- er risk of becoming unemployed buta mies. This focus is consistent with the The number of unemployed in better chance of reemployment. OECD's mission to contribute expert OECD countries tripled withina single opinion and the results of research to decade, between 1972 and 1982.An Changes in the Size and the efforts of industrialized nationsto easing of the prolonged period ofeco- Composition of the Labor Force nomic decline of the late '70s and 1) achieve prosperity and 2) maintain early The working-age population inall '80s was reduced only slightly by sub- their ability to be innovative in coping OECD countries is growingmore slow- with an increasingly integrated and sequent economic progress, and then ly than previously, thereport points competitive global economy. rose sharply again after 1990, accord- out. North America and Oceania have ing to the report. However, the upward experienced a fairly rapid decline, but The major recommendation in the trend in unemploymentwas more se- OECD report is that nations mustem- the EC, EFTA and Japan haveexperi- vere in the European Community (EC) enced a greater decline overall. brace change,rather than slow its Mo- and in Oceania (Australia and New Howev- mentum or offer too much protection to er, the sizA of the working-age popula- Zealand) 10-11% in 1994, as com- activities that have not adapted tion participating in the workforce success- pared with approximately 7% in the fully to the significant changesoccur- grew significantly to 75% in North U.S. at that time. Unemploymentstabi- America, Oceania and EFTA, remain- ring in the world. Strategies thatpro- lized at 2-4% in the European Free mote adjustments fo changeand ing largely unchangedover the past Trade Association (EFTA) countries thirty years in the EC countries enhance social cohesion whilesuch ad- Austria, Finland, Iceland, Norway, (67%). justments are being made, are viewed Sweden, Switzerland and Leichtenstein Higher enrollments and retention as the route postindustrial societies until 1990 when it rose rapidlyto al- rates in educationspelled a decline in must take if they are to provide the lev- most 8% in 1994. Japan's 1994 rate the labor participation ofyoung people, el of life quality they have traditionally was 3%, in the context of 1-3% while early-retirement plansand dis- valued for their citizens. throughout the post WWII period. ability pensions reduced thenumber of older workers. The report Facts and Assumptions The report indicates that concludes young peo- that these developments hadthe effect pleandwomen According to the report, 35 million were most affected by of infusing some labor forceswith unemployment. Austria and Germany, more highly qualified labor market people were unemployed in OECD with strong apprenticeship systems, en- countries in 1994 trants. 8.5%of the OECD were exceptions to the high youth un- labor force.Another 15 million were employment rates elsewhere. In Italy, thought to have given up looking for Job Growth Spain and Finland more than 30% of work, or had needed to accept a part- young people were unemployed. Wom- Growth in employment has been time job. A third of young workersin en had higher unemployment rates than strongest in North America and Ocean- some of these countries had no job. men in the EC, except for the United ia, the report says, and weakestin the EC and EFTA, with Japan inbetween,

76 Evaluation Forum Issue 11 s Summar 191i5 _ gAMEr.r--

Evaluation Issues and Activities Economic Reform In the European Union

The EC's slow job growth rate is felt to growth in private sector employment and their capacity and willingness to be related to its slower population has increased. adjust. This gap was created, they in- growth and the lack of increase in the sist, by rigid even if well-meaning eco- proportion of its population in the Wages in Employment nomic, labor market and social policies workforce. Job growth has occurred The OECD report reminds us that that had the unintended effect of de- predominantly in the service sector in "the process of wage determination is creasing the ability of economies to all OECD countries. Agicultural em- strongly influenced by labour market adapt to change. ployment declined from 14% in 1970 pressures, social perceptions, legisla- They ask us to review history begin- to 77% in 1994. Industrial employment tion, and industrial relations systems, ning in the mid-1960s. Essentially, ev- decreased from 40% in 1970 to under which all affect the evolution of real erything was coming up roses for all 30% in 1992. The share of OECD em- wages and wage differentials." Wage OECD countries in terms of stability, ployment in the service sector rose shares in national income, the report growth and trade. But by the middle of from less than 50% in 1970 to approxi- says, decreased to below 1970 levels the 1960s economic performance had mately 65% in 1992. Most new jobs in following the 1979 oil-price crisis. At begun to decline and inflation to rise. the 1980s were in financial services, in- the same time, wage differentials be- This produced an uneasy economic en- surance and business services, and tween low vs. high-skilled workers wid- vironment. Exchange rates became community and personal services. ened in the English-speaking countries, less predictable with the breakdown of The growth in international trade while EC differentials remained rela- the system of fixed rates. In the 1980s and technology are likely responsible, tively unchanged. In fact, low pay was the deregulation movement regarding according to the report, trade having more prevalent in the U.S. than in other fmancial and product markets enhanced grown faster than the Gross National OECD countries. In the '90s one-fourth efficiency while accelerating change. Product. Most member countries' trade of all full-time workers in the U.S. These and related trends made it more has been with other OECD nations. earned less than two-thirds of median difficult for economies to adapt. Their exports to other countries have earnings, compared with one-tenth in The challenge of adjustment was Australia and one-fifth in the EC. In tended to have a relatively high skill made stronger by continuing rapid content in contrast to their imports, 1990 almost one-fifth of all full-time technological change, particularly in which are most frequently labor-inten- U.S. workers had earnings at or below information technology, and by the in- sive raw material and manufacturing the official poverty level. This is in the ternationalization of' economic activity. products. context of a general movement of job Just at this juncture, when more flexi- growth from unskilled to more highly ble economies were sorely needed, pro- The largest OECD countries, with skilled jobs. large domestic markets, have spurred Waive social policies extending social the greatest amount of high-technology Implications for Employment benefits were introduced. The assump- manufacturing, but high-tech produc- Policy tion is that these protective policies tion has become a viable option for the made these economies more rigid. The analysts preparing the report smaller nations as well. For example, Within this general history, the U.S. Ireland, Australia, Finland and Norway on the Jobs Study view the unemploy- ment scenario with concern. Further- experience was somewhat novel. Pro- have been quite successful in the high- tective labor market and social policies tech area, in addition to Japan. more they feel that the debate over its causes has been seriously flawed. A were less extensive, labor markets more In thirteen OECD nations, non-farm number of myths have been reinforced flexible, and job growth was occurring mainly in the private sector. Many self-employment has grown faster than such as that technology increases overall job growth, particularly in the unemployment, that imports from low- jobs were high skill and high wage. UK. Part-time employment has in- wage countries depress wages and pro- Many were low skill and low pay. creased as a share of total employment duce unemployment, and that the inten- Workers without skills had no option in all OECD countries, with women ac- sity of global competition is to blame. but to accept the latter because of the counting for most of the increase. Tem- The evidence developed over the past low level of social support. On the oth- er hand, these workers might have been porary employment has not appeared four decades would suggest, they say, to increase except in France and Spain that none of these factors is the primary worse off with a more rigid economy that pre ,luded any job at.all. where there has been a deregulation of cause of rising unemployment. The work contracts. Two-thirds of the job problem is that each causal theory Both the widespread unemployment growth in the EC and EFTA since the leads to a different policy conclusion, in other OECD countries, and the di- 1970s has taken place in the public sec- whether the theory is warranted or not. chotomy in the kinds of jobs available tor in contrast to the U.S. and Japan Again, these analysts look for the in the U.S were thought to stem from whose increase in jobs has occurred the failure to adapt to the trends emerg- mainly in the private sector. However, main cause in the gap between the need for these countries to adjust to global ing over previous decades. Managerial public sector employment growth skills, the organization of workplaces, slowed in Europe in the 1980s, and the changes affecting economic systems,

Evaluation Forum.Issue 11 Summer 1906 Evaluation Issues and Activities . Economic Reform In the European Union

general productivity and workforce Macroeconomic Policies The proof of the effectiveness of preparation had not kept up with a The report proposes that the macro- these reforms for workers, the analysts more technologically advanced econo- say, is in the creation of more new jobs. my. While a lack of adjustment meant economic environment influences an economy's ability to create viable jobs more low paying jobs 'at the U.S., it in response to change. Certain ingredi- Where Job Creation Is Most meant greater unemployment in the EC. ents should be part of macroeconomic Likely to Occur The development of a low-paying job The following predictions and rec- sector in EC countries had been pre- policy, the analysts suggest, if that en- vironment is to have the right effect: vented by practices such as state-im- ommendations are made in the report: posed or union-negotiated wage/income Sufficiently high levels of national New jobs must be generated by the floors and employment protections. saving and investment: Since the private sector, given government A mid-1970s, national savings and in- budget deficits and resistance to tax Preserving Social Objectives vestment as a share of national in- increases in member nations. Although the main policy message in come have declined significantly in Many new jobs will have increasing- the report is that the solution to high OECD countries, particularly in Eu- ly high knowledge requirements, unemployment is the restoration of rope. economies' and societies' capacity to particularly those involving tradable adapt to change, the report's sub-theme Price stability and low rates of in- goods, related to the need to im- is that this restoration must not come at flation: Employers need to receive prove productivity in innovative the expense of an abandonment of so- "undistorted price signals from the businesses that have the ability to cial goals. To satisfy such restoration markets" if there are to be effective use technology effectively. while preserving protectionist values structural adjustments. has been a formidable task. Many new jobs will likely be low- Budgetary efficiency and high-quali- productivity, low-wage jobs, which The analysts propose that the key is ty public sector spending and taxa- may help absorb the significant to reform social policies such that they tion: Boosting economic growth at numbers of low-skilled unemployed. no longer have negative economic side- the cost of higher inflation should be The report cautions, however, that effects. They do not see such reform impossible to achieve, if policy experts avoided, the analysts point out, since tax concessions and/or employment will 1) study the spectrum of social pol- the subsequent need to tighten could subsidies supporting the creation of icies implemented over the past three result in lower average growth over low-wage jobs should be avoided, decades, 2) assess to what extent they a number of years. - while policies that inhibit low-wage may have crippled economies' adjust- The report recommends an integra- job creation should also be resisted ment capabilities, and 3) analyze what tion of structural and macroeconomic due to the high unemployment rates. strategies may be feasible for removing policies, since structural reforms, it Clearly these predictions and sug- such disincentives without dismantling states, will not work well in an unstable desirable social protections. The re- gestions involve a fine balancing act macroeconomic environment. On the between two important concerns: 1) the search and analysis already completed other hand, macroeconomic policy is general undesirability of deliberately should, they say, provide a useful more likely to be effective if structural framework for such reform. This frame- creating low-wage jobs, and 2) the de- policies prevent inflexibility and allow sirability of creating jobs for unem- work is summarized in the second vol- resources to be used most profitably. ume of the Jobs Study. ployed workers lacking the skills re- Structural change in good economic quired by high-productivity, high-wage times is less painful, but the need for Some Conclusions firms. such change is not as evident and is The OECD report focuses on cycli- therefore more difficult to rationalize. Technological Change cal as well as structural unemployment But when economic demand is weak, in drawing conclusions about how structural reforms can involve worker The report claims that technology, historically, has generated more worker member nations should approach their dislocation with little job creation else- major task. Therefore their ultimate rec- where to absorb those workers, and this earnings than displaced workers. Ja- ommendations involve the macroeco- effect can reduce political support for pan's shift to high-tech industries, for example, led to a 4% increase in manu- nomic environment as well as job cre- change. The report therefore recom- ation and workforce preparation that mends the simultaneous implementa- facturing employment in the 1970s and is, they consider that both macroeco- tion of structural and macroeconomic 1980s. The EC experienced a 20% de- crease in manufacturing employment, nomic and structural policies are need- reforms. ed.

70 Evaluation Forum Issue 11 6 Summer 1995 -

Evaluation Issues and Activities Economic Reform in the Euroan Union

ment (example: unemployment insur- associated with an emphasis on lower- way that favors the hiring oflow-wage, ance), but they have not shifted much wage, lower-technology industries. unskilled labor, such as eliminating ceilings on employer contribution rates of their resources to active strategies Critical to the positive relationship and reducing low-skilled workers' in- (example: public employment servic- between technological change and job come taxes. Widening wagedifferen- es). The report supports a reconsidera- creation is effective absorption, appli- tials is another option, since there is ev- tion of resource use. cation and diffusion of new technolo- idence this speeds employment growth. However, on the basis of research gies, such as current strides in informa- However, the analysts recognize that evidence, the report is critical of train- tion technology. Government subsidies none of these strategies are simple to ing programs intended to serve large, to firms tend to discourage this kindof implement. diverse groups of unemployed innovation. Pooling resources to pro- programs that are not carefully target- mote basic research and the application For instance, the down side of wid- ed. It also questions job creation strate- of new technologies, particularly those ening wage differentials is that if these gies that involve subsidized hires. In- with the higher risks, is recommended become too great, many with low in- stead it recommends the involvement as an alternative strategy. comes will fall into poverty. In Europe and Scandinavia, widening differentials of all stakeholders in developing more are generally unacceptable intermsof effective methods for reducing unem- Small Business employers, unions, educa- Entrepreneurship the commitment to equity. Neverthe- ployment less, the report stresses the need to re- tional institutions, and local govern- The report encourages support for visit the issue of safety net protections, ments. "dynamic entrepreneurship" in the giving more emphasis to "the market- form of removing regulatory barriers to In terms of education and training, busi- clearing role of wages," and to consider the report again looks to the research new and expanding private sector new ways to express equity values. nesses. Currently government resourc- evidence: es, such as protective subsidies, are tar- Many European countries have Preschool and early childhood de- geted to large firms. Small business strong employment protection legisla- velopment programs are important development requires support for the tion, which can reduce hiring and lead strategies for providing a foundation following, according to the OECD ana- to the use of contracts. conducive to the pursuit of life-long Long-term work contracts encourage lysts: workforce preparation; adequate learning opportunities. physical infrastructure; modern tele- investment in training and skills up- communications; networks of small grading, which then increase hiring. Curricula and teaching methods firms; and improved access to and use But finding the right balance between need to be individualized to meet the of sources of managerial advice, tech- work r protections and hiring has prov- learning needs of underachievers nological expertise and research. Gov- en to be very difficult. and potential school dropouts, if ernments should assist small businesses The analysts suggest the following they are to acquire workforce skills. in their development, they suggest, as important ideas forconsideration in A better balance is needed between through seed, venture and equity capi- increasing hiring: tal, and the fmancing of debt. post-secondary education, and tech- The development of rWes to prevent nical and advanced occupational The Role of the Social major firms from devising barriers training. Safety Net to entry. New forms of apprenticeship and The OECD analysts reassert \hat The establishment of rules that pro- work-based learning options are some of the policies intended to pre- hibit cartels, while allowing desir- needed as jobs demand multi-skilled serve equity have had negativeside-ef- able forms of cooperation among fects for economies. An example is dis- workers with general competencies. firms. incentives to hiring. The non-wa,e On-the-job, employer-sponsored labor costs borne by employers, such as International cooperation in moni- training should be increased through social security contributions and taxes, toring the behavior of firms in- innovations that will encourage em- represent a large proportion of the cost volved in international competition. ployer investment in training. of labor, particularly in Europe. Gov- ernments are becoming serious now Strategies Benefiting Workers More effective strategies need to be about reducing social spending as a and Their Families developed to prevent families from way to reduce labor costs and encour- falling into poverty due to the ten- The report claims that the typical age hiring. OECD country spends 2-3% of its GNP dency for high unemployment bene- Shifiing the tax base from to on labor market strategies such as pas- fits to decrease workers' incentives other taxes is one alternative. The re- sive methods for reducing unemploy- for increasing their hours of work or port suggests cutting social contribu- their investment in training. tions by employers and employees in a

71 Evaluation Forwri haw 11 Sum,' 1926 79 _

Evaluation Issues and Activities Economic Reform in the European Union

IIII The OECD analysts propose thatan ciety that have never before faceda integration of tax and benefit systems high risk of unemployment, suchas The Persistence of may work best, achieved through a white collar workers, are losing jobs, Unemployment in negative income tax or an income tax with all the personal and societal credit strategy. However they acknowl- costs that implies in terms of lost po- the Postindustrial edge that while this approachmay re- tential and lost investment. duce "the poverty trap," it may not Era raise worker incentives. The broad goals of the recommenda- tions flowing from the Jobs Studyare The Economics Department of the Overall Policy Recommendations these: to enhance the ability ofecono- Organization for Economic Coopera-. mies to adjust and adapt, and to in- tion and Development (OECD) in Paris The report explains the need and ba- crease their capacity to innovate. Their publishes a semi-annual journal, OECD sis for the recommendations advanced. major policy suggestions, excerpted Economic Studies. In this articlewe re- from the OECD report are presented in view two articles in the Winter 1993 is- More than ever since World War II, Figure 1. sue. The first is by Jorgen Elmeskov today's unemployment is causing and Maitland MacFarlan, "Unemploy- damage in ways that cannot be mea- ment Persistence." The second is "In- sured by the sheer numbers. High un- ifi;it'Noto, terpreting Unemployment: The Role of employment creates insecurity and Labour-Force Participation" by Elm- resistance to organisational and tech- For readers interested in keeping in- eskov and Karl Pichelman. nical change. Long-tenn unemploy- formed about employment and related ment lowers self-esteem, is demoti- issues in the OECD countries, theex- Elmeskov and MacFarlan vating and self-reinforcing, and is tensive OECD Catalogue of Publica- associated with health problems. The tions can be obtained at no cost from The most worriesome characteristic rise in means the OECD Publications and Information of unemployment in most OECDcoun- that many young people are losing Center at 2001 L St. N.W., Suite 700, tries in the 1990s, the authorspropose, Washington, D.C. skills or employability. Groups inso- is its tendency to continue closeto the level experienced during theworst eco- nomic times even in good economic times. This has been the outstanding .8- at- . klajoipoliCji.$66tjestiOnS..in ihe ciEeb Report tribute of the four unemploymentcy- cles since 1966. The European Union Set macroeconomic policy such that it willboth encourage growth... and make it sustainable. (EU) rate, for example,rose from 6°A, to 10% during the economic declines of Enhance the creation and diffusion oftechnological know-how by improving the early 1980s, and did notgo below frameworks for its development. 8% in 1990. The authors seek explana- Increase flexibility of working-time. .. voluntarily sought by workers and em- tions for this trend. ployers. Studying the research evidence,two Nurture an entrepreneurial climate byeliminating impediments to.. the cre- alternative explanationsemerge. Persis- ation and expansion of enterprises. tent unemployment appears to be relat- Make wage and labor costs more flexible byremoving restrictions that prevent ed to 1) changes in the deeperstructur- wages from reflecting local conditions and individualskill levels.... al influences that determine thedemand a for and the supply of labor changes Reform employment security provisions thatinhibit the expansion of employ- ment in the private sector. that have increased cyclical unemploy- ment rates, and 2) the tendency for high Strengthen the emphasis on active labor market policies and reinforce their cyclical unemployment to createstruc- effectiveness. tural unemployment. Improve labor force skills and competencesthrough wide-ranging changes In education and training systems. The authors' thesis is that the tradi- tional distinction between cyclical a and Reform unemployment and related benefitsystems ... such that societies' structural unemployment is dimmingin fundamental equity goals are achieved inways that impinge far less on the ef- the context of two trends: the tendency ficient functioning of labor markets, for unemployment relatedto cyclical unemployment forces to develop into longer-term structural unemployment, and the tendency for the economy's structural characteristics to influence the cyclical aspects of economicactivi- ty, Indicators used to describeunem-

72 80 Evaluation Ponim Issue 11 Summer 1906 rn_

Evaluation Issues and Activities Economic Reform in the European Union

ployment rate trends suggest that the FigUre.1 Theorieand VolicYAiternatiVes unemployment rate required to stabilize wage inflation has risen, that vacant Theories Explaining Policy Alternatives Persistent Unemploy- jobs are filled with greater difficulty MIcroeconomic Options Macroeconomic Options than before, and that the use of physical ment production capacity is reaching a Reform directed to threshold leading to a mismatch be- Rising natural rate of Policies seeking to ex- unemployment reversing or offset- pand the economy. tween jobs and available sources of la- ting the factors caus- bor. Overall, OECD structural unem- Slow adjustment to ing the rise in and Activist policies. ployment rose steadily throughout the economic change persistence of unem- 1970s and early 1980s, declining some- ployment. what in the early 1990s. The authors Reduction of employ- claim, however, that this general pat- ment protections. tern veils important differences across OECD countries and regions. Reform directed to improving the region- To study basic causes and the rea- al and occupational sons for differences across OECD mobility of the unem- countries, the authors utilize a particu- ployed. lar analytical framework to explore the phenomenon of rising unemployment. The framework gives attention to three On the other hand,reducing employ- employment and training professionals. key relationships: the demand for la- ment protectionsmay be an efficient It focuses onthe relationship between bor; the nature of wage-setting; and the option for speeding adjustment to labor force participation and reported labor supply. change but may do little to reduce cy- unemployment.The point the authors clical unemployment.Expansionary wish to impress en readers is that the These relationships are considered macroeconomic policiesmay have no measurement of unemployment lacks a to affect the importantinterrelationship influence on persistent unemployment clear-cut distinction betweenunem- between cyclical and structural unem- due to cyclical factors.Slow adjust- ploymentandnonparticzpation in the ployment.Applying a statistical model, mentmay have only a minimal effect labor force.This, they say, continually the authors empirically test some of the on the natural rate of rnemployment. leads to policy confusion. major theories explaining persistent un- Activist policiesthat attempt to fine- employment in the OECD countries. At The relationship in question is likely tune the economy may be counterpro- to change, the authors suggest, on the either end of the continuum of potential ductive if they lead to a neglect of causes are 1) a rise in natural (cyclical) basis of a number of influences: the de- structuralforces. Automatic stabilizers unemployment, and 2) slow adjustment mographic composition of the work- would seem to be important in prevent- force, the extent of coverage and the to economic change (a structural ing sharp economic ups and downs, but cause). The analysts come to a rather generousness of income support sys- these may also cause a rise in cyclical tems, and other incentives affecting the nebulous conclusion that neither theory unemployment which then destabilizes is fully correct! This conclusion actual- decision to join or leave the labor force. the economy. ly has important implications for eco- Low unemployment could mean that an nomic policy, such as those suggested The message is that there are consid- excess supply of labor is being moved in Figure 1. erable policy risks in misdiagnosing the toward nonparticipation, which masks causes of persistent high unemploy- the true extent of unemployment. The authors make it clear that the ment, and in relying only on one strate- problem of persistent unemployment is gy. The authors suspect that placing Trends in Labor Force quite complex, since there are likely major policy emphasis onstructural re- Participation multiple causes. Long periods of cycli- formsmay be the most effective deci- cal unemployment can dissove into Participation rateshave continually sion particularly reforms thatim- structural unemployment. At the same risen since 1970 in all OECD countries prove the skills of the unemployed and due almost entirely to the increase in time, a common set of policies may be offer them incentives to re-enter the la- effective, they say, in reducing both cy- femaleparticipation. The authors credit bor force. clical and structural problems. For ex- the gradual shift incultural and social norms and attitudeswith this char ge, ample, structural alternatives may be Emeskov and Pichelman effective in reducing both the natural but the change has also been supported rate of unemployment and slow adjust- This article on the role of labor force by the increasing availability of public- ment to change such as policies sup- participation in persistent unemploy- ly-financed day care institutions in portingworker mobility. ment is more understandable by non- many OECD countries. The authors economists, and likely more useful to feel the reform of tax systems,partieu-

woo.1.0.11 1905 73 Evaluation Forum N IOUS 11 Slimmer 81 , -

Evaluation issues and Activities Economic Reform in the Euro an Union

larly reforms that made the individual ness cycles, the authors fmd that unem- siveness, or the extent to which the rather than the family the basic tax unit, ployment fluctuations i.e. variability effect of fluctuations in cyclical em- may have been another factor. Howev- in unemployment are greater in ployment on unemployment are er, long-term trends in participation North America, the UK, Spain and Ire- softened by what the authors define rates have varied across geographical land than in Japan, Austria, Switzerland areas. For example, male participation and some of the Scandinavian coun- as "procyclical variations" in the la- rates have declined over the past twenty tries. The authors attribute differences bor force. years in almost all European countries. in unemployment variability to interac- Conclusions from the analysis: Elas- In North America and Japan they have tion between 1) employment and 2) un- ticity varies widely across countries. been stable. employment and nonparticipation. France has almost no short-run labor An analysis of the OECD data used Acknowledging data inadequacies, force response to cyclical swings in in studying the relationship between they take a look at cyclical fluctuations employment, whereas in Japan these participation and unemployment sug- in output ("volatility"), which they pro- ups and downs have a significant ef- gests that the "hidden component" of pose may lead in different ways to dif- fect. unemployment may increase as mea- ferent degrees of variation in employ- sured ("open") unemployment increas- ment across countries, and may 2. The "elasticity" o; labor force partic- es. There is little evidence in the data ultimately generate differing unem- ipation rates. A that countries with low levels or in- ployment conditions. They find that Conclusio is from the analysis: In all creases in measured unemployment ex- countries respond differentially to "de- OECD c Juntries, other than Germa- hibit these conditions at the expense of mand shocks, the degree of exposure to ny and France, total participation lower levels of participation, and dif- autonomous supply shocks, the sectoral ferences in measured and hidden unem- rates are significantly affected by composition of output, and the pres- employment rates. And the elasticity ployment across OECD countries may, ence of automatic stabilizers." the authors surmise, be larger than the of participation rates with respect to data reveal. A traditional explanation for the rel- employment opportunities is higher ative stability ofemployment,the au- for females than for males, and The authors view data on "discour- thors ccnclude, is "labour hoarding," or aged workers" as an important resource higher for youth and older workers. the tendency in economic downturns This is consistent with the fact that in understanding the dynamics. This for firms to reduce hours of work be- group represents a subset of a larger fore laying off workers. Another vari- the larger proportion of workforces group of people who have stopped able is employment protection legisla- in OECD countries are women, and searching actively for employment be- tion, and incentives unbedded in they are overrepresented in the dis- cause they have concluded that no via- unemployment benefit systems. These located worker population. ble jobs exist. Although "discouraged influence economic adjustment costs worker" definitions vary across mem- and as a result affect employment flexi- Unemployment Utilized as a ber countries, making an analysis of the bility. Variations in self-employment Socia-Economic Measure data difficult, over time these data sup- are also an influence. The authors point The forces affecting and explaining port the authors' conclusion from their out that as the volatility of self-employ- cyclical fluctuations in unemployment own analyses, namely that official un- ment drops below that of wage and sal- offer important insights to those inter- employment and ary employment, a generally high pro- ested in better social and economic in- rates are positively correlated with portion of self-employment (within dicators. The authors define three ma- the exception of Japan. The long-run overall employment) tends to reduce jor roles for unemployment in this relationship between unemployment the latter's cyclical variability. This is respect: and labor force participation seems to an important finding since self-employ- be negative. Again, however, there are ment is growing faster in OECD coun- Unemployment as a measure of un- different tradeoffs between these phe- tries than overall non-agricultural em- der-utilization of labor. nomena for different countries. Coun- ployment. Unemployment as an indicator of tries with low unemployment, such as Japan, tend to have high participation Labor Force Sensitivity social hardship. rates. Countries with high unemploy- The analysts also look at the sensi- Unemployment as a determinant of ment, such as Spain, tend to have low tivity of the labor force to cyclical la- wage pressure. participation rates. bor market conditions. In doing so they A number of implications identified Fluctuations In Unemployment study two phenomena: by the authors are noteworthy. If ex- panding output produces a virile em- Using changes in the average annual 1. The "elasticity" of labor force "trend ployment response at the same time the unemployment rate as a measure of deviations" in amployment that is, a measure of labor force respon- labor force stays the same, unemploy- fluctations in unemployment over busi- ment tends to fall strongly with rising

74 82 Evaluation Forum Issue 11 &Immo,' 1995 Evaluation Issues and Activities Economic Reform in jEuoanUnion

input but potential ougmt increases Overall Conclusions does ,Ne; as a result of increasing actual ouut, The authors comment on the evi- Emeskovis in the Economic Pros- if rising productivity leads to rising em- dence produced in analyng data and peels Division of the OECD Economics ployment which in turn increases the on the unsolved mysteries that remain Department. Plcholman is in the insti- labor force. This is why countries with (see Figure 2). Two explanations for tute for Advanced Studios In Vienna, a responsive labor force and productiv- the mysteries are possible, according to Austria. For other publications of inter- ity cat. expand longer when coming out the OECD analysts: est, please see: of an economic down cycle prior to in- Barro, R. "The Persistence of Unem- flationary pressures building sufficient- 1. The relationship described above, ployment,'In American Economic ly to require corrective measures. In between labor force responsiveness Review. Papers and proceedings, this sense, "cyclical fluctuations in pro- and unemployment may occur if a 1988. ductivity and cyclical labour-force vari- country has experienced predomi- Bertole, t."Job Semnity, Employment ations and not measured unemploy- nantly negative shocks over recent and Wages," InEuropean Economic ment -- are relevant in gauging the full decades. Review, 34, 1990. extent of under-utilisation of labour nnd the associated output loss during a 2. The persistence of unemployment Clark, K.and L. Summers. 'Labor Mar- may be greater following an upward ket Dynamics and Unemployment: A slowdown." Fieconsideration," in Brookings Pa- Using unemployment as a measure shock to the unemployment rate pers on Economic Activity, 1, 1979. than following a downward shock. of social hardship may fall victim to the Elrneskov, J. "High and Persistent Un- same malady. The authors claim that However, the degree oflabor force employment: Assessment of the changes in measured unemployment responsiveness does not appear related Problem and Its Causes," in OECD underrepresent changes in gainful em- to trends in or levels of labo:' force par- Department of Economics Working ployment and earned income. Further- ticipation. Participation rates tend to Papers, No. 132, 1993. more, these changes do not reveal the return to their underlying trends while Lindbeck, A. "Macroeconomic Theory amount of economic hardship caused unemployment rates do not. It is no and the Labour Market," in European by cyclical downturns. Unemployment surprise that the analysts conclude that Economic Fteview, 36, 1992. benefits clearly reduce such hardship, the complex interactions between un- Pederson, P.J. and N. Westergard-Nell- therefore the hardship is greater for la- employment trends and economic cy- son. "Unemployment: A Review of bor force non-participants since they cles will require further research! the Evidence fromPanel Data," in are not officially employed. In coun- OECD Economics Studies,No. 20, tries where unemployment benefits are 1993. reduced substantially as a spell of un- employment elongates, the labor force tends to respond more to changes in employment. Evidence Remaining Mysteries As an indicator of wage pressure, it Countries with volatile unemployment is the authors' interpretation of their Cross-country differences are typical for trends and cycles In unemploy- rates tend to have larger increases in data analysis that unemployment rates ment and labor force participation unemployment over long periods are more useful in explaining wage in- across OECD countries, but these dif- than countries with unemployment creases than labor force participation ferences cannot be attributed to dif- that is more cyclically stable. Since rates. This may be due to the fact that ferent participation rates. Unemploy- labor force responsiveness is a major different groups ef workers react dif- ment rates are only a partial reflection influence on unemployment volatility, ferently to ehanges in employment. of cross-country differences in trends there is a relationship between re- Workers in their prime ages are as- and levels of employment. sponsiveness and the increase in sumed to influence wage determination and level of unemployment. This does not seem consistent with the more than others. Also, workers with fact that unemployment tends to re- longer spells of unemployment influ- turn quickly to a stable natural late of ences wages less than others. unemployment following an econom- Infm. 1.1..ic sno

Wm* attar mei IlleMaledirnmillalliliiiIMWM111001.1110111011011111Bir Ill Evatuallon Forum Wee 11d St.mMaw1P98 sq Evaluation Issues and Activities Economic Reform in the European Unlon

11111 Labor Issues tom in national labor markets," or much progress in integrating markets will competition among these na- for services, capital or labor. In the in European tional systems simply increase and 1970s, the nations of Europe were pre- Consolidation become divisive? occupied with domestic problems and integration efforts slowed. High inter- This short article briefly summarizes 2. Changes in the location and com- nal unemployment and low economic a 1993 book published by The Brook- position of production, and their productivity consumed their intellectu- ings Institution, Labor and An Integrat- effect on employment. al and political energies. But the reality ed Europe. Edited by three Brookings Questions: Will the integration pro- of rising unemployment and continuing economists, Lloyd Ulman, Barry cess alter the traditional segmenta- lackluster productivity renewed Euro- Eichengreen, and William T. Dickens, tion of European internal labor mar- pean interest in the reorganization of the book provides an important frame- kets by reorganizing labor market resources and labor forces, under the work for understanding the evolution sectors and regions across the EU, assumption that excessive intra-nation of thinking about a single market for the purpose being to permit firms regulation was largely to blame. the European Union, the changes pro- anywhere within the EU that spe- The editors report that the 1987 Sin- posed to reach such a goal, and the cialize in particular products and gle European Act was expected to es- probable effects of this consolidation services to locate together to utilize tablish a common market for products, for Europe and other countries. We capital and labor by 1992. Physical and confine our attention to the first chap- concentrations of resources and pur- sue economies of scale and scope technical restrictions on international ter, by the editors, on some of the im- economic activities were finally to end. plications of consolidation for the la- or will there be resistance to the re- The Act required European Community bor force and labor market. location required in developing a members to 1) remove trade barriers, comprehensive overhaul of econom- and 2) eliminate domestic policies that Some Useful History ic activities?Will European work- seemed to inhibit intra-EC competition The Maastricht Treaty of 1991 was ers, traditionally resistant to move- that is, to develop a truly integrated to remake the European economy. Eu- ment even to new regions within market for capital and labor as well as ropean currencies were to be replaced their own countries, be mobile for products and services. However, with the "ecu," a common currency for enough to supply these new points there were serious questions about the all European Union (EU) countries. A of resource consolidation and eco- potential for some countries within the European was to take pre- nomic activity with the labor needed EC to erode others' competitive edge, cedence over the existing banks of for such reorganization? or to seek out cheap labor in another member nations. Free trade and free EC nation leading to more unemploy- movement of workers across national 3. Potential intensification of exo- ment. Consequently in 1989 a Social boundaries within the EU were to her- nomic competition between the Charter was established which guaran- ald a new era. Of the changes to be EU and other countries. teed minimum standards of compensa- achieved, by far the greatest, according Question: Given the prediction by tion and welfare throughout the EC, to the editors of the Brookings book, experts that European integration through mechanisms such as EC-wide was the transformation of the European will increase the productivity of labor and management institutions. labor market. Workers were to be free member nations and their overall to seek and accept employment in any Unions part of the EU. competitiveness, wiil the EU pene- trate outside markets previously Analyses indicate, the editors say, The ambitious Intentions in the dominated by others and limit the that market integration in the EC spawns both centralizing tendencies Treaty represented enormous challeng- acce..s of other countries to the EC's es for governments, employers and and decentralization, as has beeh the market, with negative consequeaces workers. The -ditors point to these case in the U.S. In the U.S. the creation three as some of the most difficult to for the U.S. arul other nations, par- of an integrated market across states in- meet: twulariy Eastern and Central Europe creased centralized collective bargain- and the former Soviet Union? ing as unions advocated for more uni- 1. Competition among differing na- form wages and labor costs. Later there tional systems of workfortle prepa- The Evolution of European has been a decentralization movement ration and social welfare. integration in which unions have lost membership. Question: Will the integration pro- The 1957 Treaty of Rome moved a The EU is not likely to experienee as cess result in uniform systems of in- laige number of European nations in strong a decentralization tendency, the dustrial and business relationships the direction of fraing up import du- editors suggest, because of the tradi- anti sociai welfare, throu5h negotia- ues aM quota restrictions on interna- tional strength of European "employ- tions among "major institutioual so- tional trade. But this did Cot translate to ers' associations, mandatory works

MIIIIIMOMM10011~19.610¢MKRanyAKMER Evaluallon Pomm .Issue 11 Summtr 1 90 8 -

Evaluation Issues andActivtlies Economic ReformIn the European Union

council elections, effective deterrents to in the Brookings volume is whether Political Economy of strike breaking, the representation of pension and health benefits reduce unions in public administration and on needed worker mobility. In the U.S., "Borrowing" labor tribunals, and the legal restriction workers with health problems find it difficult to change jobs because of the Issue #9 of Evaluation Forum fea- of certain benefits to union members." tured a series of articles on the new em- potential loss of health coverage. In ployment and training system adopted Europe, health coverage mandated by Worker Participation in by the United Kingdom (UK) under Management national governments remove this bar- the Train- rier to the movement of labor. The re- Prime Minister Thatcher ing and Employment Council (TEC) The Social Charter proposed that search seems to confirm that national system. This new initiative privatized workers be infused into management social security mandates appear to in- decisions. This action,was thought to is the case, the government's effort to provide edu- crease mobility. So if this cational, occupational training and em- strengthen employei ssociations and the lower mobility in the EU must, the ployment services to workers consid- unions if it were initiated by employ- experts suppose, be due to otherfac- ers. However, it wasacknowledged that dif- ered in need of a range of assistance. tors, such as historical and cultural The series of articles included an ex- increased worker participation some- ferences. times eroded union power if initiated in planation of the elaborate evaluation the context of weak unions. Both em- According to the book's editors, re- system designed to study the imple- ployers and unions have problems with moving barriers to labor mobility will mentation and performance of the TEC this component of the Charter. German, require a common policy regarding system. Belgian, Dutch and Danish workers, for workers migrating into the EU from This article analyzes some of the un- other countries. There will be strong instance, have supported employee in- derlying features of the U.S. and U.K. pressure, they say, to negotiate anEU- formation, consultation and participa- approach to investment in employment tion because there is a common em- wide compromise between "rotation policies [Germany, as an example] and and training. It is based on a Brookings ployer-employee commitment to the Institution book published in 1993 and permanent immigration restrictions[as application of high standards to low- edited by David Finegold, Laurel Mc- for example, in Sweden]." They con- standard EU countries. On the other Farland and William Richardson, hand, French labor has not supported clude that the rotation approach is most feasible, inasmuch as an immigration Something Borrowed, Something this part of the Charter. Learned: The Transatlantic Market in policy that is too restrictive could in- Education and Training Reform. Fine- Some experts writing in the Brook- crease mismatches between skillsand ings book recommend industrial rela- job, leading to more unemployment. gold is a political scientist and educa- tions systems which do not restrict or tion specialist who was involved in the unduly limit worker participation and The Effects of European development of the TEC system and its labor-management cooperation. There integration on Other Countries evaluation strategy as a faculty member is some agreement that centralizing of Warwick University in the UK, and tendencies will ultimately dominate Although the book's experts do not is currently at the Rand Corporation in economic ef- over "national differencesin industrial see a significant negative California. McFarland is an economist relations traditions and-institutions, in fect for the U.S. in the integration of who was at the Brookings Institution union organizational strength, and in European economic activity, they are when the book was written. Richardson union attitudes concerning tne desir- concerned that emerging Eastern Euro- is a senior research fellow at the Centre ability of uniform EU-wide labor sten- pean nations could sufferconsiderably for Education and Industry at the Uni- darls. from restrictive EU immigration poli- versity of Warwick. cies and a lack of access to Western Other analysts in the book make the European export markets. If economic The Theme of the Book invortant point that a significant condi- restructuring and dislocation concen- The editors reveal their interesting tion for successful economic competi- trates in regional pockets of high unem- anthropological theme in the introduc- tion is "long-term employment rela- ployment, this could lead the EU to tionships that encourage investments in adopt protective measures that would tion to the book. They begin with the premise that education and training training skilled and versatile workers." be damaging to Eastern Europe and the policy in the US and Great Britain con- They look to the German industrial re-/ former Soviet Union. The latter, the an- lations system as a model for aunified' verged in the late 1980s, both countries alysts say, would be counterproductive becoming more dependent on each oth- Europe that is economically viable. for Western Europe in the long run. er's ideas. They claim that extensive Worker Mobility and Protection borrowing. reshaping and application of each other 's employment and train- A major question about EU integra- ing ideas took place in the 1980s, high- tion in the minds of the analysts writing lighting the influence of JTPA on the British system. Their analysis of the reasons for this borrowing andits im- plications for policy is tightly packed, a

loNNIEONIMMMININMONAINIIM, 77 Evaluation Porum Issue 11 Summar 1996 85 Evaluation issues and Activities conomic Reform In the Euroan Union

well-organized and clearly-written sort- quantity of outputs from their employ- ing out of major influences and effects the American underclass creates unique ment and training systems, was a strong problems.The editors view the stronger which ties their conclusions closely to energizer for change. At the same time, expert opinion and empirical evidence. role permitted the central government pressures for educational change were in the UK in reforming education, The economic context of the analy- endemic throughout Europe even in training and employment systemsas the sis is two-fold. Their major emphasis is Japan and South Korea reinforcing most significant difference between the ongeneral economic change in the these change efforts. two countries. postindustrial period.Such change, they say, has been the main influence Common Characteristics, It is such differences that complicate the transfer of policy between the two on contemporary employment and But Differences as Weil training reform in all countries, the editors say, and mayex- advancedindus- More generally, the editors seecom- plain some of the trial nations. They point to the integra- unintended conse- mon economic forces driving employ- quences of borrowing.The greater the tion of world markets, to technological ment and training reform in all indus- change, and to increasing anxiety about differences, they contend, the more trialized nations over the 1980s.Rapid general and abstract the policy transfers sustained economic progressas charac- technological change and the interna- teristics which differentiate this period should be, if they are to be useful and tionalization of capital generatedmore productive. from the continuing economic growth intense political scrutiny of employ- and relative stability that defined the ment and training systems the ade- Transplantation period between World War II and the quacy of the skills they produced, and oil crisis of 1973, and constituteda their overall efficiency. Under such The editors' main fascination is with stimulus for reform. But they do not ig- scrutiny, the US and the UK displayed the effect of differing social, economic nore important US borrowing from common characteristics:declining com- and political contexts on transplanted Western Europe. Their conclusion is petitiveness in world markets and low policies thatis,whether particular that such transnational borrowing will skill demands by employers;a lack of policies and programs can work in iso- likely increase in the future, as the cooperative strategies and institutions lation from the system that created them, world's academic and politicalcommu- for sharing the knowledge and costre- much like an organ transplant, nities become increasingly internation- quired to develop and implement high- and how transplantation may alter the alized due to growing economiccom- skills programs; and high drop-out original idea. A case in point is the UK petition and greater interdependence rates in secondary education and low borrowing of the JTPA Private Industry among industrialized economies. participation in postsecondary educa- Council (PIC) concept, making greater demands on the British employer The second significant focus of the tion and training. com- book is on munity than JTPA makes on American differences between US and The two countries also share acom- employers, even though the research British reforms, and those of these mon view of government's role in solv- evidence has suggested that the effec- countries' main international competi- ing such problems, the editors suggest, tiveness of PICs was questionable. The tors.In this respect, the editors are in- such as the emphasis onvoluntarism Training and Enterprise Councils terested in the "shared context of rela- (allowing the market to stimulate the (TECs) are now encountering problems tive economic decline" that is, the provision of training) andweak corpo- similar to those faced by PICs, such structural features of the two as econo- rate-government arrangements.Added the maintenance of employer leadetship mies that have discouraged the adop- to all these similarities are historical at the top levels of management, and tion of high-ski/1, high-performance linguistic, cultural and governmental the development of performance indi- strategies, and the comparatively low commonalities that were strengthened cators. It is not yet known whether the levels of education and training provid- in the 1980s by the political ideologies distribution of central government ed by both. But they see important dif- nf Reagan and Thatcher, which stressed training dollars directly to employer-con- ferences between the US and Britain as the need to free market forces fromun- trolled TECs will work, given differences well, that tilt the US further than the necessary government control, cut tax- UK toward high-skill enterprises. between Great Britain's and the US's es, reduce union influence, and increase cost reduction goals, national income dis- In 1976, Britain's prime minister consumer choice in education and tributions, tax systems and approachesto criticized the UK's employment and training. private seetor risk assessment training system for failing to respond These shared attributes are not the adequately to the needs of The direction of British borrowing is industry,and full story, however. In the US, the edi- chair Ling with John Major's tenure, and launched a series of reforms. The 1983 tors propose, higher education is seen the a.owing emphasis on European report ANation At Risk served a similar as an investment most families and in- Union status, the editors observe. The purpose in the US, and was subsequent- dividuals should be willing to make; in ly supported by Workforce 2000 policy exchange eppears to be giving reports the UK it is viewed as a right of citi- greater emphasis to EU education and in 1987 and 1990, and by Investing in zenship, therefore the governmentun- training models, although they feel that Peoplein 1989. Faced with declining derwrites tuition for the smallergroup affluence, the two countries' emphasis broader, more wide-ranging borro 'ring of students involved, Also, the larger is likely to occur on the part of all in- on skills gaps, and the quality and size and the more complex nature of dustrial nations in the future. Mean.

76 86 Ewe/mien Fetum a Wee 11 Summer 1906 Evaluationissuesand Activities .Economic Reform in theEurct Union

Technological Change while, the UK/US linkage is too strong cantly different from the international to preclude continued sharing ofpolicy competition that characterizes the 1990s. It is th, apid evolution of micro- concepts. The editors say that "British Within the new competitive environment, electronics and Wormation technolo- conservatives remain covetors of the Finegold views the increasing interde- gies that Finegold cites as the major U.S. labor market's capacity to create pendence of industrial economies as the change, in that this evolution has al- jobs and convince individuals to invest most dramatic change. Less strikingbut tered what is sold in the new market in their own education and training." nevertheless influential is the growing and how goods and services are pro- At the same time, the US has become importance of international trade, par- duced. The new technologies, he says, intrigued with Britain's approach to na- ticularly among the developed coun- allow companies to respond within a tional assessment, school-to-work tran- tries and regarding services. short timeframe to shifts in customer sition programs and experiments with Joint cross-country economic ven- demand. Forcing rapid restructuring. educational vouchers. tures have grown enormously. In the these breakthroughs have reoriented three-year period between 1985 and manufacturing approaches and reorga- ninegold's Perspective on the 1988, 400 alliances between American nized manufacturing processes. They Economics of "Borrowing" and European corporations have been have also demanded new kinds and lev- There is so much of interest in the created, 150 between European and els of worker skills. As companies have various chapters of this fine book, a Japanese firms, more than 200 between moved in this direction, fewer manual thorough review must of necessity ex- Japan and US companies. Also, the in- workers have been needed, Finegold tend beyond this issue of Evaluation ternationalization of capital has stimu- points out, and higher skills have been Forum. In this issue we focus more nar- lated cross-national high-technology in demand for the workers retained. However, he makes it clear that techno- rowly on David Finegold's intriguing development. The establishment of the logical change does not always lead in chapter on "The Changing International EU and the signing of NAFTA have this direction. Managers are five to de- Economy and Its Impact on Education supported the integration of world mar- cide whether to economize and upskill, or and Training." kets further. to increase managerial control and reduce Finegold's interest is in the econom- These developments, Finegold says, the skill level of their workforces. ic context of the heavy borrowing that have moved the industrial giants away he claims occurred between the UK and from "export-driven, nationally-cen- Economic Restructuring tered companies" to a new phase direct- the US in the 1980s. Within that con- The oil crises in 1973 and 1980 pro- text he addresses three issues: ed by global competition and global corporations. In this new economic en- duced deep recessions, and increased 1. Economic change as the major influ- vironment there are formidable new unemployment and inflation. In addi- ence in stimulating employmentand competitors for the world's capital and tion, firms found themselves awash in a training reform in all advanced in- goods: in particular Japan, but also more uncertain economic environment dustrial nations since the mid-1970s Hong Kong, Singapore, Taiwan and filled with many changes. All of the in- particularly the integration of South Korea. These new entrants into dustrialized economies felt "compelled world markets, technological change the world market scene, Finegold be- to restructure ' Finegold says. A major restructuring objective was the creation and growing anxiety about econom- lieves, are "moving rapidly into higher of greater flexibility for firms, for ic health. quality, higher value-added market seg- ments once dominated by the US and workers. This new flexibility was to en- 2. The two countries' common envi- Western Europe." To explain the impli- courage more rapid adjustment to eco- ronment of "relative economic de- cations of these changes, Finegold sum- nomic change. It was played out in school-- cline" with an emphasis on the marizes Robe:t Reich's analysis thus: more interest in improving the structural characteristics of their to-work transition, in more high-tech training, and in greater attention to up- economies that have resisted the de- As the major factors of production skilling and career redirection for velopment of high-skill initiatives, (capital, technology and raw materi- als) move more freely and quickly workers dislocated by restructuring such as their low level of education across national boundaries, labor, the economies. and training. least mobile factor, becomes a more Early production systems, with their 3. The differences between the two central component of firms' search large semi-skilled workforces, perform- countries that provide the US with a for competitive advantage. Since the ing a narrow range of work assign- greater capability to create and sus- advanced industrial countries cannot ments on machines confmed to those tain high-skill economic activity. compete with developing countries purposes,were being abandoned for on thc basis of labor cost, the surviv- more specialized "niche systems" that and Economic Change al of their manufacturing base are able to rush the development and the vast array of services which sup- production of new goods and services The period between the end of ports it depends upon superior la- to meet rapidly emerging consumer de- World War II and the 1973 rise in oil bor quality and hence the skills of mands and opportunities. The Japanese prices was ono of sustained economic current and future employees. growth and relative stability, signifl-

79 &Madan Forum k laau. 11I ummet 1005 Evaluation Issues and Activities EconomicReform In the European Union

4111111iI employment and training than do other Many American ET experts have beensurprised and confused industrial countries, which tends to dis- to learn that Margaret Thatcher lookedto their country for policy courage the development of high-skill ideas. They are surprised because they tendto view the British strategies.This conclusion is not meant education and training system as relativelysuccessful; they are to ignore common factors across na- tions that affect skill requirements. confused because, in the United States,the ET system is widely Amcng such factors, the author high- considered to be in such a state of crisisthat no nation would lightsfirm size, the nature of thepro- want to emulate it. For observers of theBritish scene, these American problems will seem familiar: duction process, and the type ofmar- significantly lower levels of kets involved. Even those countries attainment in reading, math, and sciencecompared to its main encouraging high-skill strategies evi- international competitors; high drop-outrates in secondary dence aberrations. In Japan, multina- schools, particularly in the inner cities;and growing skills tionals subcontract with firms provid- shortages in specific industrial sectors.Ironically, it is the ing considerably lowerwages and less common institutional setting of these problems,rather than any job security. In Germany,a core of record of success, that may explain theaffinity between the two highly trained, strongly urionized countries' policymakers. workers co-exists with a migrant labor force. Nevertheless, Finegold's interest David Finegold, Laurel McFarland, andWilliam Richardson is in what disincengves exist institu- inSomething Borrowed, SomethingLearned tionally and otherwise in the US and KINNAMP UK that have discouraged the march 4MMEINEMUIMMONISMEN toward high-skill, high-wage manageri- were first, al practices. These are the major influ- Finegold says, to create such panies hunting high value-addedmar- "just-in-time production systems" kets, high-skill flexibility strategies ences Finegold outlines in analyzing comparative data. which used coordinated supply chains provide significant advantages, suchas and close monitoring of consumer atti- better product quality, a reduction in In Britain and the US, the majority tudes and preferences. These yielded downtimes, and the encouragement of of individuals leave compulsory ed- substantial savings on inventories while continuous innovation. ucation with a relatively low level of achieving a high level of marketre- sponsiveness. However, Finegold qual- Different firms make varying judg- knowledge and skill attainment,par- ifies the movement toward greater flex- ments about the cost/benefit tradeoffs ticularly in math and science in ibility, pointing out that companies in of these two general strategies. These fact, they rank at the bottomon the the advanced industrial nations have judgments, the author says,are influ- latter. pursued flexibility in differentways. enced by the interdependence between "corporate/product strategies, compe- In England, there is also a high Some firms, he says, have focused tence requirements, and the employ- dropout rate prior to the comple- on cutting fixed costs, reducing their ment and training [ET] system." For tion of compulsory school. Only full-time workforce through theuse of example, managers may be less apt to one-third of the age group 16-18 part-time or temporary workers, and ty- invest in their own ET if themessage remains in school or college and ing wages more directly to worker per- from the labor market is that companies less than 18% go on to higher ed- formance through pmfit sharing, bo- are organized in ways that preclude re- ucation. nuses, Employee Assistance Plans, and wards for higher skills. Managersmay other strategies. Other companies have well make choices basedon the exist- In the US, the vast majority of emphasized decentralizing controlover ing level of ET and on the institutional students remain in school until the work process to frontline workers environment they are in. In Germany, age 18, and approximately one- who are multi-skilled. Relevant to this says Finegold, the high quality ET sys- half of this age cohort continues strategy, Finegold distinguishes be- tem generates innovation becauseman- into some form of higher educa- tween measures involving multi-ski/l- agers understand they must organize tion. ing vs. multi-tasking. The former in- the work process such that workers volves increased worker responsibility continuously have opportunities to The US high school diploma in- and enhanced skills for those absorbing learn and use new skills the sanceion volves no assessment of the new technologies or production process- being that they will be absorbed by the knowledge or skills attained, and es. The latter involves the simultaneous competition. the route into vocational training performance of a number of tasks thatare at a comparable skill level. for the non-college-bound is ob- Institutional Impediments to scure. Low-skill flexibility strategies, he investment in Employment says, reduce costs through dependence and Training Students leaving the educational on cheaper labor, and through reigning Finegold's message is that the US system in the US and UK may not in the size of the workforce. Forcom- and UK evidence less investment in close the skills gap in these modern economies. 88 vidutitionliorum Issue ii Rummor1908 _ --

Evaluation Issues and Activities Economic Reform in the European Union

One-fourth of British manufac- support a short-term approach. 2. Inadequate Educational Approaches turing companies reported in The two countries' top management 1988 that they lacked skilled work- echelons look more to the stock Both countries provide few incen- ers. Teaching and nursing were ex- market than to the banking system tives to students to invest their time periencing the same deficits. as the primary source of financing, and effort in ET. a US and UK managers do notde- because of security laws. Financial The U.S. labor market does not re- mand as high a level of skills on the market analysts focus on easily ac- ward high school effort. The UK's part of the average workez as in oth- cessible company performance elitist examination structure "brands er industrial nations. measures, such as quarterly earnings the vast majority of students as fail- and revenue, which support a short- ures." Studies find that both managers term view. American executives Employers attach little importance and workers are poorly trained. give highest priority to the increase to education-based vocational quali- West German hotels, for exam- in share prices. Japanese managers fications, and apprenticeships are emphasize market share. ple, had 65% higher labor pro- limited to a minority of students. In ductivity than Great Britain, re- The sheer numblr and size of hostile Japan and Oermany, all students lated to the greater quality and corporate takeovers in the US and have incentives available to boost quantity of training of German UK in the 1980s further reinforced a their work effort in compulsory edu- frontline staff and supervisors. focus on short-term results. Govern- cation because their performance is US and UK firms introduce new ment regulations and ownership directly related to their future occu- technologies as ways to reduce costs structures in Japan and Germany pational opportunities. preclude such takeovers. and improve efficiency, rather than 3. Relative Ease of Hiring and Firing to enhance frontline workers' skills, The organization of firms into "cost US and UK models and laws dis- make the workplace more flexible, and profit centers," and the reliance courage a linking of job security to or position workers to utilize re- on traditional accounting methods, the acquisition of skills. The labor search breakthroughs to enhance discourage measurement of the less market is the main impetus for the products or services. tangible benefits of skills upgrading, individual's choice to obtain more giving training a low priority among Using a particular analytical frame- training. German labor law and the work, Finegold draws insights from the managers. Japanese "lifetime employment" UK and US experience in ET invest- Occupational trainers themselves are ment. The interdependence of three ma- model accomplish that linkage. accorded low status and are often jor players in making investment deci- The US provides strong incentives poorly trained and remunerated, mak- sions forms the core of this framework for individuals to invest in further ing training vulnerable to cost cutting. the three being company manars, education following high school. Japanese and Gennan companies have government policymakers, and individ- More high school students continue ual workers. Another factor, the institu- made skills development an integral into higher education than in any tional setting, tips the scales in the de- part of corporate structures, using eco- other advanced industrial nation in cision to invest or not invest in ET, nomic downturns as an opportunity to part because of the value employers according to the author. Emphasized increase training, and entering upturns place on formal degrees and open are two characteristics of high-level with the advantage of better-trained skills: the length of the period prior to access to higher education. The Brit- workers. realizing a benefit from an investment ish exam system essentially rations in ET, and the various correlates of the The electoral cycles in the US and higher education opportunities. Wage decision to invest. Within these at- UK mediate against a long-term per- differentials between the skilled and tributes, these are the factors Finegold spective. unskilled are not great enough to in- concentrates on: In the US, the general decentraliza- terest those not in college in occupa- 1. The Short-Term Outlook of the tion of educational decisions to tional training. US and UK states has moved policymakers away Both countries have failed to design Finegold maintains that a long-nm from strategic planning involving "a clearly defined, high-status struc- strategic perspective is more conducive training. Central governments in the ture of vocational qualifications that to investment in ET because of the high UK and the US have focused on dis- would help motivate individuals to cost and deferred nature of the returns on advantagement rather than on high- obtain training, as in Germany," this investment. However, three structural er-level skills acquisition. elements of the UK and US economies

St Evaluation hviumI Ism 11 Smote r 1995 89 Evaluation Issues and Activities Economic Reform in the European Union

4. Absence of Institutional Struc- 5. The Lack of Sustained Pressure the Organization for Economic Cooper- tures and Processes to Foster to Innovate ation and Development. Cooperation The large size of the US domestic For more information about the Brit- The lack of formal structures and market and historically low depen- ish TEC system, the UK Employment processes in both countries erodes Department produces an interesting,in- dency on experts has supported slow interest in the mutual benefits of co- formative series of reports titledRe- adaptation to changes in world com- search Series,many ofwhich analyze operation, such as those gained petition. The UK is more integrated TEC activity. The Research Manage- through information sharing, the co- into the world economy but has trad- ment Branch of the Department also ordination of activities, joint financ- ed mostly with developing coun- produces the series Individual Commit- ing of training, the clustering ofspe- ment to LearningandInvestors in Peo- tries. And the UK has insulated cer- ple. cialized businesses and industries, tain economic sectors from and the development of employer Another useful reference is a recent competition, contributing to eco- organizations. book by Bennett, Wicks and McCos- nomic decline. han, Local Empowerment and Business Cooperation between managers and Services: Britain's Experiment with policymakers is weak, leading to an The Author's Conclusions Training and Enterprise Councils, pub- lished by UCL Press, Ltd. in 1994. over-emphasis on job-specific train- Finegold's position is that the UK ing at the expense of general skills, and US cannot any longer avoid inter- The following is a sample of relevant and on the inability to keep pace national competition and its implica- reports in the Employment Depart- ment's Research Papers and Training with technological change. tions. A decade of market-oriented pol- icies, such as privatization and Research and Development reports: The Anglo-American stress on deregulation, in the context of the in- Boreham, N. and T. Arthur, Information "voluntarism" leaves training deci- creasing integration of markets Requirements in Occupational Deci- sions to managers and their employ- sion Making. University of Manches- throughout the world demand that these ter, 1993. ees. This reduces an exchange of in- countries match or outdo the high-skill formation between government and competition. So both countries are rec- Crowder, M. and K. Pupynin. The Moti- ognizing the need to improve their ET vation to Work. Minds at Work, key producers which is critical in 1993. encouraging ET investment in systems. But Finegold is not all that op- transferable skills. Connections be- timistic. Even if the attainment level of Ernest and Young. TEC Participation in tween government, unions and em- students is raised significantly, too National Development Activity,1993. many other features mediate against ployer organizations in the US and Marchington,M.et al. New Develop- major ET reform, he concludes. He be- UK ranked last in a study of eight of ments in Employee Involvement. lieves that these other influencesmay ManchesterSchool of Management, the largest capitalist economies in discourage key actors in bosh countries 1992. their ability to formulate and imple- from investing in higher-level skills de- Pearson, I. ment consensual labor market and The Role of Evaluation in velopment. TECP/anning. WMEB Consultants, industrial policies. 1994. The industrial relations arrange- Editor's Note Pupynin, K. andM.Crowder.Regional ments in the US and UK have dis- Advice Unites: An Examination of couraged managers from pursuing David Finegold is the author of Edu- Models for Delivering Advice and the acquisition of high-level skills cation, Training and Economic Compet- Guidance to TECs and Department of Employment RegionalOffices. on the part of workers. Britain's itiveness,published by the UK's Oxford Press in 1993. He has also written in Minds at Work, 1994. craft unionism has interfered with the Oxford Review of Economic Policy Rix, A. et al. Investors in People: A the reorganization of work and in- and contributed toInternational Com- Qualitative Study of Employers. vestment in new technologies, which parisons of Vocational Education and CRG People at Work,1994. are needed to support a high-skill Training for International Skills edit..d Taylor, S.and L. Spencer. Individual flexible strategy. In the US, the by P. Ryan In 1991. William Richard. son's most recent book, on the reform Commitment to Lifetime Learning: weakness of organized labor and the of post-16 education and training In Individuals' Attitudes. Social and absence of protective legislation en- England and Wales, is co-edited with Community Planning Research, suring employee's rights has al- David Finegold. Laurel McFarland has 1994. lowed managers to design corporate written numerous articles on the econo. Vaughan,P. TECs and Employers: restructoring with little concern for mios of postsecondary education and Developing Effective Links. Employ- has been Involved skill levels, Inseveral compara- ment Department, 1993. tive education projects sponsored by the National Aeademy of Sciences and

82 Evaluation Forum IIssue 11 90 Summer1905 Evaluation Issues and Activities Economic Reform In the Former Soviet Union

Economic Reform in theFormer Soviet Union

from manufacturing to services and in- Editorial introduction Why should American educational, training and employment professionals formation in postindustrial countries, and evaluators want to know about and with the internationalization of eco- In the Interviews Section of this is- nomic competition, we clearly need a sue of Evaluation Forum, wefeature an economic reform ideas in these post- socialist countries? It is understand- broader canvas on which to paint Amer- interview with Salvatore Zecchini of the ican policy options. Therefore, we are OECD concerning current economic able that our first priority must be a bet- ter understanding of our own economic making this beginning investment in the and employment policies and their con- history of economic reforms in the Eu- system and workforce. But in the past sequences in Central and Eastern former Soviet Union. ropean countries and the now-indepen- this logical emphasis has tended to iso- dent republics of the former Soviet late us from the experiments of other This cameo of reform efforts ex- Union. In this part of Issue #11 we fo- industrialized nations.This singularity presses the basic tension in all market- USSR through a se- of focus is changing rapidly. The Ameri- oriented economies between economic cus on the former train- riesof articles on pre-Stalin, pre-Veltsin can literature on employment and efficiency and socio-economic equity, economic reform efforts. These reforms ing is now filled with insights from and therefore contributes in its own dis- constructed an important foundation for Western Europe, Japan, and the re- tinct way to our current thinking about the economic reform debates continu- structured nations of Central and East- the conduct of the U.S. economy in a so- ing in the separate republics, and in ern Europe and the former Soviet cial democracy. particular in Russia. Union. In the context of a major shift

ment of that system ... would not nonproductive individuals andbusi- A Selective Overview prove an onerous burden, given the nesses, according to Hewett, andunfair of Soviet Economic educational level of the labor force in distributing rewards for successful and the smooth operation of systems competition. These correlates of mar- Reform Policies inherited from capitalism. Socialism, ket-driven systems are not supportive for Lenin, was the marriage of the ef- of egalitarianism. In fact the introduc- In thisarticle we selectively review ficiency of capitalism and the equity tion of extensive social and economic an informative BrookingsInstitution so fervently sought by generations of entitlements, Hewett contends, can seri- book by Edward A. Hewett published socialists.... Now, seven decades ously reduce a market economy's effi- in 1988 Reforming the Soviet Econ- after the Russian Revolution, Soviet ciency and profitability. Therefore the omy: Equaliry versus Efficiency. A se- leaders are still searching for a for- form Soviet economic reform has tend- Stud- nior fellow in the Foreign Policy mula that will make that "marriage" ed to take has essentially required a re- ies programat Brookings, Hewett's work. They have a system that can definition ofsocialism. analysis of the pre-Gorbachev and Gor- boast many successes, most notably Hewett views the continuing debate bachev reforms provides useful insights in achieving some of the equity goals about the Soviet experience in revital- over the nature and direction of eco- of fundamental importance to Lenin nomic reform in the USSR as a debate izing and restructuring its economy. and his Bolshevik party. The state, principles. Hewett's main message is that while over efficiency vs. equality virtually the sole employer, has a tre- He says the essence of this debate is the Stalinist system was successful in mendous appetiter labor. Thus the achieving industrialization and secur- whether a viable method can be found USSR has almo3I no involuntary to "dilute the egalitarian bias" of the ing social and economic equity, it was unemployment, still one of the main not suited to the ongoing management system in achieving more economic ef- sources of waste and human misery that is, whether a strategy of a mature and increasingly complex ficiency in much of the capitalist world.... can be implemented to accomplish re- industrialized economy or to the This is the ultimate "welfare state." achievement of economic efficiency. form goals without destroying socialist .. It isthe efficiency side of the legitimacy. Relcant to this viewpoint, the follow- marriage that has not gone at all well. ing %....cerpt from his book is enlighten- The Meaning of Economic ing: Pressures for economic reform in the direction of a market economy have Reform in the USSR Lenin's visionof the rust, socialist, been the legacy of this dilemma, but We tend to forget that Nikita Krush- phase of communism was powerful such reform remains a quandry. Well- chev proposed economic reforms in in its simplicity. He foresaw a system functioning market-oriented economies 1957, Aleksei Kosygin in 1965, and in which workers ... would enthusi- are dependent on economicefficiency Leonid Brezhnev in 1973 and 1979. astically pitch in to produce foreach andcompetitionfor customers. These However, Hewett does not view these other, distributing the fruits of their economies can be ruthless in punishing as "comprehensive" reforms. They labor fairly and equally. The manage-

83 Evaluation Forum km* 11 Summor 1005 91 Evaluation Issues and Activities Economic Reform in the Former Soviet Union

weremore in the nature of partial re- outcome of refonn efforts. Studies of earn income. This dependence on wages forms that contributed to later reform these outcomes for the Krushchev and and , rather than on assets, then efforts by alerting leaders to what Brezhnev eras suggest that the particu- had a leveling effect on income distri- seemed to work well and what did not lar mix of characteristics these reform work. Neither does he see Perestroika bution. The reduction of wages related strategies involved ultimately proved to low unemployment, and the security under Gorbachev as a comprehensive fatal, leading to economic retrenchment reform in the economy, if we define protections afforded workers irrespec- rather than to improved performance. tive of the economic profitability of economic reform as a change in the their workplaces, also equalized the dis- way resource allocation decisions are Strengths and Weaknesses in madei.e. in the "institutional ar- tribution of income. Added to these in- Soviet Economic Performance come-leveling effects was the high rangements" that determine howre- par- sources are produced and distributed. Hewett's analysis pinpoints Soviet ticipation of women in the labor force. central planning as the most critical At the same time, economic losseswere The author points out that economic source of performance problems. Sovi- absorbed not by single enterprises but reform in the Soviet Unk.. zAnd Eastern et leaders attempted to preserve what spread out across the whole system. Europe has been associated most often they judged were the strengths in the Even the shortage of goods wasan with "economic decentralizmion." This existing system, including central plan- equalizing influence. focus is understandable given these ning, while seeking to reduce or elimi- The main strengths in the Sovietsys- countries' economic performance prob- nate the weaknesses, but the two sets of tem lems. However, Hewett acknowledges egalitarianism, economic securi- attributes were so intertwinedas to re- ty, and the growth in living standards that "economic recentralization"can duce the effect of this compromise. also be seen as a reform, whether or not were generally regarded by Soviet it is appropriate in resolving economic The three main strengths Soviet leaders as socialist achievements that problems. leaders saw in the existing economic must be preserved. Consequently these system were 1) the high level of growth leaders tended to minimize theirpoten- The important issues to study inana- in the economy and in living standards, tial negative effects on economicper- lyzing economic reform el forts, accord- 2) the high degree of economic security formance in proposing economicre- ing to the author, are 1) wl tether there- in the context of this growth, and 3) the forms. Since many of the weaknesses in form decentralizes or recentralizes de- system's commitment to egalitarianism. performance were closely tied toa com- cisionmaking regarding resource pro- Hewett claims that the USSRcame as mitment to economic security and equi- duction and allocation, 2) whether the close to full employment as any indus- ty, reformers had to redefine these con- reform is comprehensive, and 3) trialized economy can hope toaccom- cepts in the process of designing new whether the reform affects all major plish "the degree of personal eco- strategies. Hewett feels this helpsex- economic institutions. Some changes nomic security in the workplace [was] plain the severe difficulties leadersen- are not reforms, he says, such as eco- virtually unparalleled in Westerncoun- countered in Soviet reform movements. nomic policy changes that utilize the tries." existing system to improve economic Inefficienc;es in resource allocation were not the primary concern in devel- performance. But such changes are im- Egalitarianism was, of course, a key portant in making significant shifts in element in Marxism. Marx's major crit- oping new approaches. Technical ineffi- the way the economy is guided without icism of capitalism was that it produced ciency garnered more attention. That is, the risk of organized resistance to and/or reinforced inequities in the dis- Soviet reformers felt that the misuse change. tribution of wealth, income andpower. and waste of resources in the production He viewed the existence of unemploy- process were more critical issues to deal However, Hewett feels that while ment as a symptom of this inevitability. with than the issue of what sectors of economic policy changes should be This ideological perspective explained the society benefited mostor least from part of reform packages, more systemic socialists' emphasis on the means of economic progress. This priority led So- economic reforms must be the major production as well as on the allocation viet planners to force enterprisesto drivers of change, particularly given of resources. The Russian revolution economize on inputs that is, on labor the performance problems that faced was the first effective strategy, Hewett and material since these were being the Soviet Union during the Gorbachev says, for acting on this perspective in a used at much higher rates thanwas nec- period, He would likely take thesame nation-state, even though the socializa- essary within Soviet society, and more position today. tion of the means of productionwas than was typical in other industrialized And Hewett cautions that reform completed only after an abusive collec- nations. movements are not all of one piece. tivization of the peasantry. The decline in economic growth Each is distinguishable from others In this kind of system, where the rates and labor productivity was Gor- through its own array of personalities, state is the only owner of financialas- bachev's key worry regarding economic forces and conditions, the particular sets and production, Hewett comments performance. In a search forcauses, So- combination of which influence the that one must be gainfully employedto viet experts identified several important antecedents:

Evaluatloa Fotum Issue 11 Summer 1006 Evaluation Issues and Activities Economic Reform in the Former Soviet Union

The low level of mechanization of mined the allocation of bonuses to en- protect their empires and enterprises a from sharing decisionmaking with oth- Soviet industrial enterprises. terprises, and in price control. Among other things, these reforms proposed a er businesses. If the,e were efficiency The tendency to produce low-quality recentralization of planning, moving it gains, Hewett's view is that they were goods that did not respond to con- back from regional authorities to cen- swallowed up by burdensome adminis- sumers' needs or world standards. tral ministries in Moscow, and a tration. strengthening of the discretion of exist- An excess demand for consumer or The author draws three key lessons ing state committees. investment goods. from the failure of these reforms: The most rapidly implemented strat- 1. Do not ask the very entities you A tendency for industaial develop- egy was administrative reorganization. hope to reform to be responsible for ment to move ahead of infrastruc- Implementing the incentive system and the reduction of their own powers. ture development, creating supply/ price reform lagged considerably be- demand imbalances. hind. This delay allowed the existing 2. Do not neglect the demand-side con- system to organize its resistance to the sequences of reform, while focusing With the slowing of economic changes ahead. Many of the reforms growth, Soviet policymakers had less on supply-side solutions the were never implemented, and the im- mergers actually increased the high discretion in setting priorities among pact of those put in place was not en- the competing interests and pressures concentration of Soviet industry and during. from consumers, enterprises and gov- the power of suppliers, which can- ernment agencies seeking sources of in- Hewett places the most blame on the celled out the reform's potential in- vestment, particularly in the area of de- design of the reforms. It involved too centive effects. fense spending. Consequently they many changes at one time, some unre- were faced with the need to preserve alistic. This created too much pressure 3. Do not allow a central government economic equity and security while on the system and led to a dilution of body to control merger activity, boosting economic growth. This chal- strategies before they had been given a since this activity then may not be lenge underlaid all reform efforts. chance to work. driven by a search for increased Since it was difficult to determine the The author concludes that the ad- profits. potential causal linkages between phe- ministrative recentralization strategy In 1979, there was another effort to nomena such as falling productivity, had the most lasting effect. It reinstated redirect the economy. The most inno- low quality output, imbalances in sup- a Stalinist decisionmaking hierarchy to vative component, Hewett believes, ply/demand, excess demand for invest- manage the economy, expanding the was to develop performance indicators. ment goods, the sustaining of full em- number of central ministries and state ployment, and the maintenance of price At the risk of oversimplification, under committees. Succeeding leaders ex- this system an enterprise using more la- stability, economic reforms always rep- panded this apparatus further in an ef- bor than a certain determined amount resented agonizing compromises, often fort to control an increasingly complex based on inadequate data and research. would not be rewarded, whereas more economic system. productive and efficient ones would be. Earlier Reform Efforts There were problems with the use of The 1970s Reforms aggregate measures of enterprise activi- Hewett is adamant that Western The reforms proposed in 1973 at- ty, however, and no serious effort was readers place the Gorbachev reform tempted, Hewett claims, to increase the made to implement this reform. package in historical perspective if they efficiency of this managerial system are to understand its importance. His without decentralizing it. This was to Reform Accomplishments emphasis in analyzing pre-Gorbachev be accomplished by merging various Taken as a whole cloth, there were reforms is to cast Gorbachev as the first enterprises under one decisionrnaking two common elements in this series of soviet leader since Kruschev to con- authorfty, permitting this authority to ceive, strongly support and actually im- generally unsuccessful post-Stalin re- shift production tasks and input allot- forms: plement economic reforms. ments from the least to most efficient enterprises. Brezhnev wanted the cen- The basic tool of reform was the The 1960s Reforms tral authority to move away from the central economic plan. This plan It is the comprehensive 1965 re- operational management of enterprise was developed by central govern- forms that provide the best'context. activity into long-range strategic plan- ment ministries and involved indica- The strategies proposed were similar in ning for entire economic sectors. tors to determine enterprises' com- many ways to Gorbachev's, and were However, the 1975 reforms, which pliance with the plan, as well as the intended to affect fundamental aspects were to merge enterprises into more allocation of bonuses to enterprises of the Soviet economy. In particular controllable units, did not result in the that met or exceeded the plan's ex- they involved ohanges in who super- changes planned. Ministers managed to pectations. vised enterprise activity, in what deter-

85 Evaluation Forum hua Ii Summe 1095 93 Evaluation Issues and Activities Economic Reform in the Former Soviet Union

The reforms did not focus on the Hewett emphasizes that reform in- ciency of central planning, but in gen- needs and demands of customers in volves a complicated process requiring eral supported the ministerial system the open market, but rather on cen- the involvement and investment of all even though opposing its authoritarian tral decisions about what constituted relevant stakeholders. This participa- control over enterprise activity. The high quality goods. There was a fear tion cannot occur without neutralizing idea of mergers appealed to most of of loss of control if world markets the effect of the proposed changeson these economists, where planning could were allowed to direct Soviet enter- those losing something, and without be consolidated for a small number of prises. enlisting the support of those having major economic sectors. something to gain. But the Sovietre- As a result of Soviet resistance to forms continued to permit a moreau- In the midst of these considerations, giving "the market" the primary role in thoritarian process. Mikhail Gorbachev opened Pandora's increasing economic efficiency, the Box, taking the position in his speech Relevant to the design of reforms, pre-Gorbachev reforms experimented at the 1985 Party Congress that the eco- he advocated a first step dedicated to mainly with plan indicators and incen- nomic situation was such that itwas no the development of very clear priorities tive systems, leaving the organizational longer possible to make partial im- mechanisms oprating the existing and emphases, followed by strategies provements what was needed, he economy largely intact. capable of anticipating and reducing said, was "radical reform." The subse- the inevitable conflict created by gains quent development of radical reform Hewett makes these comments at the and losses in the major groups affected positions took two forms: 1)compre- end of his analysis of the pre-Gor- by change. hensive reforms similar to the Hungari- bachev reforms: an model, and 2) reforms that sought a The Economic Reform Debate better economic compromise between In their totality, economic reforms In the 1880e imply a theory economic performance goals and the .. . concerning the na- vast safety net protections for individu- ture of the problems, the systemic Hewett classifies the Soviet players in the ongoing 1980s debates about als and families. The latter position changes needed to fix the problems, brought to the surface the underlying and the best strategy for bringing economic reform into three general cat- egories: and ongoing tension between economic about those changes. In the history of efficiency and socio-economic equity. Soviet reform efforts, most of the The neoconsmatives, who support- leadership's attention has been fo- ed a return to the basic principles of Although Hungary was viewed with cused on the first two components of Stalinist central planning, but with- interest, insights from the reforms initi- the thecrq: what's wrong, and what's out its correlate of political repres- ated in other socialist countries hadre- the fix? The strategic issue has not sion. markably little influence on the nature been addressed openly, and. .. the of Soviet reform ideas. Thiswas the approach has been rather simple- The moderates, who accepted the case even though Gorbachev encour- minded: the bureaucracy will, in existing system but had significant aged a formal study of reforms in East- good faith, design and implement de- ideas for improving it. ern Europe, particularly Poland, Hun- crees conforming to the wishes of the gary and East Germany. The latter party. However, the modest results of The radical reformers, who advocat- models could have been very useful, previous reform efforts have gradual- ed system-wide reforms, particularly but serious consideration of them flew ly stirred up interest in the strategic a decentalization of the manage- in the face of historical Soviet beliefs issue, with the result that a rather ment of the economic system. that the problems of the USSRwere unique and could be understood best in simple, widely held theory has The core of the neoconservative po- isolation from the broader tapestry of emerged that previous reforms failed sition was that the economic system primarily because of bureaucratic re- 20th century socialist reforms. must become better disciplined sistance centered in the ministries but ... proponents differed considerably about this view is not wrong, but it is in- The Gorbachev Reforms the means for accomplishing this. The complete and superficial...it fails to main question was whether markets The five-year plan emerging under take into account the flaws in reform should be used as the tool for imposing Gorbachev for 1986-1990 had veryam- design that create incentives for min- such discipline. b:tious goals. It proposed a reversal of istries to preserve controls over en- the downward trend in economic terprises. Furthermore, it assumes Most Soviet economists fell into the growth rates in all major output catego- that other elements of the system are moderate group, which representeda ries. Hewett reports that national in- anxious for the reforms to work... wide range of views about how tore- come was to grow at an average of this ignores the large and important structure the economy, stretching be- 4.1%, and labor productivity at 4.6%, tween a commitment to strong central political issues that affect in a funda- comparable to the early 1970s. Realper planning at one end of the continuum, mental way the choices made in the capita income growth rates were toav- design of refonns and the fate of the to radical comprehensive reform at the erage 2.7%, significantly above recent resulting decrees. other. Most wanted to see substantial rates, even if considerably below the improvement in the quality and di- early 1970s, In tbe 1990s these rates

86 Evaluation Forum Issus11 Summar 1996 Evaluation Issues and Activities Economic Reform in the Former Soviet Union

were to be increased even more dramat- warding or sanctioning enterprises for economically if it could not compete in ically. their performance. Interestingly, East- world markets. But he was equally con- ern European and Chinese economic cerned, according to Hewett, about na- The means for meeting these targets, reforrns emerged in some of these prin- tional security and the level of domes- over a two-phase process, was to be a ciples, even though uncredited. tic support for the Communist Party. massive improvement in the quality of Hewett's fear was that if the Gorbachev goods and services produced. Gor- Direct, detailed central management reforms lost momentum, the USSR bachev presented the five-year plan as a of enterprises through an annual plan might rely primarily on military power route to setting world quality standards, was to be jettisoned, leaving enterprises in the context of residual strength in not simply a way to meet the plan's responsible for the economical use of that sector but weakness economically goals. This was unrealistic, more a po- their assets, and for either the rewards and politically. litical ploy thao a rational economic or costs related to the general perfor- position. Hewett's opinion is that the mance goals set for their own economic Concernedabout the outcomes of real test of Gorbachev's reform plan sector by the central government minis- the Gorbachev reforms, Hewett propos- lay in whether he would support suffi- tries. These entexprises were also to be es four possible scenarios for the fu- cient economic discipline to achieve the freed, Hewett says, from local manage- ture: high-quality output proposed. ment interference. In essence they were 1. The success scenario: everything to be left vulnerable to competition in works as planned. The reforms oc- The first stage of reform was to fo- the marketplace. Within this frame- cur rapidly, output quality improves cus on partial changes. The second was work, state enterprises were to be to involve the more comprehensive placed in competition with new private substantially, and so does economic changes. In 1987 the major outlines of sector self-employment schemes and efficiency. the second stage were fleshed out by small . For the Soviet labor the Aganbegian Scientific Council 2. The high growth scenario: the na- force, the Gorbachev reforms, accord- tional income growth rates are close working with a system of committees ing to Hewett, amounted to "threaten- assigned to design different compo- to what was planned, with only ing the workers of poorly managed en- moderate improvement in the quali- nents of the reform. This activity pro- terprises with lower income, and even duced much open debate and better the loss of their jobs, but at the same ty of goods and services. consensus-building, and a set of guid- time ... givingthem a role in choosing ing principles were established that 3. The high quality scenario: econom- and controlling enterprise manage- ic growth performance is moderate would direct economic reform based on ment." the Council's work. but quality indicators express signif- The deadline for implementing this icant and sustained improvement, Hewett describes these principles as second and most all-encompassing even though economic efficiency is a set of interrelated changes in all of phase of economic reform was January not substantial. the key mechanisms operating in the 1991, in retrospect a deadline beyond economic system a comprehensive Gorbachev's tenure as Soviet leader. 4. The failure scenario: nothing and radical reform package: works. Annual central plans were to be The Implementation ofthe Hewett sees some scenarios as more eliminated. Gorbachev Reforms probable than others. The extremes Private sector economic activity and Hewett thought implementation were less likely to occur. However, the development of economic coop- would be very difficult, with "powerful "success" would, he thinks, require that and frequent temptations to retreat from almost everything 'go right' the de- eratives were to be encouraged in the principles of the reform." The most sign of the reform perfect, the imple- certain sectors. important political question, he sug- mentation strategy flawless, and radical Enterprises were to be allowed to go gests, was how to reduce the negative improvement in quality resulting. Re- bankrupt. effects of the transition to a new eco- ferring to the history of previous re- nomic system and therefore create and form efforts, he says npither the work- Workers were to be allowed to lose maintain the new system's legitimacy force nor industrial management was their jobs. with important stakeholders and the likely to respond at a level that would Prices were to be far more flexible. Soviet people. Serious economic ques- allow this to happen. tions attached to the pace and chronolo- The monetary system was to re- gy of reform, and the fairness with Implications of Success or spond to a truer medium of ex- which the transition was carried out. Failure for the West change. Hewett is convinced that Gorbachev It is intriguing, given the coup at- This agenda was to make substantial was more interested than any other So- tempt against Gorbachev, the subse- changes in the decisionmaking hierar- viet leader in the USSR's position in quent end of his leadership, and the dis- chy, the information system for policy- the world economy. He believed the solution of the Soviet Union, that making, and the inetintive system re- Soviet Union would not be successful Hewett predicts that the "failure" see-

INNWIEM011111~011.0111MINNINEMSNIEW &ablation Forum Issue11 Summar titteS 87 Evaluation issues and Activities . Economic Reform In the Former Soviet Union I. nario would be very unlikely, since it Gorbachev's environmental groups, and press rich would require some "dramatic turn- countries and corporations to assume around" such as the removal of Gor- Perestroika their share of the global clean-up task." bachev and a shift of power to econom- ic hardliners. Without benefit of a As president of the former Soviet Although Gorbachev hardly seems crystal bail, Hewett suspects that the Union, Mikhail Sergeyevich Gor- to qualify as an environmental czar, two middle-range scenarios would do bachev became deeply involved in eco- given some of the history of his tenure most to enhance Soviet economic rela- nomic issues, particularly reforms that as president of the Soviet Union, the tions with the West. However, he is could lead to greater economic perfor- concept of an International Green Cross less certain about the benefits for Sovi- mance. The Twelfth Five-Year Plan, was, in fact, Mr. Gorbachev's idea et foreign and defense policy. His gen- for 1986 through 1990, gave substan- a/though the organization is supported eral assessment is that successful eco- tial expression to his goals for reform: by an international group of prominent ,nomic reform would give Soviet higher national income growth rates, political, scientific and spiritual lead- leaders the capabiliry either to follow a the acceleration of economic develop- ers. And his affinity for ecological pro- less aggressive foreign policy or to fi- ment, the improvement of production tection is not as inconsistent as it might nance a greater military build-up. The and its products, and changes in the seem to the West. choice, he thinks, would rest with the public's view of the economy and soci- He joined the Communist Party in level of success in reforming the Soviet ety. The reform concept and plan for 1952, quickly becoming a leader of the political system. change was described in his book Pere- Stavropol Komsomol. In 1955 he grad- Hewett expects that successful re- stroika: New Thinking for Our Country uated from the Law Department of forms could lead to a country better and the World published in 1987, Moscow State University as part of an able to pursue traditional foreign policy which significantly influenced econom- elite group of Russian lawyers. Be- ic thinking not only in the former and defense goals, become a new cus- tween 1985 and 1990 he followed a tomer for Western goods, and a new USSR but in Central and Eastern Eu- straight-line political trajectory leading ropa. competitor in world markets. It could to the status of First Secretary of the also mean a gradual democratization of Mr. Gorbachev no longer exerts di- Stavropol Territory Committee of the the political, system, consistent with the rect political, economic or ideological Communist Party's Central Committee, long-term interests of Western nations. influence in the new republics formed then Chairman of the USSR Council of Success could lead, he believes, to by the break-up of the USSR, but he is Defense and a member of the Presidi- greater participation by the Soviet anything but a has-been. Following his um of the USSR Supreme Soviet, and Union in international economic insti- fall from power in 1991, he formed the ultimately to the president of the Soviet tutions and markets, and to new rela- International Foundation for Socio- Union. tionships with the West based more on Economic and Political Studies in Born on a farm in Southern Russia economic than defense issues. A reform Moscow, and subsequently an arm of and steeped in rural agricultural life scenario that stimulated technological this foundation in the United States, growing up, he had long been con- innovation and improved the quality of The Gorbachev Foundation USA in cerned about deteriorating fisheries and Soviet goods and serviles would, he San Francisco. The foundation serves forests, and about the management of concludes, enhance Soviet economic as an economic and political 'think farming, and had served as the USSR's ties with the West. Unsuccessful reform tank' intended to promote social de- top agricultural official prior to becom- scenarios could reverse these trends. mocracy in the former Soviet republics, ing president. Therefore the interview Hewett does not think Western in- and sponsors international seminars Mr. Gorbachev provided to Colin Greer and conferences on global economic, fluence over the outcome of the Gor- for Parade Magazine in January of bachev reforms would be more than political, environmental and peace is- 1994 is informative regarding his view sues, including defense conversion. minimal. The West would be capable of his new role vis-a-vis international only of exerting pressure for minor cor- Only sixteen months after serving as environmental activism: rections in the reform process, perhaps president of the Soviet Union and gen- making that process more likely to oc- eral secretary of its Communist Party, We need to spawn a different intel- cur. This forecast appeared to be accu- he was named president of the Interna- lectual and values climate and a dif- rate for the early Yeltsin period, but is tional Green Cross and Green Cres- ferent set of economic goals. No now debatable as neoconservative cent, which aspire to represent for envi- country actually believes the free views have gained momentum with the ronmental crises what the Red Cross market can be socially irresponsible. growing divisions in political leadership. and the Red Crescent already represent I want more socially oriented market for disaster relief. According to Micha- economies in the world.... At the el Wines of The New York Times, Mr. same time, the business marketplace Gorbachev's idealistic plan aims to that is based on economic freedom, "standardize antipollution laws world- that can function with independent wide, integrate the diverse efforts of producers and owners, forms the ma- terial basis for democracies. I see no

Evaluation Forum 0. Issue 11 Summar 1995 96 Evaluation Issues and ActMties . Economic Reform In the Former SovietUnion

In this sense,Perestroikainvolved a renewal of every aspect of Soviet life. The course of intensification is dictated by objective conditions,by the entire course of development of the country. There are no Implementation Challenges basis alternatives. Only an intensive economy, developing on the Gorbachev's thinking, the work of of a state-of-the-art scientific-technical base, can serve as a the designers of his reforms, his ac- reliable material base for increasing the welfare of workers, claimed book, and the institutionaliza- guaranteeing the strengthening of the position of the country on tion of new ideas in significant policy the international arena, ensuring that it will deservingly enterthe pronouncements, launched enormous new century as a great and prospering power. changes in the USSR which have con- Mikhail S. Gorbachev, in Pravda, December 1984. tinued to influence developments in Russia and the other republics. This is not to deny that there were serious challenges to economic reform during contradiction between the market and active, constructive collaboration to this early reform period as well as cur- democracy. If we are going to protect improve the economic situation. Pere- rently. the planet's ecology, we are going to stroika was not only about reforming need to find alternatives to the con- the economy but about opening up and One challenge has been the need to sumerist dream that is attracting the democratizing Soviet society, and en- modernize the economyandshift in- world. Otherwise, how will we con- gaging people more directly in its im- vestment policies.Another has been the offreeing central authorities serve our resources, and how will provement. necessity from an operational role,to a more we avoid setting people against each In June, 1987, the CPSU Central other when resources are depleted? general focus on improving the eco- Committee adopted principles for im- nomic environment and directing the Given this revised view of the politi- plementing this.approach to reform in basic steps in economic development. cal architect of Perestroika, what were "Fundamentals of Radical Restructur- The new economic system was expect- the majorrtarms proposed within this ing of Economic Management." This ed toemphasize more decisionmaking second kussian revolution in the was likely the most radical program for authorityandmuch greater responsi- 1900s? economic reform in the Soviet Union bility and accountability at the regional since Lenin introduced his New Eco- and local level.Theexpanded role for The Concept of Perestroika nomic Policy in 1921. It proposed a private and cooperative economic ac- transition from an excessively central- tivity also posed challenges. The most immediate priority at that ized management system to one based time, according to Gorbachev's book, ondemocratic centralism and self- Certainly the shift from obligatory was to put the economy in order, tight- SoPerestroikahad sev- State economic targets to norms linked en discipline, and raise the level of or- management. eral dimensions: to the performance of enterprises has ganization and responsibility. A pro- been a genuine challenge. The reorga- foundstructural reorganization of the The overcoming of economic stag- nization of different levels of the gov- economywas needed, he felt a re- nation and an acceleration of social ernment hierarchy, and the emphasis on construction of its material base, the and economic progress and dyna- the coordination of activity have con- development of new technologies, mism, stituted new hurdles. Establishing a changes in investment policy, higher of new incentive system affecting both en- management standards. a The comprehensive development terprises and workers, as well as the democracy and an encouragement of branches of government monitoring The government, he proposed, need- individual initiative, creativity and ed to shift economic activity from new them, has also tested the reform move- construction to the technical retooling discipline. ment. An additional and very important of enterprises, to saving resources, to A shift to scientific methods, and a challenge has been deciding whether sharply raising the quality of output. At better linkage between the achieve- some reforms should be implemented before others, and sorting out how to the same time it needed to change the ments of the scientific and techno- deal with theeffectsof the transition. moral and psychological situation in logical revolution and a planned society, and the organization, style and economy. methods of work. A policy cfopenness Comment and social justicewas needed. Better living and working conditions For Centraland Eastern Europe, and It was imperative, he contended, that for Soviet citizens. for the former Soviet Union, the transi- tion to a market economy clearly repre- government take into consideration the Greater social justice. diverse Interests of people, of work col- sents a difficult historical passage. This lectives, of public bodies, and of vari- An elevation of honest, highly-qual- is certainly to be expected given the ous social groups, drawing them into ified labor. magnitude of the economic, social and cultural changes occurring in these so- cieties in the context of their traditions.

89 Evaluation Forum Issue 11 Summer1995 97 Evaluation Issues and Activities Economic Reform In the Former Soviet Union

wit gditar's Note The USSR in the To hire business managers and ex- For readers interested in more infor- perts to increase efficiency. mation about Mr. Gorbachev's ideas, Transition Period please see: To provide fiscal incentives to work- In this article weagain take a look at ers to increase productivity, such as Peace Has No Alternatives. A collection the pre-Yeltsin period in the Soviet removing the limits on personal in- of speeches, articles and interviews. Union. The perspective this time is come end increasing the amount of in- South Asia Books, 1988. based on Alexander Samorodov's come exempt from 'taxation. Perestroika: New Thinking tor Our chapter in a book published in 1993 by Country and the World. New York, the Organization for Economic Cooper- A Framework for Reform NY: Harper Collins, 1987. ation and Development,Structr,..ral A Road to the Future. The text of Mr. Change in Central and EasternEurope A wide range of newlaws were Gorbachev's address to the United edited by George Fischer and Guy passed in 1990 and 1991 to implement Nations. Ocean Tree Books, 1990. Standing. The author gives us a more reform goals. However, Samorodov's The August Coup: the Truth and the detailed appreciation of economic and conclusion is that despite this legisla- Lessons. New York, NY: Harper Col- labor force issues in the Soviet Union tive activity no adequate legal frame- lins, 1992. prior to the creation of the independent work for a market economy had been republics. put in place by late 1991. The reasons were many. Samorodov suggests the Launching the Reforms following: Samorodov introduced his chapter Population growth rates were declin- on "The Labour Market, Social Policy ing except in the Central Asianre- and Industrial Relations in the USSR publics. Under Transition" with some bleak sta- tistics. The USSR's GNP was estimated Urbanization was continuing to have declined 17.9% in 1991 and two-thirds of the population were production by 15%. Trade with other urban residents in 1990 compared countries had fallen over 39% in 1991. to 48% in 1957. The official inflation figure was 10% Total employment was declining but for 1990, but may actually have beenas high as 25%. The rouble had been de- new employment forms were emerg- valued six times since December 1990, ing, such as cooperatives, self-em- a decline of over 80% in its value. ployment, and joint stock companies. This was a very difficult way to be- Although mass layoffs had not oc- gin a transition to a marketecenomy, curred, part-time, less than year- Samorodov muses. Nevertheless, in round, and seasonal employment April 1991 the Soviet Cabinet of Min- was growing. isters indeed proposed to launch sucha reform. A number of goals were to be Labor supply and demand were seri- accomplished within the reform, such ously mismatched in 1991 nearly as these: 97% of laid-off workers weregov- To resuscitate economic ties be- ernment employees, but 85% of the job openings were for blue collar tween the USSR and COMECON, workers. the trading network involving Cen- tral and Eastern Europe. The lack of geographical mobility, housing and employment services, To increase internal economic link- ages between enterprises. and complicating administrativereg- ulations, were hampering thereem- To provide "most favored nation" ployment of the unemployed. This status in international trade to those was in the context of a society inex- republics willing to sign the union perienced in dealing with unemploy- treaty. ment, where the tendency was to To begin privatization, particularly view the unemployed as personal regarding small businesses. failures. To ban strikes temporarily to stabi- Two-thirds of young people (teenag- lize the labor market. ers and those in their 20's) lacked vocational training.

Evaluation Forum Issue 11d Summer 1995 96 -

Evaluation issues and Activities Economic Reform ;n:the Fortrilo_n

Consequences tor Workers it became the foundation for sithse- six weeks, and there were special ex- quent market-oriented Iws. The i 991 tended benefits available for the long- Samodorov reports that a high and law and related legislation reinforced term unemployed, ar

91 Evaluation Forum Issue 11 Summer 1995 99 Evaluation Issues and ActMtles Economic Reform In the Former Soviet Union

ment than to the skill level of the work- The Market Meets Its Match (Harvard University Press), by force, particularly to structural unem- Alice Arnsden of M.I.T., Jacek Kochanowicz of the University of ployment. However, with the changed employment situation of the 1990s, re- Warsaw, and Lance Taylor of the New School for Social placing labor shortages through labor Research, offers a critque, of "shock therapy"as practiced in surpluses and increasing retraining and Eastern Europe, envisioning instead a gradual transition to free related support programs became criti- markets based on East Asia's experience.. .. But Jeffrey cal. Many workers had a narrow range Sachs, an economist at Harvard and an intellectual architect of of skills which frequently failed to fit shock the. apy, brushes aside the authors' alternative... . Mr. the emerging occupational structure. Sachs contends that the successes writ so largeon Asia's The author feels that "the scale of nearly blank economic slate can-hardly be models for retraining" at this time was inadequate. economies with legacies of bureaucratic meddling anda fierce The average Soviet worker was less sense of populist entitlement.. ..Mr. Sachs argues that the well trained than his counterpart in Eu- comparisons [with Japan, South Korea and China] are rope. Most workers were used to stay- spurious. The Asian miracle economies started theprocess of ing with a given enterprise their entire development as rural societies with little capital,no government working lives. With the move to a mar- safety net and very low expectations. Eastern Europe, by ket system, businesses questioned the contrast, is largely urban and heavily Indeed, excessively wisdom of retraining, given the uncer- Industrialized.... Eastern Europe is stuck with the crumbling tainty of long-term werker tenure. So in 1991, the USSR initiated a reorgani- remnants of a Stalinist Industrial infrastructure, wary multina- zation of their retraining system, estab- tionals that insist on controlling their own technology, and lishing formal training/retraining cen- workers who are linked to enterprises the way medieval ters. peasants were tied to manors... . As for gradualism, Mr. Sachs points out that China dropped the commune system overnight. Privatization ... And while China has never disciplined its state-owned Samodorov views privatization as a Industrial sector by opening markets or eliminating subsidies,it major component of USSR reform ef- has built a modern economy around the relatively small forts under Gorbachev. The main strat- decaying core something Eastern Europe cannot afford to egy implemented during that period do because the decaying core isn't small.... Shock therapy was the establishment of joint stock still seems superior to the alternatives." companies in place of public enterpris- es. These often involved shareholding Peter Passell, In the January 19, 1995 New York Times by employees. By the end of 1990, over two thousand industrial enterprises, ap- proximately two hundred large con- ing to larger social expenditures, a and trade unions. The market, however, struction firms, thirty-three thousand greater budget deficit, and rising infla- was to determine wage and salary levels small businesses, and eight hundred tion. Therefore wages were growing in through "collective contracts between municipal and public transportation en- spite of the decline in production and management and labour." These were terprises were placed under a lease sys- labor productivity. And higher wages bargaining agreements, quite different in tem. Based on a survey of enterprises, were, to large extent, demolished by concept from contracts under the previ- those defined as unprofitable were rap- higher inflation. Pressures from work- ous centrally-planned economy. idly privatized. In the agricultural sec- ers to right this wrong began to surge. tor, private farming began in earnest in Meanwhile, growing income in- Skill Levels and Retraining 1989 replacing a portion of the collec- equality was occurring due to the polar- Samodorov reminds the reader that tive and state farms, and by 1990 there ization between those with fixed in- the USSR's workforce was, in geheral, were approximately one hundred thou- sand private farmers. comes (grants, stipends and pensions) quite skilled. For example he reports and those with deregulated wages and that in 1991 there were 35 million spe- A new Union of Lessees and Em- salaries. New trade unions of million- cialists (skilled workers and - ployers covered more than ten million aires emerged, while pensioners' stan- als) and 82 million skilled blue collar people. The Scientific Industrial Union dard of living plummeted. A new poli- workers. Graduates from new - incorporated many of the largest enter- cy for negotiating wages and salaries al training schools had increased annu- prises and associations. This Union evolved to address this new socio-eco- ally. Over 33 million workers and 10.5 represented two-thirds of the USSR's nomic stratification. It involved "a sys- million managers and specialists were industrial output. Even so, Samorodov tem of tripartite collective labour agree- being trained on an ongoing basis that reports that only about ten percent of ments," according to Samodorov year. So the new training challenges the total labor force was employed by the government, employer associations, were more related to rising unemploy- private sector companies, AIMAIMM7,71,* 92 Evaluation Forum issue 11 Summer 1095 1 0 0 - -

_

Evaluation Issues and Activities Economic Reform in the Former Soviet Union

Unions continued to grow in power, BeckeA. Sitting on Bayonets: The Sovi- ditotial:COMMeni-v,, : even though they were part of new et Defense Burden and the Slowdown three-way partnerships. Their new sta- of Soviet Defense Spending. Santa The articles in this international sec- Monica, CA: Rand Corporation, 1985. tion of the journal have focused on a tusunderPerestroika gave them inde- range of economic and employment is- pendence from political and state insti- Bergson, A. and H.S. Levine (eds.) The sues facing all industrialized nations in tutions, Samodorov explains, and the Soviet Economy: Toward the Year the 1990s. Major efforts to reform and opportunity to provide legal and social 2000. London, England: Allen and restructure economic activity in the con- protection to workers in the private sec- Unwin, 1983. text of unprecedented international tor. Laws regarding unions assured Breda, J., E. Hewett, and T. Wolf (eds.) competition are having a significant im- equal rights for all unions, whether in Economic Adjustment and Reform in pact on cyclical and structural employ- the public or private sector, and permit- Eastern Europe and the Soviet ment in these countries. ted them to participate in negotiations Union. Washington, D.C.: The Some of these nations are in a over a range of issues, including their Brookings Institution, 1986. postiniustrial phase of development. interest in wage indexation. Brown, A. 'Gorbachev: New Man in Others are in transition from collectivist, the Kremlin," in Problems of Com- authoritarian regimes to social demo- But the author comments that few munism, Volume 34, 1985. cratic market economies. But they managers of enterprises in the newly share a major policy challenge: to in- privatized businesses had training or Campbell, R. 'Economic Reform in the crease economic efficiency and com- experience in how to operate private USSR," in American Economic Re- petitive performance while preserving sector market-oriented companies. As a view, May 1968. socio-economic fairness and equity. result, management training was accel- Gaddy, C. Perestroika and the Soviet Many advanced and developing erated, often utilizing experts outside Military Industry. Washington, D.C.: economies are experiencing substantial the republics. The Brookings institution, 1992. political change in this decade. How Gorbachev, R. Hope: Reminiscences politics will intersect with economic de- Some Conclusions and Reflections. New York, NY: velopment goals and the pace of tech- Harper Collins, 1992. nological change is yet to be decided. While the independent republics that This web of complex relationships will followed the dissolution of the Soviet Guroff, G. and F.V. Carstensen (eds.) ultimately affect what employment and Union continued to favor the transition Entrepreneurship in Imperial Russia training professionals are able to con- to a market economy, their economic and the Soviet Union. Princeton, NJ: tribute in resolving employment prob- health remained problematic. Unem- Princeton University Press, 1983. lems, and in taking advantage of inno- ployment continued to rise. Retraining Hewett, E.A. "Gorbachev's Economic vative workforce and workplace centers were slow to function. The Strategy: A Preliminary Assess- possibilities. most significant challenge for govern- ment," in Soviet Economy, October- ments was the development of efficient, December 1985. effective labor market policies that Energy, Economics and could reduce the negative social conse- Foreign Policy in the Soviet Union. quences of the market reforms. Washington, D.C.: The Brookings In- stitution, 1984. . . Hough, J.F. The Struggle for the Third Editof s Note...... World: Soviet Debates and Ameri- can Options. Washington, D.C.: The Alexander Samorodov Is with the Brookings Institution, 1986. Employment and Development Depart- ment of the International Labor Organi- Jones, A. and W. Moskoff (eds.) Pere- zation in Geneva, Switzerland. For two stroika and the Economy: New additional references relevant to the is- Thinking in Soviet Economics. sues the author addresses, please see: Washington, D.C.: The Brookings In- stitution, 1989. Samorodov, A. "Labor Market Problems and Developments In the Republics," (eds.) The Great Market Debate in in G. Standing (ed.), In Search of Soviet Economics. Washington, D.C.: Flexibility. Geneva, Switerland: Inter- The Brookings Institution, 1991. national Labor Organization, 1991. Matthews, M. (ed.) Party, State, and Standing, G. (ed.). In Search of Flexibil- Citizen in the Soviet Union. Wash- ity: The New Soviet Labour Market. ington, D.C.: The Brookings Institu- Geneva, Switzerland: International tion, 1990. Labor Organization, 1991. Zaleski, E. Planning Reforms in the So- For additional references on the viet Union, 1962-1966. Chapel Hill, former Soviet Union, please consider NC: University of North Carolina the following: Press, 1967.

Evaluation Forum Issue11 Summer 1995 101 93 Evaluation Issues and Activities At the National Level - _

At the National Level

government. This concept has been how we are to judge the performance of welded to two existing characteristics of separate programs, and programs co- In the remaining pail of this section our employment and training efforts ordinated withal workforce development on Evaluation Issues and Activities, we I.e. decentralization and privatization. systems. In the next series of articles turn to the issue of the coordination/in- The new emphasis is on system-wide we obtain a snapshot of federal efforts tegration of workforce development policy development, strategic planning, to Integrate human services, explore programs and services and their ac- simplified and coordinated service de, some of the problems in the existing countability requirements. We first ad- livery, and a common concept of ac- E&T system now well identified through dress this issue at the national level, countability at the state and local lev- process and outcome evaluations, and then at the state/local level. In discuss- els. In this movement toward system discuss some of the issues involved in ing expert opinion and research focus- development, the role of the federal developing strategies for measuring ing on coordination, we see the con- government is yet to be determined. and judging performance both in cept of workforce preparationsystems Defining rights and responsibilities at all separate programs and In workforce gaining wide acceptance at all levels of levels is nowlaced with concerns about systems._

El MI Some Thinking porters advocating coordination. And forts. Federal authority should have an its funding arrangements mediate oversight component also, according to About E&T against coordination and integration. the report, in terms of defining and Coordination at the measuring expected outcomes across Studies of Services integration: programs. The analysts propose that an National Level Some Results of Interest additional component of central control should be networking that is, bring- In this article we trace a bit of the Twelve reports detailing studies of service integration conducted between . ing individuals involved in integration history of national-level coordination efforts together to share information efforts admittedly a very selective 1972 and 1987 are reviewed for their major findings. Figure 1 provides an and build support across federal agen- piece of that history. We look at an in- cies and program bureaucracies. At the teresting but little-known review of co- overview of some of the results these reports share in common. same time, the federal government ordination efforts in 1991 prepared by needs to provide more funding flexibil- the Inspector General's Office in the Recommendations ity, they suggest, in order to reward U.S. Department of Health and Human In the IG Report separate organizational entities for Services, and at a more recent series of coming together. reports produced by the U.S. General The goal of earlier integration ef- Accounting Office. forts, the report says, was major com- An Impetus for Coordination: prehensive system reform too high The U.S. GeneralAccounting Lessons from Past Federal an expectation, the analysts claim. The Office Reports Efforts to Integrate Human evaluators suggest a more modest goal Services Management and that reflects satisfaction with shorter- With increasing interest in deficit re- Delivery term, more incrementalaccomplish- duction, recently the U.S. Congress has ments. requested a series of analyses giving at- A rather extensive literature has ac- tention to the proliferation of federally- cumulated regarding human services The Inspector General's report also funded programs in various program integration since the 1970s. Services recommends a service provision focus areas. Two general reports in the em- Integration: A Twenry-Year Retrospec- on clearly-defined target groups, con- ployment and training area, titled Mul- tive is an eye-opener in this respect. sistent with more manageable goals. tiple Employment Programs, were pub- The tragic conclusion of this report, Both recommendations underline the lished in June 1992 and July 1993. however, is that over the long haul importance of goals which can be sus- Subsequently, a series of reports were "service integration effort. nave had tained politically and organizationally. produced under the same general title little institutional impact on a highly The evaluators acknowledge that but with the following subtitles: fragmented human services system." some degree of central control (federal :the barriers the report identifies are fa- National Employment Training control) over an otherwise complex and miliar: the system is too large, com- Strategy Needed (June, 1993) fragmented human service system is plex, bureaucratic, professionalized, necessary. This would provide a needed Conflicting Requirements Hamper and specialized. It lacks powerful sup- external stimulus for integration ef- Delivery of Services (January, 1994)

=1.1=...1112,MM....=.=1.r1=1M....1 94 Evaluation Forum Issue 11 Summer 1095 Evaluation Issues and Activities At the National Level

1 Timeconstraints: seMceintegra- 3 The lack of cross-training of start funding levels leading to an unsure tion requires an evolutionary pro- and the development of "general- resource base. cess that allows time to develop 8 Deering program rules and proce- linkages, gain support and legiti- 4Competitionamong providers. dures. macy, plan, organize, and imple- ment. 5 Unclear accountability expecta- 9Lack of timely, comprehensive and tions. relevant data. 2 Negative attitudes and behavior regarding the concept of service 6 Differing funding rules/regulations. 10 High administrative costs. Integration on the part of system 7 Different funding cycles, and fluctu- 11Chganizational territoriality and po- personnel. ations in the type of funding and litical opposition.

asp

Moderate-scale service Integra- Z Linkages aro more readily devel- 8 Effective leadership and consen- flea efforts are often more suc- oped in stable rural environments sus-building are necessary. cessful than large-scale compre- than in large urban areas. 7 Long-term planning is critical, In- heNsive human service planning. 4 Staff co-location often enhances cluding setting goals and develop- 2 Human services organizational coordination. ing strategies for service integra- tion and its funding. networks must be adapted to lo- 5 R & D funds are essential. cal environments. 8 Political support is required.

,eewpte,ee-10.:4«'. ' sv... rie tra on

1 Improved service delivery acces- 3 Increased efficiency. sibility. 4 More effective assessment of client 2 Greater continuity. needs, and strategiesforrespond- ing to them.

Most Federal Agencies Do Not U.S.Department of Labor. Many of Assessment and job placement assis- Know If Their Programs Are Work- these provided similar services and oth- tance were often duplicative and ing Effectively (March, 1994). er assistance to overlapping client pop- wasted resources. ulations. A numberofproblems were Programs were not tracking partici- Overlapping Programs Can Add occurring: Unnecessary Administrative Costs pants' progress through programs or The federal network of employment (June, 1994). their outcomes effectively. and training (E&T) programs was Overlap Among Programs Raises sufficiently fragmented to be creat- In the midst of these problems, how- Questions About Efficiency (July, , ing difficulties for workers, employ- ever, the GAO was finding that a num- ber of states were reorganizing their 1994). ers and program administrators. E&T efforts to increase the coordina- Major Overhaul Needed to Create a A lack of access for potential appli- tion of service delivery at the local lev- More Efficient, Customer-Driven cants to information about these pro- el. The GAO analysts recommended Sptem (February, 1995). grams was creating conffision and this kind of energetic coordination frustration for those needing various movement at the federal agency level, Conclusions In the Reports kinds of services. and the development of a national E&T policy and strategy. In the two earliest reports, the GAO Needs assessments were often being found that 125 federal programs were In the January 1994 report, the GAO made by service providers with a summarized results from an intensive providing some form of employment vested interest in particular service and training assistance to out-of-school review of nine E&T programs primari- youths or adults not enrolled in ad- "treatments." ly serving the economically disadvan- vanced degree programs: 49 in the U.S. taged. They found them to have similar Department of Education; 30 in the goals, often to serve clients with coin-

4119+wwwwwimmwiswirMnwmfINT.Mw=130MINI Evaluation Forum Issue11 Summor 1995 95 -

Evaluation Issues and Activtties r At the National Level

mon characteristics, and to have paral- Employers had difficulty obtaining Utilize evaluation strategies for lel but separate service delivery sys- qualified workers from the E&T sys- learning about the system's effec- tems. These attributes, they concluded, tem. tiveness that are pragmatic as well as led to unnecessary administrative costs. scientific. The evaluators also discovered con- Elements of Effective EST flicting eligibility requirements and dif- Systems The GAO's assessments and recom- ferences in eligibility criteria, compli- mendations generally coincide with re- cating the eligibility determination The GAO did not hesitate to suggest cent Department of Education and De- process across programs. And differ- what an effective E&T system might partment of Labor initiatives in ences in annual operational and budget- look like. It should: education, training and employment, ing cycles hindered administrators in Be customer-driven rather than pro- with state-level coordination efforts, coordinating their services with those gram-driven, and with local-level E&T agendas. of other program3. Provide easy access to services. These initiatives and efforts are The February 1995 report character- moving workforce development in the ized the existing E&T system in this Involve an efficient use of resources. direction of greater integration through way: Offer a wide variety of E&T servic- state/local, private/public systems that are guided at the federal level by a gen- aIt was fragmented organizationally es. eral policy and strategy and by general and in terms of service delivery de- Be accountable for the results. accountability parameters, but which spite common goals, overlapping give states the major responsibility for client groups, and similar services. Afford flexibility at the local level pulling programs, services and data to- in determining how best to meet It was wasteful of resources. gether and developing methods for es- community needs. tablishing system accountability. aIts administrative structures and Move toward a consolidation of pro- functions overlapped. Comments grams. Separate funding streams reinforced The MIS and GAO reports give us .... Have clear points of entry to the sys- program territoriality. an updated perspective on the problems tem and identifiable pathways and possibilities in workforce prepara- It was difficult for clients and em- through the system for customers. tion. The concept of systems ofpro- ployers to understand ways to enter grams is the outcome. The 1970s' and Negotiate a common set of require- ,A111 and access the system, and learn ments and administrative procedures 1980s' notion of services integration about its services. was coordination at one level of such a across programs. system at the local-service delivery There was considerable data incon- Design its service delivery strategies level. The modern boundaries of "sys- sistency across programs even in partnership with employers and tern" include the policy, planning, man- though measures were being collect- unions. agement and oversight levels as well. ed on similar variables. Develop a data colleetion system ca- aThere was a lack of objective infor- pable of tracking participants, the mation about programs for manage- Editors Note services provided to them, and their ment and oversight purposes. Infor- outcomes across programs, in-state In a separate report produced in mation systems did not allow February 1995, the GAO provided a geographical areas, and states. analysts to learn about relationships useful summary of its numerous reports between client characteristics, ser- Reduce administrative staffing for and Congressional testimonies on the Department of Labor's programs be- the system to a minimum. vices and outcomes. tween 1989 and 1995. The summary Only modest E&T gains for partici- Develop clearly defined goals, mea- includes a review of DOL-funded pro- surable objectives, and performance cess and net impact evaluations of pants were identified in program these programs. evaluations. standards.

104 Evaluation Forum Issue 11 Summer 195 eir17°----etete-^77 "etee't=

Evaluation Issues and Activities At the National Level

Elm linguistics. As a result, these highly databases. Figure 1 lists these, as ex- Measuring trained individuals are taking egeater empted directly from the report. The Performance in interest in one another, and each oth- authors propose that a training stu- er's policies and strategies for estab- dent's labor market status can be Programs and lishing accountability. Not unrelated to thought of as a joint outcome reflect- Coordinated this new accomodation, there is signifi- ing three sets of influences: the stu- cant movement toward a beginning dent's qualifications, the skill require- Systems consensus about the meaning ofpeefor- ments of employers, and employment mance management. opportunities. The multiplicity of influ- This article focuses more specifical- ences underscores the conclusion that ly on some of the variables that are In this context, Stevens and Shi fo- we should not expect any single cause- important to measure in performance cus on a major issue in performance effect relationship to exist between vo- systems coordinating data across pro- management, the relationship between cational education and labor market grams. It also addresses major data vocational educational "treatments" or outcomes. sources for obtaining such measures. interventions, and employment and Two national reports are reviewed: one earnings. Several questions concern To reinforce this position, the au- emanating from the vocational educa- them: thors refer us to previous national as- tion system, one from the employment sessments of vocational education that What does information about this re- reached a similar conclusion. The Na- and training field. Although these re- lationship tell us? ports were written at the beginning of tional Institute of Education's 1981 Vo- cational Education Study, for example, extensive state-level experimentation What variables should be given at- with comprehensive workforce devel- tention? stated that research results concerning opment systems, the general themes are this relationship could not be interpret- highly relevant to the measurement of How should these variables be mea- ed as an assessment of the effectiveness performance in these systems, as well sured so that the indices are reliable of vocational education programs because of general data limitations, but as to programs within them. and valid? more importantly because of the diffi- Beyond Performance Standards Some Useful Variables in culty in attributing either economic or by David Stevens and Jinping Shi Studying the Relationship noneconomic outcomes to participation of the University of Baltimore. Between Vocational Education le various kinds of vocational educa- and Employment/Earnings tion training. And the authors report Prepared in 1994 for the National that the fmal monograph by the Nation- The authors propose several catego- Center for Research in Voeational Edu- al Assessment of Vocational Education ries of variables that should be consid- cation (NCRVE), which is funded by in 1989 concluded that "institutional the U.S. Department of Education, the ered, and which are often absent from subject of this short paper is informa- tion production, expansion and coordi- nation. Traditionally, employment and OcatiOnaliducatiOnal Labor Nteiket Outc*n:le'S. training professionals and vocational education specialists, and those who evaluate these different sets of pro- Qualification Factors grams, have practiced in relatively sep- abilities arate worlds. Beginning in the late motivation 40, eV ' 1980s, and now in the 1990s, Presiden- nonvocational education vocational education tial initiatives and federal legislation Observed Labor have been bringing these worlds to- work experience other Market Outcomes gether, albeit gradually and sometimes employment reluctantly. At the same time, new employment continuity! Requirement Factors to generate workforce preparation councils and skills mobility boards have been struggling to bring other earnings these professional territo, ies together in change In earnings a new way at the state level, at the Opportunity Factors industry aftiliationt,$) same decisiomnaking table. entry-level hiring retention These new federal and state coordi- promotion nation efforts have informed each worksite learning group of experts about the other's pro- continuing education fessional backgrounds, education, ex- perience, knowledge base, culture and

97 se IMO 11 Summar 1995 Evaluation Forum 105 ,..4FFI.egammvaammulan., ?=1-.0sowag..WIW tttt' ete----.eenent

Evaluation Issues and Activities i At the National Level

performance may reflect not only dif- Labor, in agreeing on a definition of outcome information, were drawn from ferences in program quality but also skills standards for industries. Two a four-state consolidated database de- differences in the types of students steps are involved: 1) defining the in- veloped in 1991 by The Jacob France served." Sheets and Shi clearly seek dustry itself, and 2) deciding whether Center at the University of Baltimore. greater appreciation of these complexi- the standard will be based on one of The goal of this project was to illustrate ties on the part of managers and evalua- three kinds of thresholds enay-level how wage record data could be com- tors. performance, the typical performance bined with administrative data from Qualification Factors. These fac- of those working in the industry, or state vocational education systems in tors refer to attributes employers tend leading-edge performance. order to conduct more extensive and to consider in making a judgment of a In reality, each type of definition is useful management analyses. The potential worker's qualifications, in influenced by changing technology, job project was successful in raising man- terms of initial hiring, retention in em- availability and the qualifications of agers' awareness of ways they could ployment, skills training, subsidized workforce candidates. Although UI learn more about patterns in student ex- education, or promotion. For example, wage record data can offer an opportu- periences following their participation evidence of having reached a certain nity to understand more clearly how in programs. educational threshold, such as posses- worker qualifications and employer re- sion of a certificate or degree, is a sig- quirements fit, an intervening variable Earnings Profiles nificant influence. The national skills in this relationship must also be consid- One example from the Center board, part of Goals 2000, is expected ered, namely opportunities. project was the identification of distin- to reduce dependency on program-com- Opportunity Factors. Job avail- guishable patterns in earnings profiles, pletion as a threshold, substituting in- which corresponded to descriptions of stead skill competency thresholds con- ability and the nature of available jobs are two major opportunity factors. In a whether and when students experienced sistent with the establishment of progress on a career path that is, in industry requirements and standards. fast-paced economy, it is not possible, the authors recognize, to align skill ac- jobs that involved "sustained affiliation Another attribute may be an employ- quisition with its use when the time- with a single employer, and an observ- er's recruiting strategy. The authors ac- frames are short. However, worker able increase in annual earnings." knowledge that "the value of an at- qualifications and employer require- Stevens and Shi identified five pat- tribute as a discriminator among ments mean little if there are limited terns: candidates falls as a fraction of those employment opportunities. The traditional career path pattern: who offer the attribute rises." However, immediate and sustained earnings they claim that in instances where few- Isolating Cause-Effect growth. er potential hires exhibit a particular Relationships from a valued asset, the qualification factor Complex Mix of The delayed-start pattern: low ini- may enhance or diminish labor market Influences tial earnings following by sustained outcomes associated with the training earnings growth. institution or program involved. Such The authors' ultimate goal is to ins prove vocational education managers' qualification influences complicate per- The short-circuited pattern: initial formance measurement. For instance, understanding of the mixture of influ- earnings growth followed by earn- the use of Ul wage record data in the ences affecting labor market outcomes, ings reaching a plateau. and how these tend to define the rela- absence of additional information can tionship between vocational education- The one-step improvement pattern: either help or damage a vocational pro- al interventions and employment and low initial earnings with discontinu- gram. The message is that where you are dealing witn more than one influ- earnings. The challenge, they say, is for ity, followed by a higher but flat managers to have access to enough ap- ence on multiple outcomes, the inter- earnings path. pretation of data on relationships be- propriate measures to permit them to tease out the effects vocational pro- The dead-end pattem: flat or declin- tween these variables must be carefully ing earnings over time. qualified. grams may be having on hiring and re- tention, and to merge this information The importance of distinguishing pat- Requirement Factors. The require- with a database on employment out- terns, the authors believe, is in their use ments of employers interact with the comes. The use of wage record data is in monitoring earnings over time. The qualifications of potential workers, analyzed in this context, and the au- emphasis in such monitoring is on un- making it difficult to understand caasal thors' conclusion is that it is a useful derstanding the "horizontal trajectory relationships between training and em- database for this purpose. of earnings over time," rather than de- ployment outcomes. Sheets and Shi Examples of fruitful coordination of pending on single point-in-time snap- point to problems faced in the Skills databases, which include I rariables in- shots. The longitudinal approach per- Standards Project, funded jointly by fluencing employment outcomes plus mits managers to look both at levels the U.S. Departments of Education and and directions of earnings.

1.06 Evaluation Forum Issue 11 Summer 1g95 a

Evaluation Issues and Activitlesr, At the Nalionsl Level

Cautioning that these earnings pro- with some of these issues in the process such as from other states, the federal files should not be treated as gospel, of encouraging the development of a government or educational institutions. the authors nevertheless propose that reliable, comprehensive wage record This missing data is important, the OTA report states, since analyses that such ideal-type constructs contribute database. usefully to the debate over the rele- compare the outcomes of educational and training programs can be misinter- vance of training-related placement Purposes Served by Wage standards, and over the timing and use Record information preted in the absence of such data. Ob- taining data from other sources can be of snapshot measurements of earnings. The OTA paper suggests four major complicated, however, requiring specif- Neither do the profiles allow clear-cut purposes for UI wage data: cause-effect judgments to be made. ic data-sharing agreements that assure More must be known about a student's To conduct research on trends in the confilentiality. education subsequent to the voeational employment and earnings of work- To address the latter issue, the 1992 education treatments being examined, ers, and the employment effectsof JTPA Amendments requested the Bu- or about the work-based learning ac- alternative strategies of income sup- reau of Labor Statistics (BLS) to devel- quired. port, education, training, and social op a plan for creating a National Wage Even so, the Sheet and Shi analyses services. Record Database (NWRD), which would pool all wage records in existing of data on the labor market experiences To monitor and evaluate specific ed- state unemployment insurance systems of ex-vocational education students ucation, training, work/welfare and three and one-half years following their into a national database available to all related social programs regarding states. Core data elements were to re- completion of this education are very their effects on the economic wel- innovative. They alert us to important side in the states, or be sent to a central fare of individuals, in order to make issues in data reliability and validity, data bank operated by the federal gov- that is, for perfor- ernment. The database was to consist of and to the advantages of consolidating improvements appropriate indicators for a range of mance management. a national sample of individuals, sam- critical variables in better understand- ples within states, or the entire working o71 To provide the public with informa- population. ing the effects of training programs. tion about the outcomes of pro- grams, and employment trends in Both the cost of maintaining a na- Wage Record Information tional database and the confidentiality different industries, occupational Systems. Office of Technology challenge remain a concern, according Assessment, U.S. Congress. sectors, and geographic areas that to the OTA report. The BLS has con- support their decisions as customers sidered proposing a statutory strategy Last year the Office of Technology of service. Assessment (OTA) reviewed efforts by for resolving the privacy problem. Or the National Occupational Information To enforce compliance with policies data confidentiality might be assured if Coordinating Committee to encoth-age concerning eligibility and payments wage record information were classi- fied as statistical rather than adminis- the use of UI wage records for conduct- in other social programs. ing policy studies and monitoring the trative information. Another concern outcomes of vocational education. The These purposes can be served by UI discussed in the report is whether the Committee had previously designed a data universally, the report says, since development of such a database would series of demonstrations to explore the 90% of the U.S. working population is undermine the primary purpose of the technical problems in producing and included in the UI wage record data- UI information system in the eyes of using wage records for this purpose. base. And three key data elements employers and the public. Beyond these Subsequently it was to develop proce- the SSN of all employees covered by issues, there is some anxiety that the dures for maintaining a nationally-ac- UI, their quarterly earnings, and the in- use of a national database for enforce- cessible database for wage record infor- dustrial code or business name/address ment purposes might erode its function mation. The Committee also contracted of the employer are collected by all for performance management and eval- with MPR Associates to design a train- states. Furthermore, longitudinal earn- uation. ing guide to assist states in using wage ings histories can be constructed be- record data for studying the outcomes cause state agencies accumulate em- The Level of State Use of vocational education and other pro- ployment and wage data over time. of Ul Wage Records grams. Under the 1988 amendments to the Obtaining Additional The purpose of this OTA research Deficit Reduction Act, states were re- Employment and quired to collect quarterly wage reports paper is to summarize what has been Wage Data learned from the demonstrations and from all employers as part of a new In- from other studies, to identify data For individuals not included in state come Verification and Eligibility Sys- quality issues, and to make suggestions UI information systems, information tem. This vastt-a increased the number about how the Congress could deal must be obtained from other sources, of data-sharing agreements. The Na-

Evaluation Forum luus 11 Summer 1095 107 Evaluation Issues and ActivitiesAt the National Level

liana! Commission for Employment start-up costs were considered to be they would otherwise that is, Policy reported a 50% growth in such higher than long-run average costs, and they may classify thensztelves in- agreements between 1986 and 1991. there was considerable uncertainty correctly as belonging in indus- The program providing the greatest about what cost categories were includ- tries or jurisdictions with the low- experience in using U1 wage records to ed in these estimates. Also, no esti- est tax rates. mates were made of the cost of entire track outcomes has been ITPA, the Employers may not report non- OTA report concludes. The use of management information systems of which wage records were a past, to de- wage payments as accurately as .TTPA data was a sufficiently important they do wages and salaries. option that the U.S. Department of La- termine what proportion of the costs bor funded demonstration projects in the wage record portion represented. Employers may report severance 19934994 to study this use vis-a-vis One study reported by OTA, con- payments to dislocated workers follow-up questionnaires for judging ducted by the National Commission, only after these workers are ter- compliance with 1TPA performance compared the costs for UI tracking with minated from employment. standards. the JTPA follow-up system using ques- U1 system policies may change, Meanwhile, in Florida's Education tionnaires. The estimate for the JTPA leading to changes in data defini- and Training Placement Information telephone survey was $19/per complet- tions and measures that are pocr- Program, UI earnings data were being ed intetview. In this respect, the Florida ly communicated. system provided a reasonable bench- combined innovatively with data from The "matching rates" for the indi- the state's community college and uni- mark regarding the cost of comprehen- sive information systems approxi- viduals tracked in the Ul system versity systems, the public schools, the may be too high or too low. state corrections system, and federal mately sixteen cents per person tracked, agencies. Employers were also sur- based on tracking 1.8 million people. U1 data may need to be obtained veyed to determine the occupational Clearly there are large economies of from multiple sources, with the pos- categories of all students tracked within scale. sibility of inaccuracies. the system. A common set of outcomes The OTA report asks us to view measures was used: quarterly earnings; costs in the context of the length of The choices made about the number weeks worked; occupations; further en- time individuals are tracked. Attrition and kind of sources used can affect the "matching rate", or "the percentage of rollment in adult education, community in survey samples increases with the all persons being tracked for whom college or higher education; and move- length of follow-up. Such problems are ment into the military. The consolida- virtually eliminated by the requirement some positive identification of employ- tion permitted analyses of outcomes for in UI for a quarterly submission of ment, unemployment or other activity appears in the outcome data." This rate, all state education, training and em- earnings information. And the difficul- ployment programs. This comprehen- ty in locating members of the follow-up the OTA report stated, is analogous to the response rate in questiotanaire- sive data bank has become a basis for sample using the survey raises costs. based surveys. The higher the matching state strategic planning and budgeting. The Commission estimated that the UI method cost only about one-fifth as rate, the more trustworthy the results of In 1992, the National Governors' data analyses, Association reported that 20 states had much as the survey strategy. explored the use of UI wage data for The OTA report says there are no tracking vocational education students, Quality Problems with UI Data clear-cut rules for deciding what the and 12 were using these data on a regu- But the world of information sys- matching rate needs to be in order to lar basis. Washington State, for exam- tems is never perfect. The OTA report avoid bias. The suggestion is that this ple, had combined 1) wage record data takes the position that administrative determination will require time, the ac- across five states within its region with sources of data are more difficuit to use cumulation of state experience with dif- 2) enrollment data from the state's uni- and interpret than data from surveys ferent configurations of data for differ- versity system, and 3) employment in- where samples of respondents can be ent purposes, and specific studies formation from the U.S. Department of randomly selected and response catego- comparing UI data with data from other Defense. ries are under the control of the analyst. sources. Some states had also estimated And there are some other problems. Issues to explore are differences in costs. In reviewing states' information, These were the major ones to which the 1) the rates of unemployment among the National Commission estimated av- report gives attention: the group tracked, 2) where these indi- erage development/operation costs for ts Wage record data are subject to sev- viduals are employed, 3) how they earn their income, and 4) the likelihood of a wage record information system as eral kinds of systematic errors. approximately $1.75 per person their acquiring education, joining the tracked. In vocational education, one Employers' tax liabilities may armed forces, going to prison, or en- expert estimate was $3/person. But cause respondents to report dif- gaging in another activity. People who ferently to the L1 system than receive U1 payments can be positively

11MMIli.1 ftIMMWMIMEM/004611.. 100 Evaluation Forum Issue 11 Summ(1995 _

Evaluation Issues and Activities At the National Level

identified. Those who do not receive experience all of which are relat- cantly between wage record and sur- therneannot be. More vocational edu- ed to employment and earnings. vey data. cation graduates continue on to further This information can come from Earnings estimates may be different education than do JTPA completers, for program Management Information for different client groups. Low-in- esample, and therefore the latter's Systems, but dependence on these matching rates are higher. come, disadvantaged populations information systems would tend to appear to have more earnings from To increase the matching rates for limit wage record studies to popula- unreported sources than dislocated different groups, the number of sources tions of individuals for whom there worker populations, for example. fer outcome data must be expanded, the is such data. Without such informa- Systematic differences may then report suggests. With vocational educa- tion, employment and earnings tion students, for instance, outcome persist in the accuracy of earnings trends emerging in analyses may be estimees for these groups. data can be requested from neighboring due more to changes in the composi- states, the home state's higher educa- Differing earnings est; lates can be tion system, and the federal govern- tion of the labor force than to pro- gram interventions. And the differ- related to the scientijk ment. Some experts say a matching rate offollow-up surveys: sampling of 90% is needed. It is clear that a per- ences in outcomes may be more procedures, questionnaire construc- formaeice management system based related to differences in the charac- only on wage record data from one teristics of program participants than tion, interview formats, and methods state could be very misleading. to program treatments. In terms of of analysis. The repor also points out that local performance management, the OTA Wage information recordkeeping in changes can wongly affect the results reports says, this lack of adequate the UI system can be a source of er- of analyses "when comparisons are . data could lead to "creaming." ror. No large-scale audits of the made among specific institutions and Wage records do not distinguish be- system have been conducted, there- programs rather than among statistical- tween part-time and full-time em- fore the nature of errors will not be ly-selected samples of individuals or ployment. There can be large differ- known until the large BLS audit is institutions covering much larger ar- completed. But the OTA analysts eas." ences in the earnings estimates, the OTA report states, due to very small think it is likely that the quality of Earnings data are affected by the fluctuations in the proportion of in- recordkeeping varies across states. quarter selected for reporting them. dividuals working part-time rather Some states compute annualized than full-time. If "weeks worked" is Some Conclusions earnings. If the first quarter of the included as a data element, analysts The OTA report concludes, as do year is used as the basis, actual earn- can restrict their studies to full-time most researchers, that "badpdata may be ings will be overstated. Earnings are workers. However, only seven states worse than no data." They recommend more accurately calculated if the were collecting this element in 1994. that studies look at the relationship be- fourth quarter is used. Therefore, in tween data quality and system design, performance management systems There are differences in the accura- as well as the other issues identified. where earnings are estimated on the cy of Ul records vis-a-vis survey in- The report offers recommendations basis of the first quarter's worth of formation. The New Jersey Unem- relevant to BLS's plan for the creation earnings, the results of analyses are ployment Insurance Re-employment of a national wage record database. likely to punish those programs with Demonstration Prtject found that These are useful in framing information the Jarger proportion of graduates/ earnings estimates for a sample of production issues. Given that the pur- completers. Estimating earnings dislocated workers were higher if pose of a national database would be front four quarters of data, or adjust- based on UI data on the first quarter policy research, program evaluation ing for the quarter used could yield a after unemployment, than if based and consumer information, the report includes these suggestions: very differe.et result. on surveys. In the second and fourth quarters, they were lower. The high- Congressional review of states' ex- ei Analysed of wage data cannot con- er figures in the limner case were periences with the UI wage record trol for program effects due to due to severance payments; the low- system, the improvement of state differences in the background char- er figures were due to a growing studies using UI wage data in terms acteristics of individuals. Wage number of workers occupying out- of their quality and utility, and the records do not include information of-state jobs. The National JTPA use of these data within performance on demographic characteristics, so- Experiment found the same phenom- management systems. cial background, or years of work enon. Estimates of earnings of low- Improvements in the federal govern- income individuals differed signifi- ment's assistance to state user agen- Evaluation Issues and Activities At the National Level

cies in utilizing wage record data in . Editor's Note Employment and Earnings Dynamics in comprehensive workforce develop- the 1990s: Policy Issue Needs and ment systems. For more detailed information about Potential Uses of Nationally Ar-, some of the references mentioned, as chived Ul Wage Records by C. King. itDevelopment of federal/state dem- well as related publicationa, please see Washington, D.C.: Northeast/Mid- onstration projects to encourage the the following: west Institute, 1989. development of regional consortia The Vocational Education Study: The A Feasibility Study of the Use of Unem- that can promote the use of wage Final Report. National Institute of ployment Insurance Wage-Record record data. Education, Vocational Education Data As An Evaluation Tool for JTPA Study Publication 8, Washington, by J. Baj et al. Washington, D.C.: Conducting of systematic research at D.C.: U.S. Department of Education, National Commission for Employ- the federal level on the use of wage 1981. ment Policy, 1991. record data in policy studies and National Assessment of Vocational Ed- Schools and Labor Market Outcomes performance management systems, ucation Final Report, Volume I, by D. Crawford et al. National Cen- including program-specific and Summary of Findings and Recom- ter on the Educational Quality of the workforce system-oriented evalua- mendations by J. Wirt et al. Wash- Workforce. Philadelphia, PA:Univer- ington, D.C.: U. S. Department of sityof Pennsylvania, 1992. tion. Education, 1989. State Systems for Accountability in Vo- Some Comments on Using State Unemployment Insurance cational Education by M. Rahn et al. the Two Monographs Wage Records to Trace the Subse- National Center for Research in Vo- quent Labor Market Experiences of cational Education. Washington, Interest in strategies for establishing Vocational Education Program Leav- D.C.: U. S. Department of Educa- accountability in public programs and ers by D. Stevens. National Assess- tion, 1992. ment of Vocational Education. systems has never been more intense. The School-to-Work Transition of High Washington, D.C.: U.S. Department Despite bathers to coordination and in- School and Community College Vo- of Education, 1989. tegration, the federal government, the cational Program Competers: 1990- states and local communities are mak- Using State Unemployment Insurance 1992 by D.Stevens.National Center ing strong efforts in the 1990s to col- Wage Records to Construct on the Educational Quality of the laborate, coordinate and consolidate Measures of Secondary Vocational Workforce. Philadelphia, PA: Univer- workforce development programs, and Education Performance by D. sity of Pennsylvania, 1993. Stevens. Washington, D.C.: Office of to incorporate within that movement a State Capacity to Use Ul Wage Technology Assessment, U.S. Con- Records: The Vocational Education solid notion of how to judge process gress, 1989. and performance. Experience by L. Amico. Washing- Systematic Bias in Earnings Data De- ton, D.C.: National Governors' Asso- Whatever the general approaches rived from Unemployment Insurance ciation, 1993. taken in establishing accountability, the Weqe Records and Implications for adequacy, quality and accuracy of the Evaluating the Impact of Unemploy- information systems on which perfor- ment Insurance Policy on Earnings mance management judgments are to by P. Decker. Princeton, NJ: Mathe- be based are critical issues. These two matica Policy Research, 1989. reports are useful in understanding Policy Evaluation and Archived Wage some of these issues. They also force us Record Data: Limitations of Existing to consider the potential imperfections Data Sets by J. Bishop. Washington, in whatever direction we take in inte- D.C.: Northeast/Midwest institute, grating information production, and en- 1989. courage us to seek ever-better systems for making essential decisions.

1.1

Evaluation Forum Issue 11 Summer 1995 Evaluation Issues and Activities At the National Level ins can be made confidently about the ous controlled impact evaluations.This More About quality of policy and program out- conclusion is consistent with other cur- comes. rent definitions of the boundaries of Performance performance management systems Management Defining Accountability i.e. they tend to isolate net impact eval- uation from performance management. Reviewed in this article are two The authors' definition ofperfor- guides for developing performance mance accountability is at onceuseful: Characteristics of management systems: Getting Results: " a means of judging policies and pro- Accountability A Guide for Government Accountabili- grams by measuring their outcomes or Systems ty published in 1991, and A Guide to results against agreed-upon standards." The authors distinguish also be- Strategic Planning for Performance In this sense, accountability efforts pro- Measurement, Reporting and Improve- vide a framework for measuring out- tween policy accountability and pro- gram accountabilisy. Policy account- ment Systems produced in 1994. Devel- comes as well as processes, and ability is viewed as giving attention to oped by the Council of Governors' Pol- organizes this information "so that it can be used effectively by political the outcomes for whole groups or icy Advisors and the National Institute classes of people, not only for those for Literacy respectively, these guides leaders, policymakers and program managers." Accountability systems are who have participated in particular pro- offer assistance to policymakers, plan- looks at ners and oversight staff in developing to provide feedback of information to grams. Program accountability these important groups so that policies outcomes only for those served by a viable, useful accountability systems. program or group of programs, and at Both guides direct their suggestions to and programs can be adjusted when how programs are operating. Brizius state governments. needed. And they are to disseminate useful information to service providers, and Campbell describe usefully the key The Council of Governors' Guide service customers, and the public. elements of state performance account- ability systems, in the ideal: Authored by Jack A. Brizius and Developing Accountability A focus on outcomes. Michael D. Campbell, Getting Results Systems was funded by the U.S. Departmentof The use of a few selected indicators The authors make the usual observa- Health and Human Services, and in- to measure performance. volved an array of experts on perfor- tion that performance accountability mance management. The authors ex- differs from traditional assessments of The generation of data over time. "inputs" by focusing attention on plain that the guide answers a number Continuous reporting of outcomes. of basic questions: changes in people's lives. However, they do not neglect the importance of The use of performance information DJ What is a performance accountabili- studying program implementation as an to create incentives to service pro- ty system? integral part of performance manage- viders to improve outcomes. What is the political and policy con- ment. Nevertheless, they emphasize the The use of performance information text within which accountability sys- establishment of performance standards which they contend tie desired out- as a basis for developing sound tems must be implemented? comes to longer-term program goals. questions for evaluation purposes, What are the key components of a These standards, they say, must be real- and for conducting competent evalu- workable accountability system? istic estimates of expected outcomes ations. which have been jointly developed by What are the fundamental queries those judging performance and those The utilizatinn of performance infor- that must be answered in defining being held accountable. Some criteria mation in making policy and pro- such a system? for system 4evelopment are: gram management decisions, and improving policies and programs. IV What are the specific actions that The amount of resources used. need to be taken in implementing The baseline skill levels or problems The testing of policy premises. this kind of system? of those served. These benefits of performance ac- What are the problems and pitfalls countability, or performance manage- The intensity of a service or treat- in making comparisons and report- ment systems, are qualified by some ment. ing performance? limitations. Again the authors take the The power of the incentives used to position that such systems are not capa- How can accountability systems be produce change. ble of determining the ultimate impact institutionalized within state govern- of programs and policies on people. ment? It is to the authors' credit that per- Because impact evaluations are diffi- formance accountability systems, as The appendix of the guide suggests cult and expensive, they say, net impact they define them, are not the end-all in evaluations are "seldom used as man- a step-by-step progression toward com- judging policies or programs. Judging agement tools." Therefore perfor- pletion of a design for an accountability effectiveness, they say, requires rigor- system on the basis of which judgments 103 Evaluation Forum Issue 11 Sumtnir 1995 , Evaluation Issues and Activities At the National Level

mance management systemscannot states not todevelop formal standards The National Institute for definitively pronounce policies or pro- prior to the application of a policy or Literacy's Guide grams failures. The author's persistence the operation of a program a wiser in viewing evaluation as something process is to develop and test initial While the Institute's monograph is apart from performance management is standards against actual program per- directed toward developing accountable an unfortunate and possibly misguided formance and adjust these standards state adult literacy systems, particularly conclusion, since in general net impact based on experience with their use. the states that participated in the Acad- evaluations produce the most accurate Apart from premature use, performance emy on Performance Accountability information for both policy and man- standards have been greeted with held in Washington, D.C. in 1993, the agement use, and are not always expen- mixed reviews because they have often insights are widely applicaLle. The sive. been linked to funding decisions. Also, guide is to assist states in establishing statewide outcome-oriented perfor- More to the point, accountability local economic and political conditions mance measurement, reporting and im- systems are considered by the authors clearly affectperformance but are not considered realistically in the context provement systems, formally referred to to be limited by data quality and the as PMRISs. Their elements are generic. accuracy of outcome measures, prob- of uniformly-applied standards. lems that have a significant influence A PMRIS measures outcomes, re- on evaluation activities as well. The au- Some Conclusions ports this information to key actors, uti- thors also recognize that accountability The authors are well aware that ac- lizes it to continually improve state pol- systems can involve disincentives on countability systems can have short icies guiding programs and the quality the part of service providers to respond lives if their advocates are not careful of those programs. Its structural core is direetly to the needs of clients given in designing and using them, and in an interagency team composed of di- their situations and circumstances. This mediating traditional objections to per- rectors of relevant statewide programs is particularly true, they say, whenac- formance measurement. When account- The team is the vehicle for making sure countability systems are linked closely ability systems identify problems, there that outcome information is collected to the allocation of resources. will be natural pressures to dismantle and reported, that it addresses policy them. resource allocation, management and The Design of Systems service provider needs, and that coordi- A number of strategies are suggested nation across programs is maintained. The authors explain the design of for maintaining these systems. The cen- The authors outline a series of steps in performance accountability systems as tral theme is ensuring that key stake- answers to five key questions: constructing a PMRIS, which are pre- holders have access to and use account- sented in Figure 1. a Are the goals and objectives of the ability information: political leaders, In this scenario, a vision statement is policy or program stated clearly? policy staff, legislators, and legislative staff, program administrators, service a preferred future that explains the con- How can progress toward these providers, consumers of service, the ditions and life quality to be achieved goals be measured? press and the public. in the state. Benchmarks are measur- able goals for the state as a whole, How can incentives be accomodat- ed? How will the data be reported and ccpunta4doy-Essentials.. used? 1.Define vision and benchmarks. Apply quality standards to existing How can policy development and 2. Define policy outcomes. and new programs. program management be coordinat- 3. Identify performance measures for 5. Collect data & communicate results: ed to promote accountability? policy outcomes. Define the audiences. 4. Identify program outcomes, related Develop sample reports for differ- Realistic, measurable goal statements to policy outcomes. are an essential first step. Policy goals, ent audiences. 5. Identify programperformance in the guide, are essentially system- Negotiate data collection protocols. measures. Collect the data. wide goals. Program objectives are 5. Identify target populations, related Report the resultb. more specific statements with shorter to policy and program outcomes. I. Use information from PMRIS to im- timeframes. Developing indicators of 7. Compare existing service delivery prove program interventions. progress toward goals and objectives is patterns with service needs: Link inputs to processes. the most difficult step, according to Determine what program interven- Brizius and Campbell. Such indicators tions are necessary to achieve Identify constraints andenabling should include both process variables outcomes for target groups. influences. and outcome variables. Performance Determine whether existing pro- Plan for continuous improvement standards then rely on the selection of gram can achieve these out- 10. Use information from PMRIS to these indicators. The authors caution comes. adjust resources and policies.

104 Evaluation Forum Issue 11 Summer-1M Evaluation Issues and ActMtles At the National Level

vim consistent with the vision. Benchmarks tions of performance systems is the ex- Using Performance permit the tracking of progress toward tent to which they work in practice. the vision over time. A rigorous defini- Historical attempts to coordinate sepa- Standards in tion of vision and benchmarks, howev- rately-funded programs providing com- Establishing er, is not viewed as a necessary condi- parable interventions to comparable tion for defming policy outcomes client populations have had mixed re- Workforce System even though the latter need to be gener- views. The new view of performance Accountability ally consistent with these abstractions. management depends on such coordi- The authors are insistent, however, that nation. Relevant to this issue is the Na- There is a high level of agreement policy outcomes be quantifiable. tional Commission for Employment among workforce development experts Policy's 1994 position paper "Guiding that a key element in establishing ac- In logical sequence, defining pro- Principles for Change," by Anthony P. gram outcomes is a next step, consis- countability in new state-wide work- Carnevale, the Commission's chair. face preparation systems is the devel- tent with policy outcomes. These out- The paper explores some of the coordi- comes are defined by the authors as the opment ofpe;:formance standards nation problems sustained historically, consistent with system goals and de- statuses to be achieved by programs and makes the following recommenda- and their participants ifpolicy out- sired outcomes. Unfortunately we lack tions for national-level efforts to increase a history of experimentation to rely on comes are to be achieved. Performance coordination in the context of national measures are seen as both quantitative for insights regarding the appropriate- and state efforts to build accountable ness of setting standards for "systems" and qualitative indicators of such out- workforce preparation systems: comes. These measures are considered of programs. Even our exploration of the central element of the PMRIS. Per- A national dialogue is needed on the use of performance standards in formance measures are of three kinds, systemic reform of the national edu- separate major ongoing national pro- the authors say: results-oriented mea- cation and training systems. grams is receat. sures; measures ofprogram quality; The longest running experiment is in The consolidation of programs serv- and measures ofparticipant character- JTPA.,What we have learned about the ing similar target groups should be istics. These process and outcome mea- implications of using performance stan- sures are clearly the building blocksof endorsed. dards in JTPA should be enlightening any performance management and Two interim steps for enhancing co- regarding the tradeoffs we may experi- evaluation system. ordination should be implemented: ence in applying standards to state workforce development systems. Some Comments on Other 1) the establishment of a c.mtral Performance Management waiver authority outside the Execu- One of the most astute analyses of Approaches of the PMRIS tive Branch, and 2) efforts to make the utility of developing and applying basic program requirements compat- performance standards is Burt Bar- National-level conceptualizations of ible. now's chapter on "The Effects of Per- performance management systems are formance Standards on State and Local quite similar in their basic elements and Broad planning and oversight re- Programs," in Evaluating Welfare and meaning. They have given considerable sponsibilities at the federal, state and Training Programs edited by Charles direction to states at a time when states local levels should be promoted. Manski and Irwin Garfinkel in 1992. themselves are recognizing the need to The book began as separate but related integrate related programs and assure Information about "best practices" papers commissioned for a national the accountability of multi-program regarding planning and oversight conference on evaluation methodology systems. Most performance manage- functions and tasks should be made by its cosponsors, the Institute for Re- ment concepts include measures of pro- available to state and local adminis- search on Pover01 at the University of cess and outcome variables, even trators. Wisconsin-Madison, and the Assistant though the systems are outcome-driven. More flexibility should be allowed Secretary for Planning and Evaluation This moderates the pendulum swings of in the U.S. Department of Health and the past several decades in which either in the development of state-wide Human Services. It is important to con- process or outcome variables were the and local-level planning councils or sider Bamow's position on perfor- exclusive focus. This recognizes the boards. mance standards as we move closer to strong interrelationship between 1) pol- Funds for state/local capacity-build- national employment and training poli- icy and program implementation struc- ing should be provided. cy supported by performance standards, tures and processes, and 2) the shorter and to state-level workforce systems at- and longer-term outcomes and impacts Comprehensive cross-program man- tempting to include standards in their of related policies and programs and agemert information systems performance management subsystems. the 3ignificance of this relationship in should be developed. judging policies and programs. Some Background A UI wage record database, includ- The test of these well-conceived and ing all states, should be developed at The mandate to develop perfor- logically explained ideal-type construe- mance standards for major national the national level. vmmiNOMMIIMM2 1 05 Evaluation FonnnsIssue 11i Summer 1995 113 Evaluation Issues and Activities At the National Level

programs, as a major tool in the ac- programs that meet or exceed stan- ness (does the program achieve its pro- countability kit, began in the 1980s. It dards such as through more pro- grammatic goals for those it is intended now applies to JTPA, JOBS, and the gram flexibility or funding and to serve?). Food Stamp Employment and Training sanctioning them for failing to do Program. Performance standards were Performance management systems, to involve specific measures of desired so. Sanctions may be mild, such as as theoretical constructs, have tended to outcomes, tempered by lessons from simply an absence of praise, to quite involve three key "accountability" previous program evaluations regard- harsh, such as a loss of funds. functions: program review (against ing the actual level of success employ- Barnow wants us to distinguish plans), program monitoring (for com- ment and training programs were hav- among three different concepts in talk- pliance with requirements), and pro- ing in increasing the employment and ing about performance standards: per- gram evaluation (to judge the efficient earnings of different participant groups, formance management, the achieve- achievement of program goals). But all and in the case ofJOBs and Food ment of economic efficiency, and of these constructs emphasize the con- Stamps, in reducing participants' de- evaluation. tinuous feedback of evaluative infor- pendence on the welfare system. mation for continually improving pro- Performance management is con- grams, an emphasis missing till In the Manskii Garfinkel book, Bar- cerned with the achievement of an ar- recently in the evaluation research liter- now first defines Flea performance ray of program goals, Barnow says, in- ature. management, then outlines state and lo- cluding equity. It is an ongoing cal objectives and behavior vis-a vis oversight activity that provides con- The Need for Performance standards, and ends with his view of tinuing feedback to policymakers, ad- Management Systems bow performance is measured, rein- ministrators and planners about key Barnow's rationale for developing forced and rewardel. In the process we program processes and shorter-term formal performance management sys- see some cautions emerging. program outcomes. Performance stan- tems is significant. Briefly, such sys- dards, within performance manage- tems 1) identify poor performers and The Performance Management ment, do not include impact measures, Concept Barnow states, and are less intrusive therefore permit corrections to be made, 2) locate high performers and re- than evaluations in terms of disturbing Barnow defines performance man- ward them, thus helping to defme "best program operation. However, perfor- agement as "a system whereby pro- practices" and providing evidence that mance management systems lack a grams are systematically judged against there is organizational profit in apply- clear-cut "bottom line" (such as prod- specific objectives." The following are ing them, and 3) maintain efforts in the ucts sold) similar to variables that can the main interrelated "parts" within the direction of accomplishing the goals performance management "system:" be monitored in private sector activi- ties. originally intended for the program. Pefformance measures: indicators Identifying the ingredients in com- of the performance of the entities Evaluation, Barnow proposes, fo- mendable performance also providesa being judged. cuses on program impact through sin- practical basis for replicating those at- gle snapshots in time, using quasi-ex- tributes. Furthermore, sustaining.con- Petformance standards: specific perimental or experimental research sistency with program goals reduces standards associated with each mea- designs. Economic efficiency remains natural tendencies to redirect program sure or indicator. undefined, and its relationship to per- activities toward other agendas, suchas a A standard-setting method: the formance management and evaluation those responsive to political, culturalor is left unclear. organizational pressures. method for setting standards may be based on statutory regulations, man- These distinctions are not the tradi- Important Kinds of Performance agerial goals, performance levels in tional ones. The concept of "perfor- Measures to Consider other programs, or on other factors mance management system" often cov- ers both program monitoring and Barnow identifies a range of vari- or on arbitrary decisions. The program evaluation, and involves some ables for which specific measures most useful, and perhaps the fairest, effort to use measures of economic effi- should or could be developed within Barnow claims, is a rational method ciency in the process. Program evalua- performance management systems: that is understood in the programs tion typically covers judgments of pro- being judged, even though standards gram organization and operation, Gross Outcomes can be set to take account of factors program outcomes, net program im- Gross outcomes are measures of a that vary with the characteristics of pact, and the tradeoff between costs program participant's status at the time the program being judged. and benefits. In this sense it involvesa he or she exits from a program,or study of both program efficiency (do a A rewards and sanctions system: shortly thereafter, vis-a-vis certain de- the program's outcomes warrant its sired outcomes such a system is a way of benefiting educational, skill various costs?) and program effective- training, employment, welfare reduc-

10S 114 Evaluation Forum Issue 11 Summor 1995 Evaluation issues and Activities At the National Level

tion, etc. Exit status could mean at the education, preemployment, job-specif- properly combined with outcomes mea- completion of the services assigned, or ic), which are difficult to quantify. The sures, it may "promote underinvesting in participants and serving participants simply at the point participants leave advantage of these measures, however, who require less training." Omitting the program for various identified and is that net earnings gains, can be con- unkmown reasons. Barnow says that sidereal to reflect the market value of costs from the performance manage- ment system avoids a bias toward most of the outcomes measured in cur- varying kinds of training. rent national E&T programs are gross shorter and less intensive programs. The problem is that different earn- The tradeoffs are difficult to reconcile. outcomes, because they do not neces- ings gain measurement strategies may sarily reflect program gains or losses. produce disparities in earnings gain es- Combining Performance Outcomes such as completion of ed- timates, leading to a mistrust of this Measures into an ucation, completion of skills training, version of program performance. Also, Overall Index using the determinadon of earnings placement in a job, and increased earn- Barnow proposes that, in theory at ings in employment are standard pro- gains as the most prominent way to If judge a program's achievements re- least, it is possible to develop a single gram outcomes in E&T programs. overall measure of the economic gains we were not able to compare thequali- quires quasi-experimental or experi- mental net impact evaluations, which from E&T programs, such as "the ty of educational, skills taining or em- present value of all earnings gains re- ployment outcomes for program partic- are not always appropriate or possible. Furthermore, Barnow cautions that pri- sulting from the program, or the ipants with comparable people who present value of earnings gains plus were not exposed to the program, we mary reliance on a quantitative eco- nomic measure of performance runs the welfare savings." While useful, this di- would designate these outcomes as the rects attention to a single indicator of meaning risk of precluding a consideration of program's gross outcomes performance, ignoring other outcomes. that we are unable to label these as bet- significant program outcomes that are ter (a gain) or worse (a loss) than the less tangible but may be as important or To avoid excluding other program situation similar people would experi- more so. goals, such as service equity, perfor- ence in the absence of the program.If mance measures can be developed we were able to make the comparison Inputs and Process Measures which hold programs to more of their indicated, we could then calculate the Barnow views input measures, such promises -e- for example, measures of program's net outcomes (by subtracting as the characteristics of participants, as the participation levels of client groups any loss or adding any gain),These net part of performance management sys- of special interest which can be in- outcomes are usually referred to as the tems. Process variables describing how corporated within performance stan- program's net impact. service delivery is organized, what pol- dards (as minimum levels of participa- icies and practices are involved at each tion.) Outcome measures can be The main thrust of Barnow's argu- tracked for different groups, and a dif- ment is that gross outcomes are rela- step in the service pathway, how many ferent set of minimum standards devel- tively easy to collect and analyze, and and what kinds of clients experience oped for each. Employment retention therefore constitute an efficient part of what steps, and what the "culture" of may be a higher priority in some pro- any performance system. Hequalifies the program is like (staff attitudes and behaviors, for example) are clearly in- grams than earnings gains. The reduc- this by pointing out that performance tion of welfare rolls may be more im- standards that rely on gross outcome cluded in a competent performance portant than any other outcome. measures must adjust for differing cli- management system. As an expert in designing and conducting net impact Performance measures, and ultimate ent profiles. Otherwise programs will standards, Barnow suggests, can be de- be sanctioned for not meeting perfor- evaluations, it is understandable and veloped so as to permit different expec- mance standards, even though the char- important that Barnow suggests that ex- tations for program performance. acteristics of certain subgroups within clusive reliance on inputs and process the client population make it unrealistic variables is a mistake. The challenge in the use of multiple to hold these subgroups' gross out- measures, he says, is in linking a clus- comes hostage to the same levelsof Measures of Costs ter of measures together such that an performance as one does the others. Performance management systems overall minimum level of performance must, Barnow believes, consider the can be established, and a reward/sanc- Net Outputs use being made of resources. Identify- tion system developed. A simple meth- Barnow is referring here to indica- ing the total dollars spent in achieving od is to count all performance measures equally and sum the scores. But Bar- tors of gains, such as in education, key outcomes is one approach. Or costs skills, work experience, wages, and job can be reported for each major out- now warns us that adding scores may be undesirable. Some measures may be retention. He claims that developing come, such as cost per entered employ- such measures will require a compari- ment. Separate cost measures give at- considered more important than others. Some may involve more variation than son of different kinds of programinter- tention to unit costs, an approach others, and therefore have to be weight- ventions, or treatments, such as differ- Barnow thinks is very useful. But he ed differently. An alternative is to ent types of skills training (basic cautions that if the cost measure is not

107 Issue 11o Summar 1925 EvaluationFOIVal 115 aaaaaa,

Evaluation Issues and Activities At the National Level

group performance measures and re- more difficult to deal with. Barnow for creaming, JTPA service deliveryar- quire programs to meet a certain num- says "if the gains in earnings accruing eas (SDAs) which gave priority to ber of standards in each group. to participants are at the expense of goals other than achieving high scores other workers, then not all the private on performance standards were able to A Rewards and Sanctions gains are social gains." System resist biasing the program enrollment Barnow says that giving substitution process. The most prominent of these Another important component of a effects due consideration involves ad- other objectives was the desire to serve performance management system, re- dressing two types of equity issues: certain types of clients, and to respond ward/sanction systems, also has its to the needs and interests of employers. complexities. The least complicated 1. The value placed on successfill out- Barnow believes that creaming is de- strategy is to offer praise for successful comes for particular subgroups of creasing with the increased use of state compliance through certificates or an clients. Earnings gains realized by adjustment models for developing stan- honor roll. A more energizing strategy certain kinds of participants may be dards, and with recent changes in both is to provide additional funding, ac- viewed as more valuable from a so- performance measures and adjustmant companied by more flexibility in the cietal perspective than the same models, and the dropping of cost stan- use of funds. A monetary reward is gains experienced by other clients. dards. more consistent with private sector sys- tems. 2. The value placed on a broadvs.a But clearly there are questions re- more narrow service delivery re- maining. Barnow sees several worth Inadequate performers, Barnow sug- policy attention: gests, could lose the privilege of oper- sponse. Programs serving a large ating a program, or the federal funds number of eligible people but with Whether the JTPA performance could be reduced. Or program adminis- limited services may be considered standards system promotes the maxi- trators could be required to accept re- more valuable than programs serv- mization of earnings and employ- medial technical assistance, as in ITPA. ing a smaller number but with a ment gains and reductions in welfare Tying the level of rewards to the level more intensive mix and sequence of transfers. at which standards are exceeded is an services. 'Whether it achieves the goal of option already mandated in some feder- holding SDAs harmless for serving al legislation. However, fundingre- These issues complicate the devel- wards and penalties may actually be opment of performance standards, since individuals with different character- counterproductive, Barnow says, if they involve policy decisions laced istics. more funding leads to new responsibili- with value judgments. Nevertheless, Barnow recommends that such issues Whether it should lead to indiffer- ties or declining funding denies servic- ence on the part of SDAs regarding es to a burgeoning client population. be studied in developing performance the kinds of people served. The Achilles heel here is that a lack of management systems. consideration of the characteristics of Several sources of' bias remain in the The JTPA Performance participants may encourage program JTPA performance standards adjust- Standards System administrators to "cream" for those ment models, Barnow says: 1) bias in- most likely to succeed in the labor mar- The author sorts out the advantages troduced by grouping data; 2) bias due ket. and disadvantages of an established to an incomplete set of' explanatory performance management system that variables, and 3) bias as a result of the Performance Standards and has been fine-tuned over the 1990s focus on participants rather thanon the Measuring Economic Efficiency the system for JTPA. This system has eligible population. Some states have According to the author, judginga been successful, Barnow thinks, in addressed equity issues by adding stan- dards for the participation of particular program's "return on investment" isa terms of its practicality, the level of ac- key objective at all levels of govern- ceptance by state and local stakehold- groups, others for outcomes for such ment. In discussing ways to address ers, and the richness of its database. groups. But Barnow reminds us that economic efficiency in performance The best empirical evidence to date each new variable added to the models management systems, Barnow focuses suggests, however, that JTPA perfor- requires additional data collection, and on two issues: reductions in transfer mance standards have led to efforts to as models become more complex they payments and substitution effects. Cost/ enroll those clients defined as easiest to become more difficult to use. Con- place in jobs. benefit analyses have tended to treata sideration should be given, hesays, to 1) approaches that take into account particular loss to clients as a particular However, he also points to an evalu- gain to taxpayers that is, reduced ation of JTPA by SRI in 1988, spon- important participant characteristics not welfare transfers equal reduced taxes sored by the National Commission for included in the models, and to 2) avoid- ing basing reward/sanction systems for others. This tradeoff may sound Employment Policy, which indicated solely on regression equations. simplistic but it is implicit in most E&T that although the performancemanage- legislation. The substitution problem is ment syatem created some incentives

.1comniMIIMMIMMIMIMEC 108 ,==011 Evaluation Forum Issue 11 Summar 1995 116 _

Evaluation Issues and Activities At the National Level

Performance Standards Systems efitsof providing program direction Performance in Other Programs and accountability." Performance man- agement systems offer our best chance, Management The JTPA system for Title II is he believes, for dealing with conflicting Systems: Evaluation clearly the most highly developed. Title program objectives, and forcountering III uses only one standard, entered em- the tendency of bureaucracies to act in Principles and ployment rate, and an optional wage a manner inconsistent with theachieve- Standards standard. Title IV programs also have ment of economic efficiency. performance standards systems, as well Analyses of performance manage- as the Employment Service and the System Performance ment concepts and systems have fre- Food Stamp E&T Program. The JOB quently failed to address the role of Corps includes standards for the perfor- The trend in the mid-1990s is mov- evaluation research within these Sys- mance of private sector center opera- ing inevitably in the direction of devel- tems. Sometimes evaluation is treated tors, with sanctions involving the loss oping performance measures and stan- of a franchise. Standards are being de- dards for comprehensive workforce as an independent add-on oversight function, rather than an ongoing inte- veloped for the JOBS program as well. preparation systems at the state and lo- gral element of performance manage- However, JOBS differs from JTPA in cal level. Can we apply Barnow's is- being a state rather than a local pro- sues, cautions and recommendations to ment. Yet the professional stature and perceived utility of applied social re- gram, and in being jointly fundedby these systems? Undoubtedly. The per- search in producing valid information federal and state governments, which formance management concept Barnow for "managing performance" in social affects the development of its perfor- defines fits the accountability needs of programs has grown enormously over mance management system. systems as well as programs. In this concept, performance measures and the past three decades. With the advent JOBS administrators have been con- standards are to be based on goals, ob- of larger-scaIe workforce development cerned that standards may lead to jectives and a set of outcomes consis- systems, the failure to define a viable creaming, and that the gross outcomes tent with those goals. This is true for role for evaluation research within per- on which the standards rely are not programs, true for systems of pro- formance management systems is an highly correlated with net program im- grams. enigma L: sorts, and represents a pacts. But Barnow proposes that these missed opportunity. concerns may be more reflective of a Most of the kinds of performance This missing linkage is perhaps due different service system philosophy measures Barnow identifies as partof than realistic anxiety about the influ- program-specific performance manage- most to the perception that evaluations are one-time, more expensive snap- ence of performance standards. ment systems are equally suited to sys- tem analysis. The exception is that shots of programs, and take a longer Barnow recommends that more data since there is no "comparison group" time than other strategies to yield use- be collected on participants and their for systems, net nnpact evaluations are ful results. There 1r even the view that outcomes, that a rewards/sanction sys- not appropriate. However, cross-state the results of th,: Inure competent eval- tem be developed only gradually, and comparisons may be possible, and net uations are so well-qualified that they that administrators at the federal level impact studies of consolidated pro- lose their utility. resist basing performance standards on grams within the system may be useful. This perception ignores the role of state-specific programs he suggests Longitudinal surveys that gather pro- process studies that look at ongoing that they wait for the results of a na- cess and gross outcome information tional evaluation before setting stan- program implementation processes, and can also contribute to a knowledgeof longitudinal surveys which track cli- dards. the system's overall economic efficien- ents, services and results over time. cy and effectiveness. Conclusions in Barnow's They ignore the simple institution of Analysis Barnow's exploration of issues in- formal feedback expectations and volved in developing performance stan- mechanisms within evaluation research At the end of his chapter there is an dards, and his recommendation to col- contracts, which ensure that evaluation admonition to readers that none of the lect an adequate database prior to such findings are immediately absorbed into existing performance management sys- development, are important insights for policy and planning decisionmaking. tems are perfect, or even ideal. But the new coordinated workforce devel- They also ignore the basic need in all there is also optimism. The author is opment systems. Although the author's programs for genuinely accurate infor- convinced we can develop reasonable chapter does not address the role of mation, as opposed to information gen- systems that are useful. The basic ques- evaluation research, per se, we turn our erated from more subjective sources. In tion he poses is whdther the risks of attention to that subject in the next arti- this respect, qualfications to evaluation providing inappropriate incentives due cle., research findings are required if what is to imperfect models "outweigh the ben- learned about programs is to have any ring of validity about it.

1111... 109 Evaluation Forum Issue 11 Summer 1995 117 Evaluation Issues and Activftiesa At the National Level

Evaluation Research Goals, evolutionary process. The Committee Principles and Standards The overriding purpose of evalua- also recognizes that the general princi- tion is "to provide the best possible in- One perspective on the role of eval- ples and standards of any era mustre- formation bearing on the value of what- uation research is reflected in a 1994 main sensitive to the substantive fields ever is being evaluated." This is in the publication by Sage, The Program in which they are being applied. And context of Sanders' definition of evalu- Evaluation Standards by James R. finally, the Conunittee sees evaluation ation as "the systematic investigation of Sanders. Sanders is the chair of the standards as guides, not as binding the worth or merit of a program." Fig- American Evaluation Association's rules. ure 1 provides an overview of the Task Joint Committee for Educational Eval- Force's principles. Figure 2 is The Evaluator's Major Tasks an over- uation. Although the book focuses sub- view of the 1994 principles andstan- stantively on the education field, it is The Committee believes that appli- dards which are given more substance offered as a much more general contri- cation of the standards must be judged in Sander's book. bution to the understanding of evalua- in the context of key tasks evaluators tion research, and is meant to apply perform: implications of the AEA's equally well to social policies andpro- Principles andStandards grams in other areas, such as employ- Deciding whether to evaluate. ment and training. Setting thresholds regarding the Defining the evaluation problem. background and behavior of evaluators The effort to develop principles and Designing the evaluation. is much like setting performancestan- standards guiding evaluation research dards for programs. It clarifies roles practice goes back to the early 1970s Collecting information. and responsibilities. However,no in- when twelve national professionalas- Analyzing information. centive/sanction system accompanies sociations began to explore evaluation the former, except the ability ofcus- issues. These associations discovered, Ile Reporting the e% aluation. tomers to deny contracts to evaluators not to their total surprise, that there Budgeting the evaluation. they do not feel fit the standards. This were serious, fundamental differences requires customers to be much better in their viewpoints. The Committee Contracting for the evaluation. informed about the meaning and value was formed to resolve such differences, of evaluation research than they Managing the evaluation. are which resulted in Standards for Evalu- currently. These principles and stan- ations of Educational Programs, Staffing the evaluation. dards should assist customers in signifi- Projects and Materials, published by cant ways to be more sophisticated in McGraw-Hill in 1981. Through apro- Developing evaluation policies. developing RFPs for research, and for gressive series of meetings and theso- The applicability and/or importance of making sound selectionsamong avail- licitation of ideas from a large number a given standard, given these tasks, is able researchers for periodicprocess, of experts, the Committee laterre- considered to vary with the nature ofa outcome and net impact studies and for viewed and updated the 1981 effort, given evaluation. ongoing evaluation activities suchas and reached a consensus on evaluation longitudinal surveys. research principles and standardsas of Evaluation Principles the 1990s, which are contained in the The principles and standards also Sanders book. There are a number ofways to look provide insights about the basis for at principles and standards. A 1992re- quality assurance and continuouspro- The intent of this standards-setting port from the American Evaluation As- gram improvement. As important goals process was to establish a professional sociation's Task Force on Guiding of performance managementsystems, guide for evaluation practice that Principles for Evaluators givesus one quality assurance and continouspro- would also educate evaluation custom- viewpoint. The goals for evaluation gram improvement must necessarily ers and the public about the role of that underly these principlesare: 1) the depend on the nature of the information evaluation research. The 1994 stan- improvement of "products, personnel, about programs, and their environ- dards represent the official position of programs, organizations, governments, ments, that is generated from a range of the American Evaluation Association consumers, and the public"; 2) more in- sources involving qualitative and (AEA). formed decisionmaking andmore en- quantitative information, objective and But the Joint Committee is keen to lightened change efforts; 3) the actual subjective data through ongoing or remind readers and users that standard- generation of change; 4) theempower- periodic data collection and anal---'s. setting is an ongoing process. Research ment of stakeholders through engaging The development of performaneestan- practice and the professional knowl- and investing them in the evaluation dards is highly dependenton the quali- edge base are constantly undergoing process; and 5) the incorporation of ty of this information. The AEA evalu- change, and such changes must be new insights trom research into public ation standards can, in thatsense, make con- a significant contribution. tinually reviewed and revised withinan policy development, planning and oversight.

110 11 s Evaluation Forum Issue 11 Summer 1995 -

Evaluation Issues and Activities At the National Level

. _ . Figurel . GCtiding Prineiple.S . . . data-based inquiries about whatever is being Systematic inquiryEvaluators are expected to conduct systematic, evaluated: Should adhere to the highest scientific stan- Should clarify the values, assumptions, theo- ries, methods, results and analyses thatsignifi- dards so as to increase information accuracy cantly affect the interpretation of evaluation and credibility. Should communicate their approaches and findings. methods accurately and honestly, arid in suffi- cient detail to allow others to understand,inter- pret and critique their work.

Evaluators are expected to assure stakeholdersof competent performance: Competence Should decline to conduct evaluations that fall a Should possess theeducation, abilities, skills and experience appropriate to the evaluation substantially outside those limits. tasks to be performed. Should continually seek to maintain and im- Should practice within the limits of their profes- prove their competencies. sional training and competence.

and integrity of the entire evaluation process: Integrity and Evaluators are expected to ensure the honesty that might pose a significant conflict of interest Honesty Should negotiate honestly with clients and stakeholders concerning the costs, tasks, with their role as evaluator. methodological limitations, scope of results Should not misrepresent procedures, data or and use of data resulting from an evaluation. findings. Should seek to determine their own, their cli- Should determine that if certain activities ents' and stakeholders' interests concerning the seem likely to produce misleadingevaluative in- conduct and oytcomes of an evaluation, formation or conclusions, these concerns are Should disclose any roles or relationships they communicated to the customer. have concerning whatever is being evaluated dignity and self-worth of the respondents, pro- Respect for PeopleEvaluators are expected to respect the security, __ gram participants, clieuts and otherstakeholders with whom they interact: Should abide by current professional ethics Should attempt to foster the social equity of and standards regarding risks, harms, and bur- the evaluation, so that those who give to the dens that might be engendered in those partici- evaluation can receive some benefits in return. pating in the evaluation regarding informed con- s Should identify and respectdifferences sent and informing participants about the scope among participants, such as in theirculture, and limits of confidentiality, religion, gender, disability, age, sexual orienta- Should conduct evaluations and communicate tion and ethnicity, and to be sensitive to the po- their results in a way that respects stakehold- tential implications of these differences when ers' dignity and self- respect. planning, conducting, analyzing and reporting on evaluations.

the diversity of interests and values Responsibilities Evaluators are expected to articulate and take into account that may be related to the general and publicwelfare: Re: the General stakeholders to have Public Should consider including in the planning and a Should allow all relevant reporting of evaluations the perspectives and access to evaluative information, andactively interests of the full range of stakeholders. disseminate it to them, tailoring this information to the interests of different stakeholders. Should consider not only the immediate opera- tions and outcomes of the program being evalu- Should maintain a balance between client ated but also the broad assumptions, implica- needs and other needs. tions and side-effects of that program. Should protect the public interest and public good.

111 Evaluation Forum Issue 11 Summar 1995 119 . .

Evaluation Issues and Activities At the National Level

Utility Evaluations should serve the information needsof Intended users and stakeholders. The users and stakeholders, i.e., those involved should be made explicit, so that the basis for in and/or affected by evaluation activities, judgment is clear. should be identified and their needs deter- mined. Evaluation reports should clearly describe the program being evaluated: its context, its pur- Evaluators should be credible that is, they poses, its methodology, and the findings of the should be both trustworthy and competent to evaluation. pedorm evaluations, and so perceived byusers and stakeholders. a Significant interim findings and evaluation re- ports should be disseminated to users and The information collected and analyzed should stakeholders to encourage timely use ofemerg- address key questions about organizations and/ ing information. or programs of interest and concern to users and stakeholders. Evaluations should be planned, conducted and reported in ways that promote follow-up byus- The perspectives, processes and rationalesun- ers and stakeholders. derlying the interpretation of evaluation findings

Feasibility Evaluations should be realistic, prudent, diplomaticand frugal a Evaluation processes should be practical, to re- potential misapplication of evaluation findings. duce disruption of the programs being evaluated. Evaluations should be efficient, and produce in- Evaluations should be planned and conducted formation of sufficient value and use that there- with respect for the positions of various interest sources expended on the research can be justi- groups, to obtain their cooperation, and to reduce fied.

Propriety Evaluations should be conducted legally and ethically,considering tke welfare of those in- volved in the evaluations and affected by theresults. Evaluations should be designed to assist subjects of research actrvities, and the need to organizations and programs to address and protect these values. effectively serve the needs of all the partici- pants expected to benefit from organizational Evaluations should recognize and record fully and fairly the strengths and weaknesses of activities and the receipt of program seNices. the program being evaluated, so that problem What is to be accomplished, by whom, and areas can be addressed and strengths built when should be agreed to in writing by thepar- upon. ties involved in an evaluation, to commit them The formal parties involved in evaluations to the conditions of the agreement or to a for- should make certain that the set of research mal renegotiation of the agreement if that be- comes important. findings and limitations to the findings are avail- able to those affected by the evaluation, andto Evaluations should be designed and conducted any others with expressed legal rights to this in- with a respect for the rights and welfare of the formation. Accuracy Evaluations should reveal and convey accurate(unbiased) information about the key features of the organizations or programs being evaluated. The organization or program being evaluated analysis methods) that guide evaluations should be described clearly and precisely. should be appropriate to the research question The setting and environment of theprogram selected for study. should be examined In sufficient detail to identi- The evaluation process should be monitored fy those factors most likely to affect the organi- and assessed throughout the evaluation. zation or program. The conclusions flowing from an evaluation The adequacy of the sources of information should be made explicit and should be justified; used in an evaluation should be determined. and evaluation reports should honestly reflect Data collection methods should be selected and the results of the evaluation and any qualifica- applied such that reliable and valid information tions to its findings. is the basis for interpretations of the findings. The strengths and weaknesses of the evalua- The research design (experimental ornonex- tion design, methodology, and researchpro- perimental) including methodological decisions cess should be assessed at the completion of a (about measurement, sampling methods, and study. quantitative and qualitative data collection and

112 ammo= Evaluation Forum Issue 11 Summer 1995 120 Evaluation Issues and Activities At the National Level

Mohr, L Impact Analysis for Program Stufflebeam, D. Metaevaluation. Evaluation. Thousand Oaks, CA: Kalamazoo, MI: The Evaluation Cen- For useful references on evaluation Sage Publications, 1992. ter, Western Michigan University, 1988. research, please see: Patton, M. Qualitative Evaluation and Alin, M. et al. Using Evaluation: Does Research Methods. Newbury Park, Worthen, B. and K. White. Evaluating Evaluation Make a Difference? New- CA: Sago Publications, 1990. Educational and Social Programs: Guidelines for Proposal Review, On- bury Park, CA: Sage Publications, Utilization-Focused Evaluation. 1979. Site Evaluation, Evaluation Con- Beverly Hills, CA: Sage Publications, tracts, and Technical Assistance. Cook, T. and C. Reichardt (eds.) Qua li- 1986. Boston, MA: Kluwer Academic Pub- tative and Quantitative Methods in Rossi, P. and H. Freeman. Evaluation: lishers, 1987. Evaluation Research. Newbury Park, A Systematic Approach. Thousand CA: Sage Publications, 1979. Oaks, CA: Sage Publications, 1991. Three useful journals are Evaluation and Program Planning, New Directions Herman, 1. Program Evaluation Kit. Shadish, W. et al. Foundations of Pro- for Program Evaluation, and Evaluation Nine volumes on different aspects of gram Evaluation: Theories of Prac- Practice. evaluation research. Thousand tice. Newbury Park, CA: Sage Publi- Oaks, CA: Sage Publications, 1987. cations, 1991. Levin, H. Cost-Effectiveness: A Primer. Strauss, A. QualitatWe Analysis for So- Thousand Oaks, CA: Sage Publica- cial Scientists. New York, NY: Cam- tions, 1983. bridge University Press, 1987.

of objective evaluative information Editorddstiftif6i To assure high quality throughout the various aspects of individual pro- from various sources. This series of articles yields common grams and systems of programs. The inclusion of quality assurance, elements of performance management To make assessments of systems continual program improvement, perfor- systems for judging programs, and and programs using performance mance standards and evaluation re- groups of related programs in work- measures and standards. search within the concept of perfor- force development systems. Quality as- mance management recognizes that a surance, continuous program improve- To evaluate systems and programs range of strategies for judging systems ment, performance measures and regarding dimensions of their opera- and programs are, and must be, en- standards, and evaluation research are tion, the characteristics of their client compassed in the effective manage- key concepts. groups, the nature of their services ment of programs If indeed the pur- and service delivery systems, their Contrary to some of the distinctions pose of accountability systems and the outcomes for participants and others goals of management are to maintain a made by the authors represented in thls expected to benefit from them, and satisfactory level of performance or en- section, it would seem logical to view their net impacts. performance management (or account- hance it. None of these strategies ability) systems as the holistic concept , To continuously improve systems should be cordoned off in a territory of and attribute to these systems a num- and programs through the timely use its own. ber of goals:

113 Evaluation Forum Issue 11 Summar 1 go5 14 121 s ,nrst

Evaluation Issues and Activities At the State And Local Level r.

At the State and LocalLevel

. . . Edit.dirrat tn!r,octeiel(ort. programs. But the changes recom- and effectiveness of woridorce mended go significantly further than development. The interest in program coordination earlier concepts in knitting diverse edu- and service system Integration Is not cational, training and employment el- Utilization of labor market and re- new. It even appears to be politically forts into a single "eystem,* particularly lated information to Identify avail- cyclical. In the 1950s many communi- at the state level with strong man- able jobs and oreeupations in de- ties created local social service coun- dates regarding short end longer-term mand, and the educational and cils which brought program organiza- strategic planning and the use of a training seMces available to de- tions together around common common set of measures for monitoring velop skills. community goals. The missing link was and evaluating the progress of thesys- a commitment to strategic planning in Development of methods for co- tern in meeting planning expectations. ordinating programs so that they the formal sense, and to oversightac- Secretary Reich's Reemployment Act of tivities such as monitoring compliance provide maximum coverage of 1994 was an Important expression of the workforce, ensure equitable with formal plans and judging overall the now philosophy. performance. access to all population sub- Therefore, as a prelude to this series groups, and enhance the delivery In the 1970s there was another of artioles on earlier and contemporary of services. wave of coordination efforts, under the coordination efforts, we summarize the Review and approval of budgets title L'human services integration.* This major features of the Act in the list be- was the era of the federal Joint Funding low. for federal programs, and the de- Simplification Act, which offered a Strat- velopment of recommendations egy for packaging federal funds for bet- s The establishment of state human regarding the budgets of other ter coordinated, even though not yet in- resource investment councils or their programs operating within the tegrated, human service activities. Most equivalent. These councils are to be one-stop serViCe areas. of the 1970 concepts remained policy responsible for developing a com- Monitoring of the implementation goals that were only sparsely tested prehensive state-wide workforce de- of the strategic plan and the judg- velopment plan and system, and for and implemented, and continuing Inter- ment of the overall performance overseeing the implementation of est group pressures yielded an increas- of the local workforce preparation this plan and the system's perfor- ingly fragmented social service and system. workforce preparation system. mance. The establishment of one-stopca- The Reagan Administration advocat- The identification of substateone- reer centers. ed what appeared to be a new version stop service areas. The provision of a menu of of coordination through block grantsto is The development of a common common opera- services through the centers. states. The consolidation of funding tional agreement among all the parties streams was viewed by some experts involved in the one-stop centers. The participation of federal employ- as the only way to achieve true integra- ment and training programs in the tion of existing program territories. But The creation of local workforce in- centers. the goal at this time was only second- vestment boards within the one-stop arily to pull social programs together center areas. a The development of a quality assur- more effectively. It was mainly a way to Composition: anoe and performance management satisfy political commitments to states' system. rights and to cut the federal budget, Representatives of private sector since the down side of these block employers. Private sector em- The following series of articles is in- grants was increased state responsibili- ployers are to be a majority on tended to shed some light on the histo- ty coupled with reduced funding. these boards. ry of coordination, and on current ef- forts to pull an unwieldy cluster of The Clinton Administration acknowl- Representatives of organized la- consolentiously-designed educational, edged that multiple program organiza- bor and community-based organi- training and employment efforts togeth- tions had now developed a naturalre- zations. er into a more comprehensive, coher- sistance to integration. Separate Representatives of educational ent, market-driven, user-oriented sys- funding streams for programs and institutions. tem. projects understandably reinforced the bureaucratization of these organization- Community leaders. The first article looks at some of the A al empires. Concern about the impact history of coordination The second fo- Functions: cuses on the efforts of some states to of this development on costs, andon coordinate their employment and train- the results of service efforts on custom- 4 Provision of policy guidance. ers, led to another coordination move- ing programs within state-widesys- ment In the 1990s. Development of a strategic plan for tems. The third and foutth review im- workforce development programs portant conceptual and pragmatic The reforms characteristic of the consistent with the state plan. reports supporting state coordination present movement have continued to efforts. The last explores NGA's project preclude the comingling of program identification of measurable ob- to enhance accountability in new state funds and the actual consolidation of jectives for improving the quality workforce development systems.

114 VW, Evaluation Forum issuer 11 Summar 1996 _122 Evaluation Issues and Activities At the State and Local Level

1111111 Clinton Administration recommended a Coordinated Delivery Systems A Brief, Selected consolidation of six programs for dislo- A number of strategies in the 1960s History of cated workers under a new governance held promift for integrating services. arrangement, with coordinated services However, planning inflexibilities and Coordination delivered through one-stop multi-ser- unrealistic goals reduced their effec- Efforts vice centers at the local level. Over- tiveness, according to Johnston. In sight would be the responsibility of lo- 1972, the Comprehensive Manpower Tied as we tend to be to our own era cal workforce boards under the Program was introduced. It consisted in employment and training, we often guidance of state Human Resource In- of a series of demonstration projects assume that the initiatives, or "re- vestment Councils or their look-alikes. testing the coordination of services forms," indigenous to the life or our In the winter of 1994/1995, the new across political jurisdictions and was own careers must be new and different. Congressional leadership suggested constructed on earlier coordination ef- Within this perspective these reforms other alternatives, including a 1990s' forts. But service contracting remained hold our attention and energize us to consideration of the block grant con- fragmented, and establishing linkages experiment with and adapt useful ideas cept which surfaced earlier in the Nix- among existing programs proved to be to the needs of our time. But it is also on and Reagan years. difficult. However, a revision of the important for us to understand and program appeared to be superior to pre- learn from the history that precedes The Commission's Perspective vious attempts in terms of the decen- "our time." on the History of Coordin, tion tralization and decategorization de- A report yet in draft form at the time Johnston sees the evolution of na- sired, short of new comprehensive this article was written is a unique re- tional employment and training initia- employment and training legislation. source in this respect. It is the National tives between the Great Depression and Commission for Employment Policy's the 1990s as federal experiments that Administrative Solutions monograph tentatively titled "Histori- "evolved from a collage of individual Although administrative reforms cal Efforts to Improve Coordination in contracts and grants administered by a were less attractive and dramatic to Employment and Training Programs" number of federal agencies to broaden planners than comprehensive planning and is authored by the Commission's programs aimed at specific target and service delivery coordination, Nix- acting director, Janet Johnston. This groups." The major question this evolu- on's A-95 process was expected to de- excellent in-progxess report tells us tion raises for the author is what the centralize programs, simplify federal much we need to know. role of the federal government should grant processes, standardize administra- be vis-a-vis state and local government tive rules, and integrate the administra- What la The Problem? bodies. tion of grants to state and local govern- The report raises the issue of the This question leads Johnston to ana- ments thereby providing a new U.S. General Accounting Office's se- lyze the history of coordination within environment for coordination. This ries of 1994 reports on problems asso- four issue areas: comprehensive plan- change did streamline grantmaking and ciated with the proliferation of employ- ning, coordinated service delivery, ad- monitoring systems, and promoted ment and training programs, titled ministration and fund consolidation. comprehensive program management. Multiple Employment Programs. This series clarifies that there are now over Comprehensive Planning Fund Consolidation 150 programs devoted to workforce The 1967 Cooperative Area Man- In 1971, the federal government in- preparation, administered by 15 agen- power Planning System was the prod- troduced Planned Variations, anexper- cies, and funded at approximately 25 uct of four federal agencies working to- iment to give mayorsmore discretion billion dollars. The Commission itself gether on what Johnston views as a in spending federal grants. The test completed a comprehensive inventory grass roots-upward coordinated plan- yielded mixed results in terms of the in 1994 of 55 federally-funded employ- ning concept. This system was a way to power, sophistication and effectiveness ment and training programs adminis- enclose coordinated planning and re- of mayors in utilizing this flexibility. tered by 7 agencies, Understanding source allocation within an integrated Meanwhile, Nixon's revenue sharing Federal Training and Employment intergovernmental system linking local was to replace categorical programs in Programs, which includes information communities with different levels of several broad issue areas education on who sponsors and funds them, government. But the system was volun- and employment/training being two of whom they serve, how many are tary, the different governmental levels these and gave grant recipients even served, their funding levels, and infor- disagreed over control issues, policies greater discretion in spending funds. In mation from research on their efficien- were not sufficiently supported by the process, it was to allow for a con- cy and effectiveness. The array is more specific objectives and implemen- solidation of programs at the subna- mind-boggling. tation strategies, and the results were tional level, consistent with the priori- not carefully evaluated. Consequently ties of state and local governments. But What I. The Recent Response? the system did not live up to expecta- the Congress became concerned about In response to earlier concerns about tions, Johnston concludes. the lack of monitoring and evaluation program and service coordination, the 011011111111.mlwas ...m.=111000.1111MI 115 RAM) WU* 11 Summor 1995 Evialail04 123 Evaluation Issues and ActMtios . At the State and Local Level

of the efforts made, and the idea of rev- and the entire set of a client's needs, counseling. Agranoff called this a "bot- enue sharing faded away like an old rather than on a single problem, on toms-up planning approach." Rather soldier. pressures from a single advocacy than relying on the functions of agen- group, or on a single categorical agen- cies in bringing things together, the Another Perspective on cy. p6nner organizes a service system Coordination around sets of client needs. Although The author suggested that human the imperative for coordination at the The Commission draft report chroni- services integration be defined in terms federal, state and county levels was ad- cles the important, if imperfect, coordi- of: nation efforts through the 1990s. But dressed, the emphasis was on a local- A comprehensive policy that cuts before venturing past the 1970s, we level "human services subsystem", should reminisce a moment about the across independent programs and which was to operate much like a com- larger human services integration classes of services. munity development or land use sub- system. Within this perspective, natural movement of the late 1970s. We can A service delivery system designed catch a glimpse of this movement quite linkages were seen to exist between the to meet the needs of clients whose subsystem and a myriad of resources well in a large green tome used exten- problems go beyond a single prob- sively in social welfare courses in that within the entire community. lem, program or agency. era, titled Managing Human Services. But Agranoff was not naive about It was published in 1977 by the Insti- A consolidated organizational struc- the ability of governments and commu- tute for Training in Municipal Admin- ture that supports the above con- nities to implement this concept. He ac- istration and the International City cepts. knowledged the weak supports for Managers Association. Its editors were moving in this direction. Yet this an eclectic group: Wayne Anderson The problem stimulating serious movement was receiving momentum from the U.S. Advisory Commision on -thinking about integration, Agranoff from important people. Frank Carlucci, Intergovernmental Relations, Bernard said, was 1) the expansion of categori- a former Undersecretary of Health and Frieden of MIT, and Michael Murphy cal human service programs; 2) the Human Services, testified before the of the Management Association. multi-faceted role of government (per- Congress in support of this holistic ap- forming funding, regulation, purchase The book summarized, analyzed and proach. Social welfare experts such as of services, and other functions in addi- Alfred Kahn of Columbia University commented upon a considerable body tion to service delivery); 3) generic of managerial knowledge and experi- suggested a very simple service net- problems in service delivery (fragmen- work: "an access system, or numerous ence regarding human service policy tation, discontinuity, inaccessability development, delivery and evaluation. neighborhood information centers and unaccountability); 4) whether ser- charged with giving people information The content was focused on the chang- vice strategies are meeting their goals; ing roles of different levels of govern- and options, and allowing for selection and 5) the lack of policies seeking to and monitoring; beneralist social ser- ment in assessing a range of human ser- resolve these problems. vice needs, developing human service vice offices that would offer general policies and the delivery of such servic-, The author proceeded to provide an counseling and manage the case if spe- es, evaluating their results, and using overview of federal human service inte- cialized services are needed; and spe- this information to improve policies gration initiatives such as the Commu- cialized programs." and their implementation. Sound famil- nity Action Agencies, Model Cities, fed- This human services integration iar? eral research and demonstration movement clearly has much to offer in projects such as the Services Integra- The last chapter of the book honed the current debate over how to coordi- tion Targets for Opportunity program, nate policy, planning, service delivery in on service integration. Authored by the U.S. Department of Health, Educa- Robert Agranoff, a political science and oversight in large areas of human tion and Welfare's Partnership Grants services. professor, director of the Center for program, and state and county efforts Governmental Studies at Northern Illi- initiated with or without federal assis- More RecentHistory nois University, and chair of the Amer- tance such as the Flexible Intergovern- ican Society for Public Administra- mental Grant project in the northwest. For obvious contemporary cultural tion's section on Human Resources These efforts were more extensive than reasons, "manpower planning" became Administration, the chapter gives us an we might have imagined. employment and training or workforce impressive overview and commentary system planning in the 1980s and about the thinking of that era regarding The concept of a service delivery 1990s. Such planning had benefited coordination. Agranoff defined service model serving the whole individual from the lessons learned in previous at- integration as a veritable movement. with multiple needs actually surpasses tempts to cooperate, collaborate, coor- The belief expressed in that movement current integration initiatives. It was dinate, integrate and consolidate. But was that the most useful way to address not tied to categories of need, such as territoriality continued to pose chal- people's needs was an holistic one to education, training and employment, lenges. We return to Janet Johnston's that is, services should be based on the but broadened to include diverse ser- insightful analysis. needs of entire government systems, vices, income assistance, and personal

116 Evaluation Forum Issue 11 Summer1995 124 Evaluation Issues and Activities At the State and Local Level

The State Job Training Coordinating cated that close to 30% of all Private mation networks together, and a num- Councils and more loealized Private In- Industry Councils felt state councils ber of other states have developed a dustry Councils in JTPA revised the had no impact on their activities or had core set of measures common to all notion of decentralized comprehensive actually created problems for them. Yet programs within state workforce devel- planning in the direction of privatiza- states have been innovative in the way opment systems. tion. Based on a Commission study of they have established councils in terms the PICs, Johnston suggests that their of their sensitivity to existing organiza- Consolidation at the main weakness has been a lack of train- tional arrangements across workforce National Level ing in planning and oversight for programs and institutions. Some have It is interesting that despite the ma- chairs, boards and council staff. incorporated the state job training jor efforts being made at the state level council, as in Oregon. Some have fold- to carry out ever-stronger decentraliza- Secretary Reich's Reemployment ed in the State Council on Vocational Act took the council idea a step further tion and privatization mandates regard- Education, as in New Jersey. Some ing coordination, there is also a bub- in recommending the formation not have "umbrella councils" whose func- only of statewide and regional councils bling-up of support for a national tions are limited to high-level policy workforce development council or but also local boards that could oversee development, long-term planning and one-stop multi-service centers. Strate- board. The State Job Training Coordi- oversight, as in New York. Meanwhile, nating Council Chairs' Association, in gic planning, budgeting and oversight Indiana exemplifies the kind of council were to be the cornerstone functions of their monograph Bringing Down the recommended by the Reemployment Barriers, has recommended the estab- these new mechanisms for coordina- Act. tion. Again, experimentation with these lishment of a Federal Human Resourc- innovations have encountered substan- And there are instances of states es Investment Board by Presidential ex- tial territoriality. braving the waters relatively alone. ecutive order. This mirrors the role of Johnston reports that the MASSJobs the National Commission, raising the Johnston points out that the JTPA Council has developed strong regional possibility of expanding the Commis- amendments gave governors the option employment and training boards by sion's composition and responsibilities, of developing Human Resource Invest- providing genuine financial incentives as a more inclusive vehicle for nation- ment Councils as a way to expand the for their functioning as regional plan- al-level policy development, compre- stakeholder group and service cover- ning bodies, for their devising methods hensive planning and oversight. age. The Reemployment Act requires for measuring the board's effective- this. Currently there are variations on Adding ideas to the hopper is the ness, for their developing an expanded prestigious National Research Council, the theme. Eighteen states have formed labor market and service information councils of the kind recommended in through its Committee on Postsecond- system, and for their having linked the ary Education and Training for the the act. Some have established even boards with economic development ef- broader-based bodies. But other state Workplace. The Committee's recent forts. councils have involved fewer constitu- two-year study of workforce system in- novations, funded by the U.S. Depart- encies than mandated in the act. Many Information System Technology are moving in this direction but have ment of Education, concluded that a not yet arrived. In this age of information systems, new independent Office of Workforce we reluctantly recognize that manage- Development be created, much like the Since there are no sanctions current- ment information systems often drive National Science Foundation. This of- ly for failing to create comprehensive service integration, or at least support fice would apparently absorb some of state and local workforce system its development. There is increased in- the functions now performed by other boards or councils, and limited rewards terest in automating information for federal agencies. Given the current for doing so, states have made uneven customer use in one-stop centers trend toward reduced federal govern- efforts, their coordinating bodies differ- both information about the labor mar- ment, this careful study was quite im- ing regarding leadership, composition, ket and information about service op- portant in confirming the continuing priorities, policies and implementation tions, across organizations. And there need for strategic planning and over- strategies. Bringing previously-sover- is a fluorescence of interest in comput- sight at the federal level. If we believe eign organizational entities together erizing more sophisticated and useful that cooperation and coordination around common goals, linking these monitoring and evaluation information among diverse parties is an essential entities with employers and unions and across these territories high-tech in- characteristic of American social de- related stakeholders, has not been easy. formation systems now considered a mocracy, there is hope. However, given The mixed results of efforts to coordi- key element in coordination efforts. Congressional leadership sentiments, a nate JTPA and JOBS, whose client Few states have such systems in place, coherent national workforce develop- populations overlap, is an example of but many are moving in this direction ment effort may or may not be viewed the difficulties. despite difficulties in developing com- as an essential part of unrelenting ef- Another illustration is pinned down mon operational definitions and mea- forts over time to bring greater integra- in the results of a Commission-funded sures for key variables across pro- tion and rationality to workforce prepa- survey conducted by the National Alli- grams. States such as New York and ration. ance of Business in 1985, which indi- Wisconsin have gone far to pull infor-

117 Evaluation Forum Issue 11 Sumrrw 100 125 Evaluation Issues and Activities At the State and Local Level

Establishing and methods could guide the produc- gree in system planning and over- tion of objective information for policy sight. Comprehensive and planning most effectively. Nearly all were placing heavy stress Workforce States benefited from federal coordi- on strategic planning and a review nation efforts. Closer to the pressures of results against plans, across the Development for change, however, they were devel- Systems: oping their own strategies. Having al- entire workforce development sys- State-Level Efforts ready taken the initiative, many states tem. were thrther energized by the Clinton a Most were developing linkages in the 1990s Administration's coordination priori- across programs through local work- ties and legislative proposals in support force boards, local planning, and There has been a gradual recogni- of workforce development systems. one-stop centers for integrated ser- tion over time that what the federal These priorities placed emphasis on government and state governments emerging state-level efforts to develop vice delivery. across the country wished could be de- comprehensive state-wide workforce The majority were working toward a scribed as a nicely integrated set of ed- preparation systems that were bringing common core set of data elements ucational, training and employment ser- together individual programs and ser- and definitions, and a uniform set of vices for American workers was, in vices under one policy and planning performance measures and standards fact, a patchwork of separate service roof, and making the system's services that were to apply to all workforce programs, however well-conceived and accessible through coordinated multi- designed as separate initiatives. The de- service centers. Paralleling this boost to development programs. velopmental history of each major na- coordination were efforts to address ac- Most had not chosen to develop in- tional program, and a myriad of smaller countability issues in judging the per- centive/sanction systems related to state efforts, had resulted naturally in formance of these new entities. Now performance. diverse policies and planning priorities, the focus was on the outcomes and im- organizational structures, service deliv- pacts of systems of programs. About half were giving serious ery procedures and practices, ways to consideration to achieving consis- This article provides a selective monitor what was happening within the overview of state efforts in 1994 to ad- tency between system goals, skills program and its results, and methods standards, certification systems and for assessing the effect of influences dress accountability issues in new state- A wide workforce preparation systems. It accountability methods. imbedded in program environments. So is skewed toward states' beginning concern about and enthusiam for find- A slight majority was committed to ing solutions to the lack of coordinaton conceptual efforts, which have likely integrating data systems across the undergone considerable revision since has been building. workforce system, with the others originally conceived. The President, the Congress and the moving toward coordinated or federal agencies began to take the coor- Characteristics of Selected linked data systems. dination problem more seriously begin- State Workforce Systems The vast majority were requiring ning in the late 1980s, with JTPA, Based on materials provided by ap- that the system's programs be made JOBS and vocational education. A proximately nine states concerning accountable through multiple strate- number of studies of coordination were their state-wide workforce system's conducted to determine "best coordi- gies, including evaluation research. missions and goals, the following at- There was substantial interest in nation practices." At the same time, tributes were identified: there was an increased interest in social both process and outcome evalua- program accountabiliry. There were Most of the states were integrating tion, and net impact studies i.e., many reasons for this, among them education, job training, employment, a formal evaluation system. public anxiety about the economy, gov- income support, related supportive Few states had developed strategies ernment deficits, and the proliferation services, and economic development for disseminating and utilizing eval- of programs ahead of their evaluation. efforts. uative information for continually This "accountability movement" Most were assigning planning, coor- improving the system. was occurring at all levels of govern- dination and oversight responsibili- ment, as well as in the private sector. ties for the entire workforce system Common Features of State Evaluation researchers were brought to a single central body a state- Workforce Development into this debate and the search for solu- Systems tions because of the growing prestige wide council or board. of applied research and the perception About half of the states were involv- A prevailing characteristic of these comprehensive systems was that they that utilizing social science principles ing business and labor to a high de-

118 Evaluation Forum.Issue 11 Summer 1995 126 Evaluation Issues and Activities At the State and Local Level

merit Initiative to promote the produc- _had moved toward a market and states extending this effort to program tion of information on efficiency and ---(ntstomer-driven approach, away from implementation variables, and to a de- scription of services as well as client effectiveness, replete with a single traditional program-driven approaches. "performance index" for each program. This involved the private sector in new characteristics. A few states were at- plan- tempting to develop a single automated Florida attached the development of ways, particularly in strategic "industrial incubator partnerships" to ning, coordination and oversight service delivery "front-end" for intake, client assessment and case manage- its workforce system, promoting inno- and in the design of skills and certifica- vation within the system and creating a tion systems, as well as labor market ment. Most had been developing ex- new area for evaluation. information systems. panded and more integrated labor mar- ket information systems to support Texas applied sunset rules to pro- It was a new era also in the emphasis policy development and service deliv- grams within the workforce system, re- states were placing on performance ery as well as monitoring andevalua- quiring systematic evaluation. Provid- management systems oriented to out- tion. demand come measurement and the develop- ers had to document the market for services, and there were incentives ment of performance standards. In this In most states, outcome measure- federal ment and performance standards devel- for "best practices" based on evalua- sense, states were adopting tions. The state targeted money specifi- models and tailoring them to their cir- opment focused on a common set of cally for performance management and cumstances and needs. The stress on outcome variables: completion of edu- evaluation, and the training of profes- performance supported a greater inter- cation or training; acquisition of certi- fied skills; continuation of education or sional personnel needed for those func- est in going beyond monitoring and re- tions. It also created a formal Evalua- porting, and the review of results training; placement in employment (by against strategic plans, to a commit- occupation, industry and relationship to tion and Performance Committee. ment to evaluating the efficiency and the training received); earnings in em- North Carolina planned to seek pri- effectiveness of programs within the ployment, job retention and cost per vate sector funding for evaluation ac- workforce system, and of the system it- positive outcome. tivities. Iowa was pursuing an interest- self. Some states had given attention to ing, useful chronology of steps in integrating the workforce system's in- Few states had formal evaluation the economic development side of workforce development: job creation, formation system, by beginning with plans, but some had targeted funding structured interviews and focus groups for evaluation, had created advisory incentives for employers to retrain and with providers and stakeholders which committees for that purpose, and had retain workers, incentives for employ- ers to remain in the state, and entrepre- contributed insights to the IBM design developed professional evaluation consultants. The design was then im- strategies. Some had required consis- neurial projects emphasizing the devel- opment of high performance work- plemented in several phases that built tency across system goals, measurable investments objectives, certification system expec- places. on the state's existing MIS and allowed key operational and pro- tations, and performance measures, technical linking these elements within an ac- Some Unique State Approaches gram issues to be resolved as computer environments were created. countabilii), system. Despite important common elements The Great Lakes Guarantee, which Nearly all the states reviewed had across state-wide workforce systems, sought to weld together into one coher- been interested in pulling their data each state had customized its approach systems together more effectively to to its own environment, needs, inter- ent system the accountability efforts of several states within a regional labor support expanded oversight interests ests and resources. Michigan, for exam- market, existed at the high end of the and commitments. Some were integrat- ple, had classified outcomes according accountability continuum in proposing ing existing systems into a single uni- to whether they served customers or ad- the most integrated and comprehensive fied data system with common defini- ministrators. The state proposed com- approach to information production for tions and measures of key program mon forms and procedures, thedevel- policy development and strategic plan- elements. Some were developing new opment of common management data linkages across existing systems. standards, and the creation of inte- ning. Some were creating cross-walks be- grated local MISs. The combination of common ele- tween established MISs and data dic- New York's annual strategic plans ments and state-specific strategies tionaries. Some had expectations for lo- required an evaluation component. sketched above reveals the growing cal-level data integration which would Oregon was interested in technical as- emphasis on accepting more uniform link state and local systems. sistance and training for personnel car- principles within and across states for aa In all cases, a first step had been to rying out accountability tasks, and in guiding the integration, consolidation and/or coordination of more inclusive reach a consensus on a common set of common standards for the designof in- workforce development systems, while data elements describing desired out- formation systems. The state estab- comes for the entire system, with some lished a formal Performance Measure- at the same time preserving discretion

Evaluation Forum Issue 11 Summer 1995 . A7'

Eraluation issues and Activities At the State and Local Level II and flexibility in the way these princi- Contemporary consolidation; and the conscious ples were applied. shrinkage of business and industrial Coordination Efforts: workforces in an effort to preserve and The Implications of State-Level enhance profitability. System Development The Core Data At the same time, continuing tech- The renewed interest in 1995 in Elements Project nological change was contributing to block grant approaches, and a further an ever-widening gap between educat- devolution of federal monies to states In the late 1970s and in the 1980s, a ed, skilled and high-wage workers and and local areas, will require if im- number of states had seen the need for plemented greater organizational and service de- the undereducated, underskilled and very serious and careful low-wage. Both greater program strategic planning and coordination by livery integration across the many coor- dination and more precise accountabili- state governments, and stronger public- agencies and programs assisting indi- ty were becoming accepted ways of private partnerships at that level. The viduals in acquiring education, occupa- tional skills and sustaining employ- dealing with these new postindustrial experience of those states that have challenges. moved rapidly on federal coordination ment. The federal government's initiatives, such as state-wide work- anxiety about coordination seemed re- newed as well. This was not unexpect- Background of Recent force system development, should be in National Efforts a far safer position than before to cope ed. The public, and therefore the Con- with expanded decentralization and the gress, was concerned about the Consistent with the times, there responsibilities that accompany it. growing proliferation and fragmenta- were renewed efforts at the national tion of education, training and employ- level to reduce duplication, become ment programs with its attendant costs, more efficient organizationally, and re- and about the preoccupation with pro- form service delivery systems in the gram inputs to the neglect of outcomes. user-friendly direction. Giving the inte- In response to these concerns a gration movement legitimacywas a "second wave" of interest in human new president who was actively en- services integration occurred in the couraging greater coordination not only 1990s. Much of the force behind the from the bully pulpit but throughnew second wave was a desire to reduce Administration-designed legislation. government expenses. Influential also The purpose was to make coordination was the perceived need to reduce ser- happen. vice consumer's confusion and frustra- Relatively early in its tenure, the tion, make services more readily acces- Clinton Administration completed three sible, and assure greater service quality. interconnected proposals regarding 'nu- The first wave had arrived toward man resource development and coordi- the end of the 1970s under the Carter nation: the Reemployment Act, which Administration, but nothing much had proposed pulling programs for theun- come of this churning of the waters. employed and dislocated together; the The service integration movementwas Welfare Reform Act, which includeda given new life a decade later due to the greatly expanded and better coordinat- extensive economic restructuringoc- ed work/welfare system; and the curring in the latter half of the 1980s School-To-Work Opportunities Act that and the early part of the 1990s. New sought to knit schools, training institu- problems were emerging in an econo- tions, apprenticeships and workplaces my now imbedded in fiercer interna- together. In addition there was Ameri- tional competition. corps. And beyond these reforms there were new defense conversion and di- Of particular concern was the degree versification programs that addressed of displacement of skilled blue collar the dislocation issue with militaryper- workers, white collar middle managers, sonnel and with civilians andcommu- and other categories of workers inoc- nities affected by military cutbacks. cupational sectors affected by trade These initiatives were laced withcoor- imbalances; the movement of firms to dination mandates. areas of cheaper investment and labor; the downsizing of the military; there- These changes and related proposals configuring of corporations through involved unprecedented collaboration mergers, buyouts and other kinds of and coordination at the highest levels

120 Evaluation Forum Issue 11 Summr 1995 126 - - ,

Evaluation Issues and Activities At the State and Local Level

of government across the Depart- The second phase will concentrate willing, able translators and negotia- ments of Education, Labor, Health and on the development of a common set of tors. Unfortunately this ends in ineffi- Human Services, Agriculture, Com- measures and defmitions describing cli- ciencies, and often in ineffective ser- merce, and Defense, in particular. The ent statuses in workforce preparation vice provision and outcomes for federal government was essentially pro- systems: the status of clients entering, workers anci employers. viding a viable coordination model in progressing through, and leaving the In this context, the 1992 JTPA the way it was developing service inte- system. The workgroup is expected amendments required that "a core set gration alternatives. also to develop a common set of mea- of consistently defined data elements" sures and definitions for determining be identified which could be used for Service providers, now well social- eligibility across workforce systems. ized to thinking territorially, and essen- information-gathering across major federal employment and training tially rewarded for doing so by funding The Historical Rationale policies, did not always embrace these programs.The Departments of Educa- reforms. In the evaluation research Often viewed by policymakers and tion, Labor and Health and Human Ser- community, however, many social sci- program managers as a 'technical task' vices followed these amendments with entists were ecstatic. The public, the for computer specialists and therefore the Core Data Elements Project. They Congress, and the Administration somewhat removed from policy and were joined later by the National Occu- seemed to be speaking with a single management, information production pational Information Coordinating voice about workforce system evalua- concerning programs within the work- Council, the U.S. General Accounting tion, demanding that objective, science- force development system (and con- Office, the Census Bureau, and the Na- based judgments of new initiatives cerning the system itself) actually re- tional Governors' Association. that is, of both their implementation flects critical policy decisions about and impact be synonymous with what program features are most impor- The Product "establishing accountability." tant. These priorities subsequently af- The Project's workgroup recom- fect the way programs are managed. mended operational definitions and The Common DataElements Although information systems are measures, or indicators (data elements), Project meant to be a support for, policies, in for key process and outcome variables practice they often direct. and redirect Faithful to the JTPA Amendments, across Adult Education, Vocational Ed- policies. Consequently decisions about ucation, the Employment Service, the Departments of Education, Labor, what kind of information should and Health and Human Services, and Agri- JTPA Titles H and III, the Job Corps, can be gathered for various purposes JOBS, and Food Stamps. This accom- culture jointly sponsored a two-phase subtly channel program effort. project beginning in 1993 to develop a plishment was expressed in the form of core set of variables, operational defi- Therefore this unprecedented effort a sizable compendium, which was re- nitions and quantitative measures that to increase the coordination and inte- viewed by experts and practitioners at cut across mult:ple employment and gration of workforce systems through the federal, state and local levels. The training programs. The ultimate goal access to common information about review was undeniably positive. In fact was to integrate information production the programs within them is quite sig- it was the reviewers' conclusion that by standardizing the "information glos- nificant. And such access greatly en- the project's sponsors should expand sary" for the entire workforce prepara- hances the ability of administrators to the compendium to include variables tion system. A Workgroup on Common monitor and evaluate such systems for relating to client status and eligibility. Core Data Elements was established, policy purposes. In the compendium, common defini- involving an impressive number of rel- Each program has come off the Con- tions were provided for key terms used evant organizations and constituencies. gressional presses with its own linguis- in the array of employment and training The first phase of the project has fo- tic patterns, tailored to its legislative linguistic systems. These were to sup- cused on the identification of a com- era, the.assumptions of its sponsors, port the use of the core data elements. mon set of data elements (measures) for and the nature of its program design. The latter were clea lad within five use across programs in workforce sys- Different linguistic patterns tend to groups: tems, and their common definition. In produce different program cultures and Demographic elements. the summer of 1994, a draft report ti- environments. Programs can become tled Core Data Elements and Common hostage to that inevitability. Preserving Services received. Definitions for Employment and Train- programs' uniqueness, and protecting Service completion. ing Programs was released for com- their rationales and intents from being ment and further revision. The discus- eroded, these linguistic barriers also Program outcomes. sion in this article is based on this fight against coordination and integra- Employment descriptors. initial draft report, which may undergo tion. Collaborative policy setting, plan- The core elements are listed in Fig- change as a result of the final approval ning, program operation and account- ure 1, excerpted directly from the process. ability activities suffer from a lack of

Evaluation Forum Issue 11 Summor 1995 129 121 Evaluation Issues and Activhies At the State and Local Level

ra Streamlining and Integrating Human ..: Resource Development Services for Demographic Elements Service Completion Adults, supported the need for a stan- Social Security Number Completion of Occupational Skills daedized information system glossary. It Date of Birth Training (non-OJT) also clarified that the benefits of coordi- Gender Completion of On-the-Job Training nation were greater than its risks Race/Ethnicity (OJT) greater flexibility in using funds, in- Disabled/Individual Completion of Work Experience creased knowledge and communication, Education/Highest Funned Grade Completion of Pre-Employment Skills/ Level at Entry Job Readiness Training more efficiency in selvice provision, Highest Degree/Credential at Entry Completion of Basic Skills Education greater ability to use specialized exper- Labor Force Status at Entry through 8$ Grade tise, better monitoring, and enhanced Scheduled Hours of Work Per Week at Completion of Basic Skills Education at performance. Entry (if employed) Secondary Level (9 through 12) The report on the Core Data Ele- Homeless individual Completion of English as Second Lan- Veteran guage ments project also comments on the Public Assistance (federal, state, or lo- Completion of Postsecondary Academ- benefits of a more logical information cal) ic (non-occupational) Education collection approach, listing the follow- Long-term AFDC Recipient Reasons for Not Completing Planned ing major advantages: Service(s) Services Received Removal of identified barriers to the AssessmerVesting Program Outcomes coordination and integration of ser- Counseling/Career Development Advanced to Higher Level of Education vices across employment and training Job Search Assistance or Training programs at the state and local levels. Occupational Skills Training (non- Attained Additional Degree/Credential OJT) Assessed Learning Gain in English as Enhanced program planning and On-the-Job Training (OJT) a Second Language oversight capability at the federal, Work Experience Assessed Learning Gain in Basic Skills state and local levels. Pre-Employment Skills/Job Readiness Entered Unsubsidized Employment Training Entered Subsidized Employment Removal of barriers to efficient cus- ; Basic Skills Education through 8th tomer service. Grade Employment Descriptors Basic Skills Education at Secondary Scheduled Hours of Work per Week Prologue Level (9 through 12) Earnings (Hourly orAnnual) English as a Second Language Occupation The work of the Job Training 2000 Postsecondary Academic Education Performance Standards Subgroupin (non-occupational) *Common definitions have been devel- 1991 revealed considerable ambiguity Supportive Services oped for the italicized terms. in the goals of programs and the lack of outcome measures in a number of them. Not surprisingly, the Subgroup recom- workgroup's draft report. Definitions with the corresponding element and its mended the establishment of a core set of the data elements are listed in the common definition. Project participants of performance outcomes. By 1993, the draft report's Appendix. Detailed there are encouraged to add more detailed in- Government Performance and Results also are current differences between formation to their information systems Act was requiring agencies to establish programs regarding 1) whether data are consistent with the common defini- program goals, and performance mea- collected on individuals, cases or ag- tions. In this way, sponsors hope this sures consistent with them. At the same gregate characteristics, and 2) the num- convergence toward a common under- time, the NGA Peformance Manage- ber and type of data elements collected. standing of terms and indicators will ment Project was pinning down some The report clarifies that proems may yield a common set offorms and inte- basic principles for establishing ac- adopt the common definitions but grated data systems. countability in coordinated state work- choose to make selections from among force development systems, emphasiz- the core elements, in moving toward The Benefits of Information ing a logical progression from system more comprehensive information sys- Consistency goals to system performance measures. tems. Part of the prelude to this innovative A core cluster of standardized defini- The Project's sponsors requested project was a 1991 NGA survey of tions and measures was viewed as an es- that if programs represented in this ma- state and local policymakers and pro- sential dimension of state E&T "ac- jor effort collect information on a sub- gram administrators from 13 federally- countability systems." ject included within the core set, they funded education, training and employ- Meanwhile, information system in- should do so in a way that is consistent ment programs. The survey report, novations at both the federal and state

122 Evaluation Forum Issue 11 Summer 1995 1 3 `7,7_77 -

Evaluation Issues and Activities At the State and Local Level levels were beginning to capture infor- Comprehensive State quate information for improving pro- mation on participants' characteristics, grams and suggesting major reforms. In services and outcomes across program Workforce Development that vein, they recommend the follow- territories, increasing the ability to Systems: Key ing: track clients in coordinated systems. Simultaneous study of a program's This was a significant achievement at a Elements or a system's 1) organizational and time when 39 states had developed service delivery policies and pro- one-stop centers. As a first step in a 1993 National Governors' Association project consid- cesses, and 2) its outcomes and So with this kind of evolution in ering the goals and strategies involved longer-term impact, or petformance. thinking about the important role of in- in useful performance management sys- formation systems, the Core Data Ele- Promotion of continuous improve- tems, NGA commissioned a monograph ment in the crganization and opera- ments Project is looking at the compen- on these issues from the Center for Gov- dium, and the additions made to it in ernmental Studies at Northern Illinois tion of programs or systems through the next phase of the project, as more University. Authored by John Baj and the use of a combination of 1)pro- of a launching pad than a landing Robert Sheets in 1994, Building State cess standards and 2) outcome stan- place.They have recommended these Workforce Development Systems Based dards (performance standards). subsequent steps: on Policy Coordination and Quality As- Use of a capacity-building (quality surance did more than provide a frame- The incorporation of the compendi- assurance) system for adopting work for NGA's Performance Manage- um of core data elements by addi- "best practices," based on the results tional federal agencies, when report- ment Project. Its perspective on the statewide coordination of policy and of process and outcome studies, and ing requirements and new for the purpose of continuous im- management information systems service provision, and on the evaluation of state workforce systems, links evalu- provement. are developed, and by state councils ation research principles and methods and agencies involved in compre- The authors summarize their posi- with the principles of management and tion in stating that "the major challenge hensive workforce development sys- program operation, giving attention to a tems. in building cohesive workforce devel- range of accountability issues rarely dis- opment systems for labor market en- The expansion of the core set into a cussed simultaneously or merged in a trants, unemployed workers and eco- comprehensive dictionary of com- single approach. nomically disadvantaged individuals is mon definitions across education, Inputs versus Outputs: The to improve the quality of programs and training and employment efforts, Authors' Position service delivery throughout the system maintained and updated by OMB. while achieving greater accountabili- The authors begin with the conclu- ty." Therefore the purpose of their re- The use of such a dictionary in de- sion that we have lived for several de- port for NGA is to suggest a conceptual veloping new legislation. cades with an imbalanced focus on in- model that is responsive to that kind of 4:1 puts (service "treatments") to the It is to their great credit that the mandate. exclusion of adequate output (outcome) project's sponsore hope to take their The authors sketch the model in this measures. We have essentially empha- mandate further, to the ultimate end of way. The outer boundary is a "broad sized the quality of services and service this exercise, namely policy consisten- policy coordination framework" de- systems, failing to hold programs ac- cy. The focus on information systems is fined in terms of a logical, internally countable for their results. But profes- to lead in this critical direction. consistent progression of priorities: the sionals are beginning to question wheth- system's mission; its goals and objec- er the pendulum has swung too far to the output side. The belief that petfor- tives; its implementation strategies; its desired outcomes; its target groups, mance standards can lead sometimes to client eligibility criteria and types and selecting those most apt to succeed and intensifier of services; and its allowable can inappropriately redirect a program's costs. Quality assurance is seen as a intent, and that not meeting standards mechanism for maintaining and en- may torpedo programs important to pre- hancing the structures and functioning serve and refine, has led to new cau- of this coordinated system. Quality as- tions. surance systems are viewed as having a So Baj and Sheets take a middle po- performance management component sition on the swing of the pendulum. which guides the workforce develop- They say that an exclusive emphasis ei- ment system in the direction of its goals ther on inputs or outputs denies us ade- and objectives, and supports continu-

Evaluation Forum Issue 11 Summer 1995 131 123 Evaluation Issues and Activities . At the Stat. and Local Level

ou.s: improvement in system processes grams," and should encourage better occupational licensing and certifica- and outcomes. communication between the public and tion, and private professional certifi- The performance management corn- private sectors. We shouli investigate cation and recognition systems. this perspective on performance stan- :, ponent is expected to accomplish the dards. Supplier certification and recogni- following: tion: training and technical assis- As mentioned earlier, the authors Incorporate the principles of continu- tance related to the certification of distinguish between process, or design ous improvement. standards, and performance, or out- suppliers (service providers). Judge the entire system's perfor- come, standards: Program and subsystem perfor- mance, as well as the performance of Process standards: address specific mance management: monitoring and programs within it. characteristics of the service deliv- reporting performance based on de- Make these judgments using multiple ery process, such as internal organi- sired outcomes and standards, and measures of outcomes, including ed- zational policies and procedures, diagnosing performance problems. ucational achievement, occupational assessment and instructional ap- Baj and Sheets insist that the devel- competencies, customer satisfaction, proaches, staff qualifications, and opment of common sets of standards is educational and employment contin- monitoring and evaluation strate- essential to performance management uation, employment retention and gies. and therefore to quality assurance at any level of the workforce system: that earnings, and industry-based skill Performance standards: address the standards. is, educational and occupational skill results of service interventions, or standards and credentialing systems; Use a common set of outcome vari- "treatments," on those receiving performance standards; and supplier ables, definitions, measures and per- them, including desired changes in process and outcome standards and cer- formance standards in arriving at their academic and occupational tification requirements. these judgments. competencies vis-a-vis national and The authors are unequivocal about state skill standards. Cffer incentives and apply sanctions the role of evaluation. The improve- associated with performance stan- Monitoring and evaluation are es- ment of system performance, and the performance of programs within the dards, including capacity-building sential functions in arriving at objective system, "require a strong linkage be- assistance. judgments of the movement toward compliance with these standards. And tween performance management and Although the various corcepts in this movement toward the standards is evaluation." This linkage supports and 1990s "system"-oriented vocabulary viewed as supporting a commitment to reinforces continuous improvement. quality assurance systems enclosing continuous quality improvement. The Evaluation, they suggest, plays several performance management systems, and authors seem to be saying that the dual roles: performance management systems en- emphasis on judging the quality of both To validate quality standards. compassing continuous improvement the design aspects of the workforce sys- are not always as clearly defined or as tem and its performance is a new devel- To diagnose performance problems. well differentiated from one another as opment. However, a more equal em- To establish continuous improve- one might wish, the monograph is enor- phasis on process and outcomes came ment as a supplier process standard. mously significant and useful. Let's into vogue in the late 1980s, even look further into it. though it could only be applied to indi- Figure 1, excerpted directly from the vidual programs in the absence of com- report, gives us an overview of this part Quality Assurance prehensive coordinated workforce sys- of the authors' model. The authors de- tems. What is new is this dualism in fine the traditional role of evaluation The establishment of quality assur- research as focusing on basic provision ance systems, as an element of adminis- passing judgment on the new networks of programs. decisions (who should receive what trative coordination, should respond, the type of services and at what cost), and authors say, to customer-defined out- The authors do not neglect, in their assessing the returns on high-cost ser- come standards. They claim that quality model, subsystem quality assurance. vices (such as case management) to tar- assurance systems require "the major They propose three major quality assur- geted populations. Evaluation research- restructuring and integration of all per- ance functions at the level of programs ers will likely quarrel with this view. formance standards, skill standards, and within workforce systems: As part of a management system, the related program accreditation and certi- Trainee credentialing and recogni- authors feel evaluation should provide fication standards used by federal and decisionmakers with information on the state workforce development pro- tion: educational credentialing, state

124 1 32 Evaluation Forum Issue 11 Summer 1995 . Figure 1. aContinuous Inwroyemeht FrOce,dures and their Links to Evaluation and FieSeatch

POlicCqordination ..-Fim'ewo.rk' a

cn

Educational and IFProcess Standards Supplier or Service Provider Subsystem and Program Occupatinal Skill Outcome Standards Outcome Standards Standards

Academic Continuous Trainee Competencies m Trainee Competencies improvement Occupational Trainee Returns Trainee Returns Procedures Basic Workplace ODevelop an organiza- Customer Satisfaction e Customer Satisfaction tional plan to ad- Social Cost Reduction dress performance imprcaements. Involve all staff and all major functions and pnxesses of the organization. Use recognized Im- provement method- State and National ologies. Evaluation Use valid and reli- able data with ap- Vaiidation of Quality Standards , A propriate analytical Best Practice technologies. e Effectiveness of Services a Determination of Who Benefits from What Services

-4 133 134 BEST (ITVAVAILABLE _ - _

Evaluatbn Issues and AettvItkis At the State and Local Level

"validity and cost-effectiveness of all as to become effective indicators of net Information Systems major provision decisions." Their ex- impacts, based on the differing charac- tension of the role of evaluation re- teristics of client groups (unskilled vs. Without adequate data, quality as- surance systems can accomplish little. search to validating process and out- skilled, for example). But effective and Therefore, the authors recommend that come standards stretches that role 'valid are two different things. Further- automated management information considerably, perhaps unrealistically more, weighting and adjusting are more and inappropriately. appropriate in judging compliance with systems at all levels must be capable of supplying the kind of information performance standards Than in inter- Evaluation research has been of needed to implement the various over- preting program outcomes, it being greatest use in providing the least sight strategies suggested in their re- biased information about the nature possible to disaggregate results for dif- ferent client groups in interpreting data port. The emphasis is on a common set of service provision and its results for of quality indicators for process and analyses. clients. In the process, it can contribute outcomes. A related intetest is in the information about the level of compli- Confusing also is the authors' final development of standardized instru- ance with performance standards. But conclusion about the differences be- ments, meeting scientific standards, for the validation r.t quality standards has tween evaluation and management. conducting customer satisfaction sur- not been wit'. 'n its purview. Having advocated for evaluation as an veys. The authors also recommend the The authors claim that evaluations integral part of performance manage- development of cost-effective methods of national programs and demonstra- ment systems, they later differentiate for gathering and analyzing informa- tion projects have not been designed to between evaluation and performance tion at the point of service delivery, to improve the ongoing delivery or pro- management by pointing to the infre- serve supplier certification purposes duction of workforce development ser- quency of impact evaluation and the and to provide a basis for judgiag con- vices. This is open to debate. It is true, need for performance management sys- formance with qualiy standards. This however, that evaluations have not fo- tems that supply more immediate feed- should be paired, they indisate, with ef- cused on different kinds of service pro- back to administrators, program opera- forts to train suppliers in the continu- viders at the point ofservice delivery, tors and service providers. Therefore ous improvement process and provide which the authors feel must be a re- they comment that "quality assurance ongoing technical assistance to them. quirement of performance management systems are largely concerned with out- systems within quality assurance sys- come and process questions that Major Recommendations tems within workforce development emerge from accountability rather than in the Report systems, if continuous improvement re- impact issues." What does that mean? There are three key recommenda- quirements are to be met. Are they saying that evaluation re- tions:1) the encouragement of state Some evaluation researchers have search can contribute to performance experimentation and demonstration in concluded that some of the program management but is not suited to an on- policy coordination and quality assur- outcomes typically measured are not going oversight role? Performance ance; 2) the development of federal- necessarily good indicators of longer- management begins to look a great deal state initiatives for establishing a quali- term impacts. But the authors take this more like an expanded version of com- ty assurance system within a national conclusion a step further by interpret- pliance monitoring, rather than as a policy coordination framework; 3) the ing it to mean that some measures are source of objective, science-based in- promotion of federal-state initiatives not valid indicators of program impact. formation about the critical relation- for developing a national policy coordi- Indicaror validity usually refers to the ships between client characteristics, nation framework beyond quality as- extent to which a particular measure or services, and outcomes. surance. The positioning of the recom- group of measures accurately represent The authors also say that continuous mendations reflects the authors' belief that initiatives should begin in the the outcome variable being measured. improvement should involve ongoing Determining indicator validity is a sci- organizational self-evaluation through states, and build toward a national framework entific process. One cannot simply leap internal review teams, as well as from position A i.e., the conclusion through third-party process and impact What do these recommendations in- that measures such as job placement evaluations. And the basis for continu- volve? and off-welfare do not necessarily pre- ous improvement should be a clear def- Recommendation #1: States are dict well to net earnings and welfare inition, they suggest, of what "im- identified as the location for experi- savings to position B i.e., the provement" is to mean, in the form of first two measures are not valid indica- criteria and standards. This definition mentation, but the federal govern- tors of the latter two. process is to be applied at all levels: the ment is expected to 1) encourage and assist states in experimenting, The authors suggest that outcome system as a whole, the programs within and in testing policy coordination measures be weighted and adjusted so the system, and the service provider or- ganizations involved, frameworks that include school-to-

126 135 Evaluation Forum Issue 11 Summer 1995 ': aaideal ad; v tam' ;ta;tiiiaiataiitti,

Evaluation Issues and ActMties At the State and Local Level

work, workforce entry, and worker The National Phase Two: The Six-State retraining and transitioning, and 2) Accountability Effort develop a national policy coordina- Governors' Following the production of the tion framework for all publicly- Association monograph, in the spring of 1994 NGA funded workforce development pro- invited states with state-wide work- grams. Performance force boards or councils to apply for Management Project technical assistance to develop anac- Recommendation #2: Several na- countabiloi modelfor their workforce tional initiatives should contribute to With funds from the Joyce Founda- development systems i.e. to apply state experimentation: tion, NGA worked with the Centerfor for a second phase of the performance Governmental Studies at Northern Illi-management project. Linking national skill standards nois University in 1993 to develop a to state quality assurance sys- Technical assistance was to be made conceptual frameworkfor improving tems. available for fifteen months, the first services and accountabilha across twelve of which were to be devoted to Restructuring public and private workforce programs, with a special in- the development ofa state performance program accreditation and certifi- terest in the development of compre- management model.Funds for this as- cation systems. hensivestateworkforce systems. This sistance were to be provided by two framework was described in NGA's private foundations the Joyce Foun- Establishing national/state capac- publication, Building State WorVorce dation and the John D. and Catherine T. ity-building programs supporting Development Systems: The Critica: MacArthur Foundation, and the U.S. quality assurance. Roles of-Policy Coordination and Departments of Labor, Education, and Quality Assurance.This monograph Health and Human Services. This phase Linking evaluation to perfor- was prepared in anticipation of a six- mance management within the was to be coordinated with the National state project sponsored by NGA, and Institute for Literacy's performance quality assurance system. served as the major resource material management project. for that project. With respect to the last initiative, the The states selet ted on a competitive authors suggest that the results of im- Phase One: The NGA Working basis to receive assistance were expect- pact evaluations should be used within Paper ed to designoutcome-based account- the quality assurance system to exam- abilio, modelsfor 1) preparing youth ine policy decisions, their implementa- The concept proposed in the mono- for the school-to-work transition, tion, and the validity of quality stan- graph wasa performance management 2) retraining and transitioning of other dards. Again, it may be more accurate systemthat incorporatedquality assur- workers, and 3) helping disadvantaged and to talk abot t theappropriateness anceandcontinuous program improve- adults and those with multiple barriers utiliV of quality standards. mentin a model that allowed policy- to enter and re-enter the workforce. The makers and planners to judge I) a Comments on the Report six states were to serve as laboratories workforce system's implementation for establishing accountability across Despite some fuzziness in distin- (how it was organized and made things workforce development programs. The guishing among system-defining con- happen, including its level of coordina- concepts of quality assuranceandcon- cepts, the overall model the authors tion); 2) its participants' outcomes; and tinuous program Improvement were to propose is admirable. It includes all the 3) the system's impact and net effects be dimensions of the models these major elements reflected in what we judged against its most significant states were to develop for application have learned from experience, expert goals. The idea of "continuous im- in their own state and for potential use provement" was considered to involve opinion and research about policy de- elsewhere. velopment, administration, planning, ongoing self-evaluation, and a redesign program operation, service delivery and of various elements of the workforce The RFP Process system in response to that evaluation. oversight, bringing these elements to- NGA's project RFP made the as- In The notion of "quality assurance" in- gether withina coordinated system. sumption that many states were now in developing this model they ably resist volved the development and use of per- formance standards, educational and the process of restructuring their work- the temptation to ignore the different force development systems into what the occupational skill standards, and stan- levels of human resource activity was described as "seamless, customer- equally important national, state, pro- dards for judging the operation of the system. The concept placed heavy em- driven delivery systems" in response to gram-specific, and service delivery- these well-recognized problems: di- specificlevels. State workforce devel- phasis on evaluation research and the verse and often contradictory federal opment councils and boards, and innovation it was capable of encourag- ing. and state program goals, performance pragam administrators operating with- expectations, funding incentives, oper- in comprehensive systems, will find ational rules and regulations, and defi- this report stimulating and useful. nitions of program terms. The RFP considered the participating states to be

1111.1111~11.11. 122 Evalua(ion Forum Issue 11t Summer 1095 136 -

Evaluation Issues and Activities At the State and Local Level

developing performance management models for much larger entities than those to which performance manage- Create a plan for the development ment principles had been applied in the ofthe a Develop specific monitoring and eval- performance managementI uation methods for hording state and past this time to compithensive model thatwill outlinetheaction state-wide workforce systems, not local program managers accountable steps to be taken, Including the pro- for achieving system goals. merely to individual programs or sets cess to be pursued, the deliver- of related progxams within suchsys- ables, the timelines, and the teani- fi Incorporate qualify assurance and tems. The design of requests for waiv- cal assistance needed. continuous program improvemont within the performance management ers to federal and state statutes and im- involve the state's workforce board - or council. model. plementing regulations were clearly to a Develop occupational standards for be part of participating states' responsi- s Invest swedes providers in the project. use across programs within the sys- bilities. tem. Designate anInteragency policy More specifically, states Applying teamand a 3 if desired, develop an incentive/sanc- technical work group, tion system for rewarding successful for the project were expected to pro- and identify a coordinator who will duce the results listed in Figure 1. The head the policy team and act as tha system operation and performance, ultimate goal, in developing perfor- liaison with NGA. or for providing technical assistance to resolve system problems. mance management models, was to Focus on three priority groups of support the survival and enhancement workers a Define federal, state and local over- youth, adults needing re- sight roles and responsibilities vis-a- of a single integrated workforce system training, and disadvantaged work- vis the system. one in which key system functions ers with the option of gMng greater emphasis to one or more af 3 Refine the model, working with na- such as policymaking, coordination, in- tional experts. formation production, monitoring and these. evaluation would be commonly defined Establish a set of state goals for the e Participate in interstate coordination efforts, and share information and and applied across an array of work- state's workforce system, for each priority group. problems with other states in the force programs. This kind of standard- NGA project. identify quantifieole outcomes for ization was to result in a more orga.: Participate in three project meetings nized, efficient and accountable the system, for each priority group, given these goals. sponsored by NGA. organizational structure and service de- Identify federal and state regulatory livery system. a Develop a common glossary of pro- gram terms for the system. barriers to applying the model. Clearly the RFP was asking a great Develop a common set of opera- Produce a state-approved report on deal. But states were receiving some- tional definitions and quantitative the twelve-month design phase, thing important in return assistance measures for the key factors to be which details the dimensions of the from research experts at the national studied in judging the accountability state's model arid presents a plan for level who were not otherwise available of the system. implementing it. to them, validation of their approaches Elegin to implement the model devel- oped. to accountability from their peers, and a certain level of national notoriety for contributing to the dialogue on system plicant's project organization plan coordination. Based on the above criteria, six appeared to be, and the strength of statea were to be selected. The current The Selection of Project the state's commitment to attending status of each of these states'efforts to Participants the three national meetings. develop a performance management strategy for its comprehensive work- NGA's selection criteria focused on How clearly the applicant had artic- force aystem was to be taken as the the following issues: ulated technical assistance needs. baseline forjudging its progressover How the apnlicant would build on How substantial the results of the the life, of the project. Each of thena- and enhanee current state efforts to project were to be. tional meetings was to further refine each state's model through input and develop comprehensive workforce Applicants who planned to focuson materials front the national consultants, policies, integrate local planning and all three groups of workerswere given structured peer reviews, and the sharing service delivery, and develop and preference. In an effort to assure diver- of information on common issues. use common monitoring, perfor- sity in the models developed, other fac- States would be able to obtain technical mance and evaluation standards tors were considered, such as the com- assistance from the consultants and across programs. plexity of the applicant's workforce from NGA staff on-site, and through system, the nature of the institutional conference calls and correspondence. What priority groups would be em- environment of the system, the size of Representatives from any of the six phasized by the applicant. the state applying, and its geographical states could share information and ad- location. How appropriate and feasible theap- vice onssite with other states if this seemed useful. 128 Evaluation Forum Issue11 Summer 1995 137 Evaluation Issues and Activities At the State and Local Level

The Technical Assistance solidate at the federal level, the states NGA also provided participants with Offered selected were to have an opportunity to a Glossary of Performance Manage- work closely with relevant federal ment Terms which was an extremely These were the kinds of technical agencies. useful tool for sharing insights across assistance states could access in the the participating states. project: Project Orientation 1. The provision of reports and work- General ideas in NGA's To prepare the six states to under- issue Paper ing papers from NGA's Reinventing take a clearly-defined conunon mis- Government Task Force, appropriate sion, even though their accountability As a context, NGA staff reminded private sector representatives, and models would reflect the diversity of states in the paper that the quality of national experts. their circumstances and interests, NGA educational and other workforce devel- held the first of three national project opment programs has been judged tra- 2. The provision of information on meetings in Detroit in mid-June, 1994. ditionally on the basis of accreditation successful state practices. A number of working papers were pro- criteria, program approval processes, 3. The outlining of a process for dis- vided to give participants a framework and planning requirements oriented to compliance with rules and regulations seminating and utilizing research for carrying out the mission. These in- concerning inputs. The pendulum had findings for lOCZ1 program planning. cluded the substantial report, Building State Worlcforce Development Systems, swung, they said, in the direction of 4. The provision of information on a and the first draft of the report prepared judging outcomes replete with in- set of process and outcome criteria by the Workgroup on Common Core centive/sanction strategies, affording for continuous program improve- Data Elements, Core Data Elements flexibility in the design and refining of ment and self-assessment. and Common Definitions. programs, and relying on competition to promote high quality. However, they 5. The provision of assistance with the The materials also included the Na- acknowledged that an emphasis on both development of waiver requests. tional Institute for Literacy's 1994 re- inputs and outcomes was important, port, A Guide to Strategic Planning for 6. The provision of information on leg- and that a commitment to continuous Performance Measurement, Reporting improvement must involve an interest islation, and reports on school-to- and Improvement Systems, an issue pa- in both. work performance measures, adult per prepared by NGA for the first education quality indicators, com- project meeting, and an overview of The new wrinkle in this ongoing di- mon data definitions and measures, Texas' process for developing core per- alogue about accountability, they pro- formance measures.. posed, was the emergence of more and the use of In data. comprehensive and coordinated service 7. The provision of a list of consultants NGA also outlined general expecta- systems that cut across individual pro- tions regarding the characteristics of specializing in performance manage- grams. This has generated interest and the accountability models the states ment, concern about how to appropriately were to develop: judge system-wide processes and out- The Selection of Project States Measurable state-wide goals for : comes. Homogeneity of program terms 1) youth transitioning from school and goals is merely an essential preface Thirteen states applied to be part of to developing system-wide visions, the NGA project. The following six to work, 2) workers needing retrain- ing and transition assistance, and benchmarks, goals, objectives, outcome were selected: Illinois, Iowa, New York, gni 3) adults with multiple barriers to expectations, and performance stan- North Carolina, Texas and Washing- dards. ton. These states were considered to employment. The movement is toward what NGA have made significant progress in re- A common set of measurable de- stxucturing their range of workforce de- terms system-wide shared accountabili- sired outcomes for the customers of velopment programs and projects into ty. This approach envisions a system the workforce development system. high-quality, more customer-driven with these dimensions: workforce preparation systems. In the State and local performance stan- 2 A set of measurable policy goals for NGA project they were expected to dards consistent with the desired the entire state. take a subsequent step: to develop "the goals and outcomes. next generation" of performance man- A set of measurable goals for the

agement systems, which would incor- A common information system .workforce development system con- porate the concepts of quality assurance (variables, definitions and measures) sistent with state policy goals. and continuous improvement, and for reporting progress against the a A common set of measurable out- would stress joint accountability across goals. different programs and levels of gov- comes for all workforce develop- ernment. Given the Clinton Adminis- Joint accountability across the sys- ment programs within the system, tration's efforts to restructure and con- tem for achieving the goals and consistent with the goals for the sys- meeting performance standards. tem. 101 Evaluation Forum Issue 11 Summer 1995 129 136 Evaluation Issues and Activttles At the State and Local Level

A single incentive/sanction strategy management information, monitor- velopment, with performance stan- that can be applied system-wide to ing information and evaluation re- dards phased in incrementally so as support the achievement of system sults? to take advantage of program intbr- goals. 5. Is it useful to focus on the different mation which can inform the choice A commitment to ongoing self-eval- populations served, or should there of standards. uation to determine the level of be a general accountability policy Measures of clients' access to the progress in achieving system goals. that applies to all customers? system should be included, as they This directs a very logical progres- 6. How should program outcomes be are important indicators of service sion of thinking and planning. Howev- linked to outcome expectations for provision equity and the level of re- er, it is questionable whether state-wide the system as a whole, and what sponse to key target populations. policy goals can be stated such that should the consequences be for they are measurable in the same sense Key outcomes to track are: labor meeting, exceeding or failing to as system and program outcomes can market outcomes, learning out- be measured. Benchmarks are by nature meet system-wide performance ex- comes, and client/employer satisfac- abstract and difficult to measure either pectations? tion outcomes. qualitatively or quantitatively. The Survey Undertaken by the Survey respondents expressed cau- In discussing this progression, the Texas Council on Workforce and tions as well as making recommenda- writers of the issue paper distinguish Economic Competitiveness tions. They viewed "entered employ- between performance management ment rate" as a good measure if used tasks and continuous improvement The Performance and Evaluation with other measures such as "employ- tasks. They suggest that goal/outcome Committee of the Texas council con- ment retention," the "appropriateness development is the former type of task, ducted a survey of eleven national ex- of the job," and "the degree of self-suf- and evaluation research and capacity perts in the performance management ficiency." However, they pointed out building the latter kind. However, eval- field. The survey instrument consisted that placement rates might not be ap- uation researchers would likely consid- of a questionnaire with ten discussion propriate for judging educational pro- er evaluation research activities to be questions. The experts were asked to grams or further training. They did not an integral part of a comprehensive respond in writing and subsequently think "training-related placement rates" performance management system throughtelephone interview. Eight were useful in all cases because of which incorporates quality assurance, responded. Some of the major insights 1) the ambiguity of definitions of continuous improvement, and the iden- from the survey are summarized in the "training-related," 2) the uneven im- tification of "best practices." sections below. portance of this measure in educational and employment assistance vs. occupa- The issue paper poses some very im- Labor Market Outcomes tional skill training programs, and 3) its portant and useful questions: Multiple quantitative and qualitative different meaning for longer-term spe- 1. What should be the roles and re- measures of key factors should be cialized occupational training than for sponsibilities of federal, state and lo- developed to compensate for defi- short-term training. Nevertheless, in cal governments and the private sec- ciencies in any one indicator, al- some instances and with some training, they considered the training-related tor in promoting and assuring shared though the system must remain man- measure very important. accountability? ageable. 2. Does there need to be shared respon- All agreed that "wage rate" mea- Measures should not only include sures were important. However, they sibility for planning and oversight at sheer outputs, such as placement felt it was difficult to sort out their each level of the system, and what rates, but indicators that provide in- meaning vis-a-vis other factors such as kind of governance arrangements formation on changes over time, economic changes, or personal attitudes should be put in place to assure such as wage rates. and situations. They also agreed that UI shared accountability? Wage Record information was the best Adjustments in performance stan- 3. How can the contribution of each source of information on "earnings," dards, although to be kept to a mini- "employment affiliation," and the char- funding stream to the collective ef- mum, should consider local eco- acteristics of firms where program par- fort be determined and judged? nomic conditions and the ticipants were placed. They recognized 4. Is it enough to have information sys- characteristics of participantpopu- that tracking information specific to tems with common definitions and lations. particular occupations, developing indi- cators for the quality of jobs, and dis- measures of outcomes, or are inte- Core measures to be used across a grated data systems needed which tinguishing between full-time and part- workforce system should be afirst time employment in the wage records offer customer information, case step in performance management de-

130 Evaluation Forum issue 11 13 9 Summe; 1995 Evaluation Issues and ActivitiesI. At the State and Local Level

were a problem. They were ambivalent the six states and the history of their re- The measures to be collected, repre- about client interviews as a way to en- cent efforts to coordinate multiple edu- senting these outcomes. hance the information on wages, be- cational, training and employment pro- The Illinois progress report followed cause of the general unreliability of grams to create a new workforce this logical sequence of decisions self-report data, but thought this strate- 'system'. Figures 2 and 3 provide a closely. Twelve programs were includ- gy was feasible if used in combination brief overview of the content of their ed in the system. State policy goals en- with UI data. responses. compassed early childhood develop- All the experts surveyed recom- Some Observations About the ment, educational performance mended the use of "job retention" mea- RFP Responses standards, a school-to-work system, lit- sures used in combination with place- eracy and high school equivalency, ment indices. This could best be done, Of the six states participating in the professional and technical degree and they suggested, by using information NGA project, Washington State ap- certificate programs, worker retraining, from the UI files, by conducting cus- pears to have done the most thinking and economic independence and long- tomer service surveys, and collecting about the requirements of an account- term employment. Significant out- additional information on benefits and ability model that utilizes scientific comes were the acquisition of academ- career ladders. But there were caveats. principles and methods that are now ic, occupational and basic workplace It is difficult to track employment over well established evaluation research ap- skills; the realization of sustained em- time as people move in and out of jobs. proaches. Although Washington is con- ployment and earnings; customer satis- Also, the value of retaining a given job tinuing to explore the issue of general faction; and reduced welfare dependen- may not be high if it translates to a lack benchmarks and measurable goals, ob- cy. of mobility. jectives and outcomes, no other state proposed the kinds of scientific sur- Iowa placed specific timeframes veys, process evaluations, and net im- around these system elements utilizing Learning Outcomes these guiding principles: a focus on the pact studies recommended by the Possession of a GED, a post-second- Washington workforce board. customer, an emphasis on system out- ary educational certificate, a license, comes, and a commitment to shared ac- or a degree are important indicators Mid-Project Progress countability, continuous improvement, of learning outcomes. greater collaboration in workforce de- In November 1994 the states partici- velopment planning and execution, and e Multiple measures are needed, how- pating in the NGA project met to report flexibility. ever, including "skill attainment", on their progress. The informal reports New York's benchmark-develop- "job placement" and "job retention." they shared were guided by NGA's pri- orities for covering the key elements of ment process consisted of three phases: There was unanimity about measur- performance management systems. In 1) planning, 2) data collection and ing skill attainment, but respondents general, these elements were the fol- analysis, and 3) reporting, building on acknowledged the difficulty of devising lowing: twenty key components derived from a measures in the absence of well-de- study of the research and business liter- fined credentials and licenses for most The definition of "accountability." ature, federal legislation, and other occupations. They were hopeful that The mission of the workforce sys- states' efforts. System goals gave spe- the National Skills Standards Board tem. cial attention to simplicity, quality and under Education Goals 2000 would re- accountability: The system's purpose. duce these problems. Some experts Efficient and comprehensive plan- suggested that the first step should be Long-term benchmarks. ning and policy direction. to define occupational and/or industry skills standards needed for a given oc- Shorter-term policy goals. Service integration. cupation and/or industry, and then al- The system's building blocks. Targeting of education and training low those standards to shape training The key features of the system. services to customers. curricula and programs. Its participants and stakeholders. Responsiveness to world-class edu- Project States' Accountability cational and training standards. Concepts Its set of roles and responsibilities: federal, state and local. Resource allocation tied to the prior- The states that applied for and were itization of activities and their dem- The target groups for service. selected to participate in NGA's Perfor- onstrated effectiveness. mance Management Project proposed The larger environment of the sys- North Carolina's report outlined the similar concepts and tasks consistent tem. workforce system's benchmarks, strate- with NGA's framework. However, The outcomes expected of the sys- gic policy recommendations, policy there was a range of responses to the tem, given its goals. RFP related to the unique situation of

Evaluation Forum Issue 11 Summer 1995 131 140 Evaluation Issues and Activities . At the State and Local Level

Figure ? har,46.teristics fAp!ifiQñ 514bolittect by State's. Sélecteci tor t1GA Project.. Illinois Previous Accomplishments Technical Assistance Requested :A Task Force on Human Resource De- A Establishment of subcouncil to define Identification of models for support- 4 velopment developed 7 goals: broad occupational clusters and skill ing continuous program improve- To develop comprehensive early standards applicable across indus- ment. childhood development system. tries. identification of models for develop- To establish standards for educe- Development of educational guaran- ing system-wide definitions, mea- tional performance. tees for statewide community college sures, and incentives/sanctions. To institute comprehensive school- system. Development of strategies for ob- to-work transitien system. Development of Inventory of Ac- taining better data sources, coordi- To develop s% stem giving assis- countability Measures containing 35 nating automated data systems and tance to out-of-school youth and measures for judging students' out- networks, and sharing data. disadvantaged adults failing to comes. Identification of strategies for as- meet educational standards. Development of operational frame- sessing program participants' skills oTo develop system for awarding work for using performance mea- across different assessment instru- technical/professional certificates/ sures/standards and evaluation strat- ments. degrees based on mastery of skills. egies within a continous program Development of performance mea- improvement system for secondary To urge employers to invest in re- sures for use at the local level con- and postsecondary vocational educa- training as major strategy for high sistent with state measures. tion programs. performance work organizations. Development of employment/earn- To provide access to coordinated ings measures for using Ul data to system of education, training, case judge program performance. management and supportive ser- vices for youth/adults facing em- Development of automated system ployment barriers. for tracking pre/post information on employment, earnings, Ul receipt and Development of coordination criteria job retention for all participants in involving all relevant agencies. state's workforce preparation pro- Requirement that single Compre- grams. hensive Workforce Preparation Establishment of state/local partner- Agreement be developed for each ship to achieve State Goals for service area involving vision and Learning. measurable goals and objectives. Agreement on twelve outcomes and Development of multi-agency, state- their measures, for use in improving wide School-to-Work Transition Sys- workforce preparation activities. tem.

Iowa Previous Accomplishments Technical Assistance Requested Creation of Target Alliance estab- o Stimulation of efforts to influence Development of benchmarking lished by governor and representing laws and regulations inhibiting col- process. six state agencies, to study the laboration and coordination. Identification of "best practices" in workforce preparation system & de- a Implementation of a collaborative developing quantitative measures velop a theme and general ideas for process to develop a comprehen- consistent with benchmarks. achieving a more integrated system: sive State Human Investment Poli- Consideration of fund search *Theme: greater attsntion to "hori- cy and appointment of a Work- strategies. zontal systemic connections" be- force Development Coordinator. tween programs and organizations Development of strategies and Development of funds for local work- methods for studying processes and to support local efforts to cross or- force development centers. ganizational territories in providing outcomes re: continuous program Development of outcome-based services. improvement. Ideas: School-to-Work /owa Transition model. Identification of performance Creation of Council on Human invest- 0Consideration of establishing an measures used at federal level and ment to set long-term human invest- overall workforce development by other states. ment goals and benchmarks, and to council representing all programs. conduct customer satisfaction surveys. Figure 2 Continued

132 Evaluation Forum Issue 11 Summer 1995 141 - -

Evaluation Issues and Activities At the State and Local Level

New York Previous Accomplishments Technical Assistance Requested

Governor's Plan for Coordination of mends assessment methods for as- identification of sources of informa- Workforce Development and NYS suring continuous program improve- tion for reducing barriers to the de- Strategy for Workforce Excellence ment. velopment and use of common out- support a comprehensive customer- ACCESS program provides compre- come definitions and definitions of oriented seMce system, skilled hensive educational and training/re- core competencies. workforce and high quality services. training services via one site using a identification of 'best practices' in Concepts involved are: common Client Assessment Frame- other states. *Development of measurable out- work. Assistance with waiver requests. comes for entire workforce system. GATEWAY initiative increases coor- Redesign of education/training sys- dination of goals/objectives across tem. workforce preparation programs and Enhancement of job development/ oreates network of coordinated ser- placement. vices in 24 sites, and provides infor- mation about 'best practices' in inte- Workforce Preparation Evaluation grating customer services. Act requires state agencies respon- sible for administering workforce Career Pathways School-to-Work preparation programs to prepare an- Opportunities initiatives test an inte- nual program plans, identify funding gration of workforce preparation with sources, and report actual program academic education. outcomes against stated program Education for Gainful Employment le- goals and objectives. verages additional JOBS funds to ex- . Educational community's New Com- pand the range of options for AFDC pact for Learning emphasizes prepa- clients regarding educational and . ration for meaningful work, sets training alternatives. learning standards, and recom-

North Carolina Previous Accomplishments Technical Assistance Requested Governor's Commission on Work- Literacy Task Force is developing Development of statewide goals and force Preparedness is responsible strategic state plan for increasing standards and program- level stan- for creating a coherent, comprehen- adult literacy, with assistance from dards, and the design of system- sive workforce development system the Corporation for Enterprise De- wide information systems to track with common vision and goals velopment. achievement. across all relevant programs. Is ex- *High Performance Task Force has Development of quality criteria for pected to develop a 4-year strategic produced report on characteristics self-assessment and other evalua- plan and a 2-year plan which in- of high performance firms, assisted tive purposes. cludes state workforce goals and ob- by a national consultant. jectives. The Commission has a +Welfare Reform Task Force is de- number of task forces that are re- veloping a customer and outcome- sponsible for different dimensions of focused strategic plan for reforming the workforce system: the state welfare system. School-to-Work Task Force is im- Interagency Coordinating Council plementing statewide business/ed- provides technical advice to the ucation initiative, JobReady, to cre- Commission on policy, needs, Infor- ate a school-to-work system, mation sharing and continuous pro- ' . including development of outcome gram improvement. Has developed a strategies such as certificates of model for a comprehensive workforce mastery and competency mea- preparation system, Virtual Work- sures with assistance from the force Preparedness Paradigm. University of Maryland's Center for Learning and Competitiveness.

Figure 2 Continued

133 Evaluation Forum Issue 11 Summer 1995 142 -

Evaluation issues and Activities . At the State and Local Level

North Carolina Previous Accomplishments Technical Assistance Requested Commission on Standards and Ac- the application of "total quality man- countability is setting new standards agement" throughout state govern- and skill measures for public school I ment. system. North Carolina has also developed a Commission for a Competitive North preliminary set of categories for out- Carolina is developing four-year stra- come and performance standards, tegic plan for economic development, and actual measures and standards a performance-based budgeting within this categories, across the en- strategy, and a plan for increasing tire system.

Texas Previous Accomplishments Technical Assistance Requested

The Texas Council on 'Workforce Development of uniform state-wide Quality improvement process. and Economic Competitiveness (an client application and eligibility de- Data system models. 1-1111C and a separate state agency) termination system for major work- was established to unify policy de- force development programs. integration of adult education. velopment, planning and evaluation Development of intake/assessment Waivers. of workforce development efforts. process that can distinguish be- The Council is responsible for devel- tween those for whom education oping a strategic state plan which and training are appropriate vs. ties funding to performance and in- those for whom basic labor market cludes goals, objectives and perfor- information, job search assistance mance measures for all workforce and placement are appropriate. development programs. The Council has an Evaluation and Development of state/local strategic planning, evaluating and account- Performance Committee and a num- ber of workgroups under it. Purpos- ability system to enable the state to es of the Council are to: assess results achieved and "return on investment", and to improve the Develop industry-based skills stan- quality of programs and their deliv- dards/certification system. ery systems. Develop comprehensive labor mar- Improvement in individuals' literacy, ket information system. basic education and basic work- Design school-to-work transition place skills, including the proportion process. completing secondary and post- *Create local workforce boards and secondary programs. service centers. Adoption of standard definitions and Develop statewide system for eval- levels of achievement of workplace uating the effectiveness and effi- basic skills, and incorporation of ciency of all workforce develop- these standards into education and ment programs. training programs. Texas Tomorrow is the current stra- Preparation of youth for meaningful tegic plan for the workforce prepara- work and life-long learning. tion system, developed by the Coun- 2 Current strategic plan has two parts: cil. Goals and objectives include: the first is on vision, philosophy, mis- Consolidation of workforce devel- sion, system goals, and program opment boards and creation of goals and objectives under each one-stop centers at local level. goal, and has been completed; the second is on core performance mea- sures which is in process.

OM. Figure 2 Continued

134 Evaluation Forum limo 11 Summer 1995 143 _ _

Evaluation Issues and Activities At the State and Local Level

. . . . ;Iv Continued

Texas Technical Assistance Requested Previous Accomplishments Related efforts: skill jobs and produce a quality Workforce Development Initiative workforce through coordinating ed- for Youth is the Texas school-to- ucation and training. work project and will include per- 4. The Texas Assessment ofAcadem- formance standards. ic Skills involves quantifiable educa- 4. The New Texas Prosperityis a gu- tional objectives. bernatorial eoonomic development Texas community colleges now op- effort. erate within performance standards. *The Smart Jobs Plan is a compre- hensive governor-sponsored strat- egy to develop high-wage, high-

Washington Previous Accomplishments Technical Assistance Requested Information about 'best practices' in Investment in Human Capital study OTo promote competency-based identified employer/employee needs education and training with equal other states. and organizational barriers in state's emphasis on acquisition of aca- Identification of competent consult- workforce development programs demic and occupational skills. ants. and evaluated the performance of o To leverage and generate addi- Federal waivers. these programs. It established a tional resources for an integrated Federal developments relevant to standard for how workforce systems, state-wide education and training workforce development systems. as well as programs, could be evalu- system. ated scientifically. Major recommen- Washington's role In the National dations from the study were: Conference of State Legislature's improve coordination. Jobs for the Future project: one of Increase accountabiiity. five states involved, focusing on Create a state coordinating board school-to-work transition, skills up- for workforce education and train- grading, and coordination of work- ing. force system. Creation of the Washington State Washington's new Performance Part- Education and Training Coordinating nership: brings governor, legislature, Board (WETCB) by the Washington state employees and private sector State Legislature, to be appointed by together to create a performance the governor. This board is responsi- management system for all elements of state government. Will focus on ble for designing an accountability system for workforce education, the development of measurable out- training and employment, as well as comes and continuous program im- for policy development, planning and provement. coordination re: a customer-driven WETCB's state workforce develop- system. Its vision and mission are: ment plan: establishes benchmarks Vision: "to develop a globally com- and measurable goals and objectives petitive workforce supported by an for workforce preparation system. accessible, flexible, competency- The Board has developed prelimi- based and technologically current nary Targets for Excellence for the education and training system". workforce system. Utilizing the NGA project, it has continued its analysis Mission: of issues surrounding the develop- a To establish new workforce part- ment of measurable system goals nership among labor, business, and performance standards, and education and government. the coordination of information sys- To improve interprogram coordi- tems. nation.

Figure 2 Continued

135 Evaluation Forum.Issue 11 Gummy 1995 144 Evaluation issues and Activities At tha State and Local Level

Fig Ure Contirwed

Washington Previous Accomplishments Technical Assistance Requested

*Accountability section within the six years on a range of issues re. new state workforce plan is consis- iated to workforce preparation. tent with Board's accountability 0 A survey of employers every six mandate: years. 0Assessment of state education O Surveys (and focus groups) to de- and training needs. termine customer satisfaction ev- o Establishment of an inventory of ery two years. programs. o Periodic conducting of a compre- o Evaluation of the outcomes of the hensive process and impact study workforce system. of programs within the workforce o Development of minimum stan- system: a process study of inter- dards for data collention and pro- program coordination, private sec- gram evaluation. tor participation and other issues 0 Development of reliable, valid and of interest; a net impact and cost/ accessible data base for the sys- benefit study of major new pro- tem. grams. Accountability section recommends Matching of participants' program the following: outcomes at approximately nine A number of scientific surveys: months and at five years beyond program participation, with Ul data DA state-wide household survey to determine earnings and obtain and data match with Ul data every other information.

136 Evaluation Forum Issue 11 Summer 190 145 Evaluation Issues and Activities r At the Stateand Local Level

for the NGA Prolect Figure3 Other:Characteristics of the States Selected

Measurable Goals, Organizational Arrangements Objecuivaa, and Outcomes Monitoring and Evaluation Illinois Will develop alternative models for Governor's Task Force on Human Will develop common set of measur- Resource Development is mandated able outcomes for each group, as achieving continuous program im- provement, and make recommenda- to shape integrated, performance- well as standards for each, which driven human resource system, so will tions. apply across all programs. provide the leadership re: NGA project. Updates, focus groups, public hearings, articles in agency newslet- ters will be used to update all relevant agencies, employers, public.

Iowa A state Workforce Development Will develop policy goals for work- Will develop a common performance management system for all programs. Council will be created, associated force preparation system, and with the Human Investment Policy benchmarks for achieving them for Will develop a method for monitoring Council. Meanwhile an existing inter- each population group, with the as- and judging performance across sys- agency policy team will be responsi- sistance of a formal Research tem, including customer satisfaction ble for the project, assisted by a Group from three state universities. and service quality. Technical Work Group. Will develdp measures to assist ser- m Will develop commonperformance Service providers, customers, com- measures and definitions for each vice providers in implementing con- munities and others will be invested population group, to be used at both tinuous program improvement. in the process through symposia the state level and at the level of lo- Will develop self-assessment tools. and other forums and thrrugh a cal workforce centers. Will develop an integrated automated Special Communications Group. information system to support inter- agency efforts. Will seek to identity 'best practices.'

Now York Will use Evaluation Act to support de- NY State Job Training Coordinating Will identify system-wide bench- Council will coordinate existing state marks for each population group us- velopment of common set of outcomes and strategies for continuous program workforce agencies, councils and re- ing outcome-based measures and lated organizations via a new project process (quality) standards, and re- improvement. team. lying on skills assessment/certifica- Will seek to pinpoint "best practices." tion system to be developed by Na- Will develop a "feedback loop" to In- tional Skills Standards Board under crease utilization of programitildata Goals 2000. for continuing program improvement. Will develop MIS to collect, store, Will develop economic incentives and and analyze outcome data (also Ul sanctions to support the use of analy- data). ses to improve outcomes. Will study barriers to implementing a comprehensive performance man- agement system. Will develop strategic plan for work- force preparation system specifying goals, benchmarks, objectives, mea- surable outcomes, and information system, drawing on US DOL's Core Data Elements and Common Defini- tions . Will design pilot project to test viabil- ity of concepts in the state plan. Figure 3 Continued

Evaluation Forum Issue 11 Summer 1995 Evaluation Issues and Activities At the State and Local Level

Measurable Goals, Objectives, and Outcomes Monitoring and Evaluation Organizational Arrangements North Caroline Will develop and define quantifiable Will focus on a number of elements state goals for each client group Governors' Commission on Work- of a quality assurance system, such force Preparedness (NC's HIRC) will across entire workforce preparation as continuous program improvement, system. have ultimate responsibility, butan program accreditation, and capacity expanded Interagency Coordinating Will identify both outcome-based building. Council will oversee NGA project. measures, definitions and stan- Will develop methods for tracking dards, and measures and definitions system progress in meeting state for assessing service delivery and goals. service quality for diverse clients. Will develop common set of perfor- mance standards for all programs in workforce system. Will rely on Secretary's Commission on Necessary Skills and Goals 2000 in defining basic skills and compe- tencies. Will recommend changes in legisla- tion and regulation re: waivers.

Texas Will refine performance measures Will link performance management for state plan goals and objectives. a Texas Council's Evaluation and Per- system with system-wide continuous formance Committee will oversee Will use the eight core performance program improvement and quality as- NGA project. measures developed in collaborative surance systems. Committee's Performance Measure- process by Texas Council on Work- Will develop evaluationprocess for force and Economic Competitiveness ment Workgroup will serve as inter- continuous program improvement agency policy group. as general framework for NGA project. and quality service delivery. Will view development and implemen- tation of performance management A system as six-year project involving: *Creation of inventory of current in- formation systems and consider- ation of ways to connect them. Development of plan for coordinat- ing information systems. *Reduction of barriers to coordina- tion/integration at all levels. 4 Formation of workgroups to devel- op implementation strategies.

4

Figure 3 Continued

138 147 Evaluation Forum Issue 11 Summar 1995 Evaluation Issues and Activities At the State and Local Level

Measurable Goals, Organizational Arrangements Objectives, and Outcomes Monitoring and evaluation Washington Will further develop and refine evalu- a WETCB iscoordinating the NGA a Will establishbenchmarks and mea- project. Its established Interagency surable goals, objectives, outcomes, ation system for judging both the workforce system's and its programs' Committee will play a major role, and performance standards. progress against goals andobjec- and a representative of governor's *Will further refine Targets for Ex- tives based on scientific principles office will link project with state's A cellence. and methods. new Performance Partnership. technical workgroup on accountability Will consider more thoroughly the Accountability section of new state will have operational responsibility. pros and cons of using perfor- workforce development plan will mance standards. serve as framework for furtherdevel- Data system coordination/integra- opment and refinement of methods tion: efforts will be focused on a for ensuring accountability. Special more coordinated and integrated consideration will be given to: workforce development information Using skills standards and compe- system, emphasizing: tency-based assessments as part of Common set of data elements and system-wide and program-specific definitions. evaluations. +Common reporting requirements. Using program evaluations to sup- Participant confidentiality protec- port continuous program improve- tions. ment and quality assurance. Reduction of data duplication. Developing reports for customers of Information portability. the workforce system (workers, em- Staff training needs. ployers and other stakeholders) to increase informed choice among State-of-the-art computer hardware education, training, and employ- and software needed. ment options.

139 Evaluation Forum Issue 11 Summer 1995 148 Evaluation Issues and Activities At the State and Local Level

goals, and more specific objectives. programs within the system, and the der over-arching state boards Accountability efforts were to focuson or coun- utilization of evaluation results forcon- cils predated this shift, however. This retooling the existing workforce,pre- tinuous quality improvement. The poli- movement was encouraged by the paring the emerging workforce, and cy outcomes focused on employment, building the workforce infrastruc- evolving positions of national employ- earnings, skill competencies, worker ment and training organizations and ture and on performance-based bud- productivity, the reduction of poverty, geting in the areas of social andeco- governors, and by incentives from the customer satisfaction, and the system's U.S. Department of Lahrras part of nomic well-being, corrections, health general return on the public's invest- and safety, economic development, the Secretary Reich's effon Move the ment. The Washington report advocat- employment and training system froma environment, justice, education, agri- ed the use of multiple measures for largely-fragmented unemploymentsys- culture, cultural resources, and general each of the major desired outcomes,to government. Measurable objectives tem to a more collaborative and inte- increase the precision and validity of grated reemployment system. were being developed for each area of outcome evaluations. interest, as well as outcome measures Motivated and funded quite differ- consistent with policy goals. Of partic- Some Comments on ently by these disparate supporters of ular concern were the quality of jobs, the NGA Project state-wide workforce initiatives,a com- employment retention, and povertyre- mon trend toward state authority and duction. Additional attention was being A number of issues are involved in innovation in employment and training given to strategies for strengthening the six states' efforts to developan ab- (with or without substantial federal political support and marketing thesys- stract system concept, and to describe oversight) has come to pass. The NGA tem. its potential implementation. Among project has demonstrated what them are these: states TheTexasreport focused on desired can accomplish on their own, with spe- system outcomes and quantitative mea- How do planners move logically and cial assistance from national experts, not only in developing workforce sys- sure for identifying them, based on five realistically from abstract missions, major system goals and a set of objec- benchmarks and goals, to morespe- tems but in evaluating them usingcom- tives under erch. The goals emphasized cific objectives and measurableout- petent social science principles and comes? methods. 1) the development of a statewidesys- tem supporting local workforce devel- Must all the outcomes be measur- opment centers and providing a net- able quantitatively? work of services and informationto customers, 2) the creation of a state/lo- How should "continuous improve- ment" and "quality assurance" be The NGA references for this article cal strategic planning and evaluation are listed below. They were working pa- defined, in order for planners tore- subsystem for judging the performance pers provided to participants of the Per- of the system, 3) the acquisition of alistically monitor and evaluate formance Management Project these attributes of systems? in June literacy, basic education and basic and July of 1994. workplace skills by customers, 4) the How much should be expected, Developing Comprehensive Perfor- acquisition of occupational skills by pragmatically, of government/pri- mance Management for the Work- customers that meet workplace require- vate sector partnerships in achieving force System: Project Description. ments for long-term sustaining employ- the more idealistic visions of work- Glossary of Project Terms. ment in high-skill, high-wage occupa- force systems promoted by the kind tional areas, and 5) the preparation of of system planning effortsencour- Issues in Developing Comprehensive youth through obtaining knowledge, aged in the NGA project? Performance Management for Work- skills and behaviors necessary for the force Development Systems. Should new workforce systems be transition to high-skill, high-wage Texas's Process for Setting Core Per- car- permitted to function for rers and through lifelong learning. De- a time be- formance Measures. fore crystallizing the outcomes to be sired outcomes included labor market, expected of them, and developing Attachment C of the report prepared learning, customer satisfaction, and performance standards? by the Texas Council on Workforceand system equity outcomes. Economic Competitiveness summariz- Washington explained the state's Although many such questionscan ing the results of a survey of national emerge from these impressive planning performance management specialists. performance management framework activities in states, the NGA project is in its report, as well as describingsys- tem policy goals and outcomes. This particularly important in the context of framework emphasized the intercon- the renewed political interest in budget nections among components of the balancing, greater state-level discretion, and the consolidation of workforce system: state-wide policy programs and service delivery. The movement goals, measurable outcomes, the data to de- system infrastructure, the outcomes of velop state-wide workforce systemsun-

140 Evaluation Forum Issue 11 Summer 15 etee

Evaluation !ssues and Activities Editorial Commentwy

tatnal, mmen ed program and service delivery coordi- their approaches, only minimal techni- ee!ttstrt::s f; set ate:Neel nation under the aegis of broadly repre- cal assistance has been available. States have long recognized that sentative state-wide boards or councils Therefore, each state's strategy for whose responsibility was to set policy, workforce coordination has been some- some workforce developmentpolicies and procedures have been conflicting or develop short and longeeterm plans, what idiosyncratic, beyond the necessi- Incompatible. This they knew was darn- promote coordination, and establish ac- ty to respond to state-specific needs aging service tivality, precluding inter- countabliity for all state workforce-relat- and interests. And accurate, detailed in- agency collaboration, and ;educingthe ed programs and cervioes. These enti- formation on the nature and status of ability of participants to move Into and ties were becoming the major tool for workforce coordination efforts across diverse states has been hard to come between peograms. Simultaneously, seeking better integration while reduc- states were appreciating the importance ing costs and making programs more by. Only time-limited, selective 'snap- of encouraging the development of accountable for achieving broad, cross- shots" were available, which artificially high-skill, high-wage workforces, which cutting goals. froze states' activities for a moment, ac- would demand a reorganization of the The emphasis was now on state tivities that were an important part of an workforce system. workforce systems, rather than on an evolutionary prooess that was constant- ly changing. To pull education and skill training increasingly unwieldy collection of pro- together better, many states introduced grams with diverse goals and regula- In the midst of continuing experi- educational reforms in the 1990s to tions. These systems were to involve mentation with coordination strategies, achieve a better linkage between aca- clear-cut benchmarks, goals, objectives there has been a strong steady move- demic knowledge and workforce skills, and performance indicators. Locating ment to ensure the "accountability" of integrat- stimulated by the National Education these tasks at the state level supported new more comprehensive and Goals established by the President and the decentralization of authority and re- ed workforce development systems. the governors in 1989. Some states cre- sponsibility launched in the 1980s. The methods for establishing such ac- ated new state boards or councils re- Such deeentralization was to permit countability have moved ever closer to sponsible for the full gamut of workforce the federal government to restructure it- evaluation research standards. The programs and special projects. Some self, and later to "reinvenr itself, as a NGA project is an example of this com- were moving toward coordinated, some- leaner, more coordinated, and more mitment to making professional judg- times integrated, information systems efficient body. ments of the results of system policies that would support these new statewide Decentralization has meant, howev- and activities, and the impact of system systems and their evaluation. er, that states have had few national interventions for workers and employ- ers. Obtaining such objective resultsIs The JTPA amendments spawned models or guidelines to assist them in pursuing workforce integration. Apart now an accepted input to comprehen- some of these human resourceboards. sive workforce policy development and Reflecting Congressional interest in co- from their own efforts to share informa- planning. ordination, these amendments promot- tion with other states and learn from

IMEMONNWILOI 141 Evaluation Forum Issueti Summer 1995 7 Resources

at° ri a 1)6'6 mainly onU.S.- specific issues; publica- double asterisk (*) those that deal tions providing Insights about other In the first more directly with worker dislocation part of this section of countries; and a special bibliography on and displacement Evaluation Forum we sample referenc- workforce development coordination is- es from the plethora of publications on sues. We have indicated with a single The second part of the Resources economic change, restructuring and asterisk (*) the publications in the first Section updates readers regarding worker dislocation. The list of useful two categories which deal mainly with some of the federal research conducted books, articles, reports and profession- general economic Issues, asa context since the last issue of the journal. This al papers Is loosely separated into for learning more about the context of Is a limited overview, based on there- three categories: publications focusing worker dislocation, and Indicated witha sponses of relevant federal agencies to the editor's request for information.

Referenceson Economic Change, Restructuring and WorkerDislocation Information about the United States *Abraham, K. and S. Houseman. Job **Baily, M. "Wages and Employment *Becker, G."Investment in Human Security in America: Lessons from Under Uncertain Demand" in Re- Capital: A Theoretical Analysis" in Germany. Washington, D.C.: The view of Economics and Statistics, Journal of Political Economy, 70, Brookings Institution, 1993. 41,1974. 1962. *Abraham, K. and L. Katz. "Cyclical *Baily, M. and A. Chakrabarti. Innova- **Bednarzik, R. and M. Lovell. U.S. Unemployment: Sectoral Shiftsor tion and the Productivity Crisis. Labor Market Adjustment Pro- Aggregate Disturbances?"in Jour- Washington, D.C.: The Brookings nal of Political Economy, 94,1986. grams. Paper prepared for the Presi- Institution, 1988. dent's Competitiveness Policy **Aho, C. and J. Orr. "Trade-Sensitive **Balderson, K. M. Plant Closings, Council, 1991. Employment: Who Are the Affected Layoffi., and Worker Readjustment. **Bednarzik, R. Trade-Sensitive U.S. Workers'?" in Monthly Labor Review Washington, D.C.: National Gover- 104,1981. Industries: Employment Trends and nors' Association, 1986. Worker Characteristics. Washing- **American Management Association: **Baldwin, S. Technology and Em- ton, D.C.: Bureau of International 1993 Survey on Downsizing and As- ployment Policy. Washington, D.C.: Labor Affairs, U.S. Department of sistance to Displaced Workers. New National Commission for Employ- Labor, 1991. York, 1993. ment Policy, 1987. **Bendick, M. The Role of Public Pro- *Ashenfelter, 0. "Estimating the Ef- **Barton, P. A Better Fit Between Un- grams and Private Markets in Reem- fects of Training Programs on Earn- employment Insurance and Retrain- ploying Workers Dislocated by Eco- ings," in Review of Economics and ing. Washington, D.C.: National In- nomic Change. Washington, D.C.: Statistics, 60,1978. stitute for Work and Learning, 1986. The Urban Institute, 1982. **Bailey, T. "Changes in the Nature *Bamow, B. The Education, Training **Benus, J. and R. Bymes. The St. and Structure of Work: Implications and Work Experience of the Adult Louis Metropolitan Reemployment for Skills and Skill Formation," Re- Labor Force from 1984 to 1995, Project: An Impact Evaluation. port of the Conservation of Human Washington, D.C.: National Com- Washington, D.C.: U.S. Department Resources. New York, NY: Colum- mission for Employment Policy, of Labor, 1991. bia University, Nnvember 1989. 1985.

Na.11011~ 142 1111111 Evaluation Forum Issua 11 Summar 1996 151 =7177

Dislocation Resources References on Economic Chan e Restructurin and Worker

** The New Jersey Unem- **Benson, D. Workforce Development: *Brooks, H. and L. Schneider. Poten- ployment Insurance Reemployment A Survey of Ohio Employers. Co- tial Impact of New Manufacturing Demonstration Project. Washing- lumbus, OH: Appropriate Solutions, Technology on Employment and ton, D.C.: U.S. Department of La- 1993. Work in Industrial and Developing Countries. Policy paper. Cambridge, bor, 1989. *Blackburn, M. and D. Bloom. The Ef- MA: John F . Kennedy School of * * Process and Imple- fects of Technological Change on Government, Harvard University, mentation Issues in the Design and the U.S. Distribution of Earnings 1985. Conduct of Programs to Aid the Re- and Income. Paper for Panel on employment of Dislocated Workers. Technology and Employment, 1987. **Bruno, L. Operating Effective Reem- ployment Strategies for Displaced Princeton, NJ: Mathematica Policy **Blalock, A.B. "Analyzing Worker Workers. Paper for U.S. Department Research, Inc., 1984. Displacement in the 1980s: The of Labor by CRS Incorporated, International Trade Landmark Reports of the Office of 1986. Technology Assessment," in Evalu- and Worker Dislocation: Evaluation ation Forum, Issue #7, Summer *Carnevale, A. America and the New of the Trade Adjustment Assistance Program. Princeton, NJ: Mathemati- 1992. Economy. Washington, D.C.: Amer- ican Society for Training and Devel- ca Policy Research, Inc., 1993. *Blinder, A. Paying for Productivity: A opment and the U.S. Department of **Creticos, P.A. and R.G. Sheets, Eval- Look at the Evidence. Washington, Labor, 1991. uating State-Financed Workplace- D.C.: The Brookings Institution, Based Retraining Programs. Wash- 1990. *Carnevale, A. and H. Goldstein. Em- ployee Training: Its Changing Role ington, D.C.: National Commission *Bloom, H. Estimating the Long-Run and An Analysis of New Data. for Employment Policy and the Na- Effects of Job Training Using Longi- Washington, D.C.: American Soci- tional Governors' Association, tudinal Earnings Data. Cambridge, ety for Training and Development, 1992. MA: John F. Kennedy School of 1986. *Cyert, R. and J. March. A Behavioral Government, Harvard University, Theory of the Firm. Englewood 1982. *Chinloy, P.T. and E. W. Stromsdorfer (Editors) Labor Market Adjustments Cliffs, NJ: Prentice-Hall, 1963. Back to Work; Testing Re- in the Padfic Basin. Boston, MA: *Dertonzos, M., R. Lester and R. So- employment Services for Displaced Klewer-Niuhoff, 1987. Workers. Kalamazoo, MI: W.E. Up- low. Made in America: Regaining john Institute for Employment Re- *Chubb, J. and R. Moe. Politics, Mar- the Productive Edge. Cambridge, MA: MIT Press, 1989. search, 1990. kets and America's Schools. Wash- ington, D.C.: The Brookings Institu- *Bluestone, B. and B. Harrison. The *Duncan, G. et al. Private Lives and tion, 1990. Great American Job Machine: The Public Policies: Confidentiality and Proliferation of Low Wage Employ- *Cole, R. Work, Mobility, and Partici- Accessibility of Government Statis- tics. A report by the Panel on Confi- ment in the U.S. Economy. Paper for pation. Berkeley, CA: University of dentiality and Data Access. National Joint Economic Committee, U.S. California Press, 1979. Research Council. Washington, Congress, 1986. *Committee for Economic Develop- D.C.: National Academy Press, ment: An America That Works: The The 1993. of America: Plant Closings, Com- Life-Cycle Approach to a Competi- munity Abandonment, and the Dis- tive Worlcforce, 1990. *Edwards, R. Rights At Work: Employ- ment Relations in the Post-Union mantling of Basic Industries. New **Cook, R. "Insights Concerning Suc- Era. Washington, D.C.: The Brook- York, NY: Basic Books, 1982. cessful Strategies in Dislocated ings Institution, 1993. *Boyd, W. and C. Kerchner (Editors). Worker Projects," in Evaluation Fo- *Ehrenberg, R. Labor Markets and In- The Politics of Excellence and rum, Issue #2. Choice in Education. New York, tegrating National Economies. **Corson, W. et al. An Impact Evalua- Washington, D.C.: The Brookings NY: Palmer Press, 1988. tion of the Buffalo Dislocated Work- Institution, 1995. *Brock, W. and R. Hormat. (Editors) er Demonstration Program. Prince- The Global Economy: America's ton, NJ: Mathematica Policy Role in the Decade Ahead. New Research, Inc., 1985. York, NY: W.W. Norton, 1990.

MN, 143 Evaluation Forum Issue 11 Summer 1995 152 Resources References on Economic Chane Restructurin and Worker Dislocation

**Pechter, A. Forecasting the Impact *Hirschhorn, L. Beyond Mechaniza- *Katz, L. and B. Meyer. "Unemploy- of Technological Change on Man- tion: Work and Technology in a ment Insurance, Recall Expecta- power Utilization and Displacement. Postindustrial Age. Cambridge, MA: tions, and Unemployment Out- Working paper. Washington, D.C.: MIT Press, 1984. comes," in Quarterly Journal of The Urban Institute, 1974. **Hollenbeck, K. Dislocated Worker Economics 105, 1990. **Fedrau, R. H. "Responses to Plant Human Capital Depreciation and *Kennedy, P. Preparing for the Twen- Closures and Major Reductions in Recovery. Working Paper. Kalama- ty-First Century. New York, NY: Force: Private Sector and Communi- zoo, MI: W.E. Upjohn Institute for Random House, 1993. ty-Based Models," in The Annals of Employment Research, 1990. the American Academy of Political *Kochan, T. and H. Katz. Worker Par- and Social Science, 475, 1984. **Horvath, F. W. "The Pulse of Eco- ticipation and American Unions: nomic Change: Displaced Workers Threat or Opportunity? Kalam4zoo, **Flaim, P. and E. Sehgal. "Displaced of 1981-1985," in Monthly Labor MI: W.E. Upjohn Institute for Em- Workers of 1979-1983: How Well Review 110, 1987. ployment Research, 1984. Have They Fared?" in Monthly La- bor Review 108, 1985. *Huang, W. (Editor) Organized Labor *Kochan, T. et al. The Transformation at the Crossroads. Kalamazoo, MI: of American Industrial Relations. *Fretwell, D. et al. Employment Dimen- WE. Upjohn Institute for Employ- New York, NY: Basic Books, 1987. sions of Economic Restructuring: A ment Research, 1989. Review of Related Labor Policies **Kulik, J., D. Smith and E. Stroms- and Programs in Industrialized *Flunt, H. and T. Hunt. "An Assess- dorfer. The Downriver Community Countries. The World Bank, 1991. ment of Data Sources to Study the Conference Economic Readjustment Employment Effects of Technologi- Program. Cambridge, MA: Abt As- *Graham, C. Safety Nets, Politics, and cal Change," in Technology and Em- sociates, Inc., 1984. The Poor. Washington, D.C.: The ployment Effects, report of the Panel Brookings Institution, 1994. on Technology and Women's Em- *Krugman, P. The Age of Diminished Expectations: U.S. Economic Policy *Guile, B.R. and H. Brooks. (Editors) ployment, National Research Coun- cil, 1985. in the 1990s. Cambridge, MA: MIT Technology and Global Industry: Press, 1990. Companies and Nations in the **Jacobson, L. et al. Earnings Losses World Economy. Washington, D.C.: of Displaced Workers. Working pa- *Lawrence, R. "Sectoral Shifts and the National Academy Press, 1987. per. Kalamazoo, MI: W.E. Upjohn Size of the Middle Class" in The Brookings Review, 3, 1984. **Hall, R. "The Importance of Lifetime Institute for Employment Research, Jobs in the U.S. Economy" in Amer- 1992. *Lawrence, R. et. al. A Vision for the ican Economic Review, 72, 1982. **Jensen, M. and J. Lederer: Workforce World Economy: Openness, Diversi- ty and Cohesion. Washington, D.C.: **Hall, R. and-E. Lazear. "The Excess in Transition: A Guide to the Coor- The l3rookings Institution, 1995. Sensitivity of Layoffs and Quits to dination of State Dislocated Worker Demand" in AmericanEconomic Services. Washington, D.C.: Center **Leigh, D. Retraining Displaced Review, 69, 1984. for Policy Research, National Gov- Workers: The U.S. Experience. ernors' Association, 1992. Geneva, Switzerland: International **Hamermesh, D. "The Costs of Work- Labour Organization, 1994. er Displacement," in Quarterly **Johnston, W. and A. Packer. Work- Journal of Economics 102, 1987. force 2000: Work and Workers for Retraining Displaced the Twenty-First Century. Indianap- Workers: What Can Developing *Hart, R.(Editor) Employment, Unem- olis, IN: .Hudson Institute, 1987. Countries Learn from OECD Na- ployment and Labor Utilization. tions? The World Bank, 1992. London, U.K.: Unwin Hyman, 1988. *Kasarda, J. The Regional and U,ban Redistribution of People and Jobs in *Leonard, J. Technological Change *Hartman, H. (Editor) Computer Chips the U.S. Paper for National Research and the Extent of Frictional and and Paper Clips: Technology and Council's Urban Policy Workshop, Structural Unemployment. Paper for Women's Employment. Washington, 1986. Panel on Technology and Employ- D.C.: National Academy Press, ment, 1986. 1987.

144 153 Evaluation Forum Issue 11 st Summar 1905 Resources References on Economic Chane,Restructurin and Worker Dislocation

**Levitan, S. and F. Gallo. A Second **National Commission for Employ- *Rivlin, A. et al. Beyond Alliances: Chance: Training for Jobs. Kalama- ment Policy: Workers Dislocated by Global Security Through Focused Partnership. John D. and Catherine zoo, MI: W.E. Upjohn Institute for Economic Change: Toward New In- Employment Research, 1990. stitutions for Midcareer Worker T. MacArthur Foundation and the Transformation. Washington, D.C., Rockefeller Foundation, 1990. *Levitan S. and D. Werneke. Produc- 1982. tivity: Problems, Prospects, and *Robertson, D. B. "Political Conflict Policies. Kalamazoo, MI: W.E. Up- ** Displaced Workers: and Lesson Drawing," in Journal of john Institute for Employment Re- New Options for a Changing Econo- Public Policy, 1991. search, 1984, my. Washington, D.C., 1983. *Rogers, E.M. The Diffusion of Innova- *Levy, F. Dollars and Dreams: The ** The Permanence of tions. New York, NY: The Free Changing American Income Disiri- Dislocation: 19794 983 Evidence. Press, 1983. button. New York, NY: Basic Washington, D.C., 1986. **Romero, C. et al. The Potential Ef- Books, 1987. Assisting Dislocated fectiveness of the Employment Ser- *Lindberg, L. and C. Maier. The Poli- Workers. Washington, D.C., 1991. vice in Serving Dislocated Workers Under EDWAA: Evidence from the tics of Inflation and Economic Stag- Upward Mobility Pro- 1980s. Washington, D.C.: National nation. Washington, D.C.: The grams in the Service Sector for Dis- Brookings Institution, 1985. Commission for Employment Poli- advantaged and Dislocated Work- cy, 1991. *Lipset, S.M. Unions in Transition: ers. Washington, D.C., 1992. Entering the Second Century. San **Secretary of Labor's Task Force on *New York Stock Exchange: U.S. In- Economic Adjustment and Worker Francisco, CA: Institute for Contem- ternational Competitiveness. porary Studies, 1986. Dislocation: Economic Adjustment *Nothdurft, W.E. Going Global: How and Worker Dislocation in a Com- *Litan, R. et al. (Editors) American Europe Helps Small Firms Export. petitive Society, 1986. Living Standards: Threats and Washington, D.C.: The Brookings *Sleigh, S. (Editor) Economic Restruc- Challenges. Washington, D.C.: The Institution, 1992. Brookings Institution, 1988. turing and Emerging Patterns of In- *President's Commission on Industrial dustrial Relations. Kalamazoo, MI: *Magaziner, I. and R. Reich. Minding Competitiveness: Global Competi- W.E. Upjohn Institute for Employ- America's Business. New York, NY: tion: The New Reality. Washington, ment Research, 1993. Harcourt Brace Jovanovich, 1982. D.C.: U.S. Government Printing Of- **Stromsdorfer, E.W. "Plant Closings **Mangum, S. and J. Tansky. Self Em- fice, 1985. and Worker Displacement," in Eval- ployment Training As An Interven- "Podursky, M. and P. Swaim. "Job uation Forum, Issue #7, Summer tion Strategy for Displaced or Dis- Displacement and Earnings Loss: 1993. advantaged Workers. Management Evidence from the Displaced Work- and Human Resources Faculty, Ohio *Summers, L. Understanding Unem- er Survey," in Industrial and Labor ployment. Cambridge, MA: MIT State University, 1987. Relations Review, 41, 1987. Press, 1990. *Mincer, J. Schooling, Experience, and *Reich, R. Education and the Next *4' Swigart, R.(Editor) Managing Plant Earnings. New York, NY: National Economy. Washington, D.C.: Na- Bureau of Economic Research, Closures and Occupational Read- tional Education Association, 1988. justment. Washington, D.C.: Nation- 1974. The Work of Nations: al Center for Occupational Readjust- *Mishel, L. and R. Teixeira. The Myth Preparing Ourselves for 2 Ist Centu- ment, 1984. of the Coming Labor Shortage: ry Capitalism. New York, NY: Vin- *Tan, H. Technical Change and Its Jobs, Skills and Incomes of Ameri- tage Books, 1992. ca's Workforce 2000. Paper of the Consequences for Training and Economic Policy Institute. Washing- *Rivlin, A. Reviving the American Earnings. A policy paper, Santa Monica, CA: The Rand Corporation,- ton, D.C., 1990. Dream: The Economy, the States, and the Federal Government. Wash- 1987. **Moore, R. and D. Blake. Does ETP ington, D.C.: The Brookings Institu- Training Work? An Analysis of the tion, 1992. Economic Outcomes of California Employment Training Panel Pro- grams, Northridge, CA: California State University, 1992.

145 Evaluation Forum NIssue 11 Swine 1995 Resources References on Economic Change,_ Restructuring and Worker Dislocation

*Temin, P. Lessons from the Great De- *U.S. Departments of Defense, Com- pression. Cambridge, MA: MIT merce and Energy: NASA and the Information About Press, 1989. National Science Foundation. De- Other Countries *Thurow, L. "A Surge in Inequality" in fense Technology Conversion, Rein- Scientific American, 256, 1987. vestment, and Transition Assistance. Books and Articles Washington, D.C. 1993. Dangerous Currents: **Abraham, K. and S. Houseman. Job The State of Economics. New York, **U.S. Department of Labor: Employ- Security in America: Lessons from NY: Random House, 1983. ment and Training Administration. Germany. Washington, DC: The Economic Adjustment and Worker Brookings Institution, 1993. **Topel, R. "Inventories, Layoffs, and Dislocation in a Competitive Soci- the Short-Run Demand for Labor" Earnings Inequality in ery. Washington, D.C., 1986. Germany. Kalamazoo, MI: W.E. in American Economic Review, 72, ** 1982. Study of the Implemen- Upjohn Institute for Employment tation of the Economic Dislocation Research, 1994. *Ulman, L. et al. (Editors) Labor and and Worker Adjustment Assistance Ayub, M. A. and S.O. Hegstad. Public an Integrated Europe. Washington, Act. Washington, D.C., 1992. D.C.: The Brookings Institution, Industrial Enterprises: Determi- 1993. **U.S. General Accounting Office: nants of Performance. The World please see Part II of the Features Bank, 1986. *U.S. Congress: Congressional Budget Section for GAO reports. Office. Economic Conversion: What *Bamber, G.J. and R. D. Lansbury (Ed- Should Be the Government's Role. *Vroman, W. and D. Wissoker. Alter- itors) International and Compara- Washington, D.C., 1980. natives for Managing Production tive Industrial Relations. London, ** Cutbacks. Report for the National U.K.: Allen and Unwin, 1987. Dislocated Workers: Is- Commission for Employment Poli- *Beauregard, R. (Editor) Econo-nic Re- sues and Federal Options. Washing- cy. Washington, D.C., 1990. ton, D.C., 1993. structuring and Political Responses. ** *Walton, R. "From Control to Commit- London, U.K.: Urban Affairs Annu- Displaced Workers: ment in the Workplace" in Harvard al Review, 1989. Trends in the 1980s and Implica- Business Review, 64, 1985. tions for the Future. Washington, *Best, M. The New Competition: Insti- D.C., 1993. *Walton, R. et al. Innovating to Com- tutions of Industrial Restructuring. pete: Lessons for Diffusing and Cambridge, England: Polity Press, *U.S. Congress, Office of Technology Managing Change in the Work- 1990. Assessment: Information Security place. San Francisco, CA: Jossey- *Blanchard, 0. et al. Reform in Eastern and Privacy in Network Environ- Bass, 1988. ments, 1994. Europe. Cambridge, MA: MIT *Waltman, J. and Studlar, D. (Editors) Press, 1991. ** Technology and Struc- Political Economy: Public Policies *Bosworth, B. and A. Rivlin. (Editors) tural Unemployment: Reemploying in the U.S. and Britain. Jackson, MI: The Swedish Economy. Washington, Displaced Adults. Washington, University Press of Mississippi, D.C.: The Brookings Institution, D.C., 1986. 1987. ** 1987. Trade Adjustment As- *Wilensky, H.L. and L. Turner. Demo- *Boyer, R. The Search for Labour sistance: New Ideas for an Old Pro- cratic Corporatism and Policy Link- gram. Washington, D.C., 1987. Market Fleribiliry. Oxford, England: ages: The Independence of Industri- Clarendon Press, 1988. ** Worker Training: Com- al, Labor-Market, Incomes, and peting in the New International Social Policies in Eight Countries. *Brunetta,R. and C. dell'Aringa (Edi- Economy. Report by the Advisory Berkeley, CA: University of Califor- tors). Markets, Institutions and Co- Panel on Workplace Training. nia Press, 1987. operation: Labour Relations and Washington, D.C., 1990. Economic Performance. London, England: Macmillan, 1990. U.S.-Mexico Trade: Pulling Together or Pulling Apart? *Chitty, C. Towards a New Education Washington, D.C., 1992. System: The Victory of the New Right7 London, England: Falmer, 1989.

mom...m.11W 146 155 Evaluation Forum Issue 11 Summer 1995 Resources Information about Other Countries

: New Technologies in the *Cline, W. International Economic *Helpman, E. and A. Razin. Interna- Policy in the 1990s. Cambridge, tional Trade and Trade Policy. 1990s. Paris, France, 1988. MA: MIT Press, 1994. Cambridge, MA: MIT Press, 1991. : Education and Training *Cukierman, A. et al. (Editors) Politi- *HeWitt, E. Reforming the Soviet Econ- After Basic Schooling. Paris, France, cal Economy, Growth, and Business omy: Equality Vs. Efficiency. Wash- 1985. Cycles. Cambridge, MA: MIT Press, ington, DC: The Brookings Institu- : OECD Jobs Study: 1992. tion, 1988. Facts, Analysis and Strategies. Par- **Elmeskov, I. and M. MacFarlan. *Hill, C. and I. Pickering. "Divisionalisa- is, France, 1994. "Unemployment Persistence" in tion, Decentralisation and Perfor- Pavlin, I. Human Resources Under the OECD Economic Studies, 21, 1993. mance of Large U.K. Companies," in Industrial Restructuring Paradigm in Journal of Management Studies, **Elmeskov. J. and K. Pichehnann. "In- Socialist Countries. Entrepreneurship 1986. Development Institute of India, 1987. terpreting Unemployment: The Role of Labour-Force Participation" in **Houseman, S. and K. Abraham. La- *Osterman, P. Employment Futures: OECD Economic Studies, 21, 1993. bor Adjustment Under Different In- Reorganization, Dislocation, and stitutional Structures: A Case Study Public Policy. New York, NY: Ox- *Finegold, D., L. McFarland, and W. of Germany and the U.S. Kalama- Richardson (Editors) Something ford University Press, 1988. zoo, MI: W.E. Upjohn Institute for Borrowed, Something Learned: The Employment Research, 1993. *Piore, M. and C. Sabel. 2'he Second Transatlantic Market in Education Industrial Divide. New York, NY: and Training Reform. Washington, Johnstone, D. Enterprise USA: Em- Basic Books, 1984. D.C.: The Brookings Institution, ployment and Training Through *Pirie, M. Privatisation. London, En- 1993. British Eyes. The Planning Ex- change, Glasgow, England. gland: Adam Smith Institute, 1985. **Finegold, D. "The Failure of British **Postlethwaite, N.J. "Research and Training: Analysis and Prescrip- *Keep, "The Grass Looked Greener," in Policy-making in Education" in T. tion," in Oxford Review of Economic International Comparisons of Voca- Husen and M. Kogan (editors) Edu- Policy, Autumn 1988. tional Education and Trainingfor In- termediate Skills edited by P. Ryan. cation Research and Policy: How *Fischer, G. and G. Standing. Structur- UK: Basingstoke, Falmer, 1991. Do They Relate? UK: Oxford Press/ al Change in Central and Eastern Pergamon, 1984. Europe: Labour Market and Social *Kochan, R., H. Katz, and R. McKer- **Prais, S.J. "Educating for Productivi- Policy Implications. Paris, France: sie. The Transformation of Ameri- ty: Comparisons of Japanese and OECD, 1993. can Industrial Relations. New York, NY: Basic Books, 1986. English Schooling and Vocational *Freeman, C. Technical Change and Preparation," in National Institute Full Employment. Oxford, England: **Mehaut, P. The Challenge of Compe- Economic Review, February 1987. Basil Blackwell, 1987. tence and Further Training Policies. Nancy, France: G.R.E.E., University **Prais, S.J. and K. Wagner. "School- *Gorbachev, Mikhail. Perestroika: New of Nancy, 1994. ing Standards in England and Ger- Thinking for Our Country and the many" in National Institute Eco- World. New York, NY: Harper and Michalski, A. and H. Wallace. The nomic Review, May 1985. Row, 1988. European Community: The Chal- lenge of Enlargement. London, *Prokopenko, J. and I. Pavlin. Entrepre- *Grossman, G. and E. Helpman. Inno- England: Royal Institute of Interna- neurship Development in Public En- vation and Growth in the Global tional Affairs, 1992. terprises. Geneva, Switzerland: Inter- Economy. Cambridge, MA: MIT national Center for Public Enterprises Press, 1991. *Organization for Economic Coopera- in Developing Countries, International tion and Development: The Labour Labour Organization, 1991. *Grubb, D. and W. Wells. "Employ- Market in Poland. Centre for Coop- ment Regulation and Patterns of eration with the Economies in Tran- **Rose, J. and F. Pottier. Training and Work in EC Countries," in OECD sition. Paris, France: OCED, 1993. the Labour Market: The Use ofData Economic Studies, 21, 1993. in Decision-making. Berlin, Germany: : Education and the *Hashimoto, M. The Japanese Labor European Centre for the Development Economy in a Changing Society. Market in a Comparative Perspec- of Vocational Training, 1993. Paris, France, 1989. tive with the U.S. Kalamazoo, MI: W.E. Upjohn Institute for Employ- ment Research, 1990.

147 Evaluation Foam limo 11 Summar 1995 156 Resources Information about Other Countries

*Rose, R. "What's New in Youth Un- *U.S. Office of Technology Assess- *National Commission for Employ- employment?" in University of ment: U.S.-Mexico Trade: Pulling ment Policy: The Employment Ef- Strathclyde's Studies in Public Poli- Together or Pulling Apart? OTA fects of European Economic Inte- cy, 177, 1989. Report Brief. Washington, D.C., gration: Background Study. *Scott, B. and G. Lodge (Editors) U.S. 1992. Washington, D.C., 1993. Competitiveness in the World Econ- : Competing Economies: *Organization for Economic Coopera- omy. Bu.ton, MA: Harvard Business America, Europe, and the Pacific Rim. tion and Development: The OECD School Press, 1985. OTA Brief. Washington, D.C., 1991. Jobs Study: Facts and Analysis **Schoppa, L. Education Reform in Ja- Willeval, M. "Labour Market Restruc- Strategies. Paris, France: OECD, pan. London, England: Routledge, turing and Deprivation Processes," 1994. 1990. in Labour and Society, 16, 1991. *Reis, F., F. Rychener. Training and *Sengenberger, W., G. Loveman and "Williams, S. Youth Without Work: the Labour Market: The Use of Data M. Piore (Editors) The Re-Emer- Three Countries' Approach to the in Decision-Making. Nancy, France: gence of Small Enterprises: Indus- Problem. Paris, France: OECD, 1981. University of Nancy, Research trial Restructuring in Industrialised Group on Education and Employ- *Wood, S. The Transformation of Countries. International Institute for ment, 1993. Work? Skill, Flexibility and the La- Labour Studies. Geneva, Switzer- land, 1994. bour Process. London, England: Additional Articles and Book Unwin Hyman, 1989. Chapters of Potential Interest "Senker, P. International Competi- Elmeskov, J. and K. Pichelmann. "In- tion, Technical Change and Train- Walking Papers and ing. Papers in Science, Technology Remsarch Reports terpreting Unemployment: The Role of Labour Force Participation," in and Public Policy. University of *Abraham, K. and S. Houseman. Earn- OECD Economic Studies. Paris, "* Sussex, England, 1988. ings Inequality in Germany. A staff France: OECD, 1993. *Steenbergen, J. Change and Adjust- working paper. Kalamazoo, MI: Elmeskov, J. and M. MacFarlan. "Ur ment: External Relations and Indus- W.E. Upjohn Institute for Employ- - trial Policy of the European Com- ment Research, 1994. employment Persistence," in OECD munio,. Boston, MA: Kluwer Law Economic Studies. Paris, France: "Bennet, R. et al. Training and Enter- OECD, 1993. and Taxation Publishers, 1983. prise Councils (TECs) and Voca- Grubb, D. and W. Wells. "Employment *Steiner, J. European Democracies. tional Education and Training: Regulation and Patterns of Work in White Plains, NY: Longman Pub- Conference Papers. London, EC Countries," in OECD Economic lishers, 1994. c' England: London School of Eco- nomics, July 1989. Studies. Paris, France: OECD, 1993. *Storey, D. and S. Johnson. Small and Houseman, S. "Job Security v.Labor Medium-Sized Enterprises and Em- *Houseman, S. and K. Abraham. Labor Market Flexibility: Is There a ployment Creation in the EEC Adjustment Under Different Institu- Tradeoff?," in Employment Re- Countries. Brussels, Belgium: Euro- tional Structures: A Case Study of search. Journal of the W.E. Upjohn pean Commission, 1987. Germany and the United States. A staff working paper. Kalamawo, Institute. Volume 1, No. 1, Spring "The Training Agency: Training and MI: W.E. Upjohn Institute for Em- 1994. Enterprise Councils: A Prospectus ployment Research, 1994. for the 1990s. London, England: Sengenberger, W. The Role of Labour Training Agency, 1989. "Leigh, D. Retraining Displaced Standards in Industrial Restructur- Workers: What Can Developing ing: Participation, Protection and *Turner, L. Democracy At Work: Countries Learn from OECD Na- Promotion. Discussion Paper No. Changing World Markets and the tions? Washington, D.C.: Popula- 19. International Institute for Labour Future of Labor Unions. Ithaca, NY: tion and Human Resources Depart- Studies, 1990. Cornell University Press, 1991. ment, The World Bank, 1992. Villeval, M. "Labour Market Restruc- *Ulman, L, B. Eichengreen, and W. "Mehaut, P. The Challenge of Compe- turing and Deprivation Processes," Dickens (Editors) Labor and an In- tence and Further Training Policies. in Labour and Society. Journal of tegrated Europe. Washington, DC: Nancy, France: University of Nancy, the International Labour Organisa- The Brookings Institution, 1993. Research Group on Education and tion, 1991. Employment, 1994.

148 157 Evaluation Forum Issue 11 Summer 1998 - -

Resources State-Level Woridorce Coordination, Integration and Accountability

Brizius, J. and M. Campbell. Getting Ganzglass, E. (Editor). Excellence at State-Level Results: A Guide for Government Work Washington, D.C.: National Workforce Accountabiliity. Washington, D.C.: Governors' Association, 1992. Coordination, Council of Governors' Policy Advi- Gardner, S. Roles for General Purpose Integration and sors, 1993. Governments in Service Integration. Carter, R. The Accountable Agency. Rockville, MD: Project SHARE, Accountability Beverly Hills, CA: Sage, 1983. 1976. Agranoff, R. "Human Services Admin- Condelli, L. and M. Kutner. Quality In- Henton, D. The Feasibility of Services istration: Service Delivery, Service dicators for Adult Education Pro- Integration: An Evaluation Pre- Integration and Training" in Human grams: Lessons Learnedfrom Other pared for the HEW Interagency Ser- Services Integration edited by T.J. Programs. Report to the Office of vices Integration Project. R&D Task Mikulecky. Washington, D.C.: Vocational and Adult Education; Force, 1975. American Society for Public Admin- U.S. Department of Education. Institute for Educational Leadership: istration, 1974. Washington, D.C.: Pelavin Associ- An Overview of Skill Standards in ates, Inc., 1992. American Evaluation Association: Education and Industry. Washing- Guiding Principles for Evaluators. Council of Governors' Policy Advi- ton, D.C., 1993. Report of a Task Force on Evalua- sors: Getting Results (r,, 3. Brizius Hallman, H.H. Pilot Comprehensive tion Standards, 1994. and M. Campbell). Washington, Manpower Program: Implications D.C., 1991. American Society for Quality Control: of Its Experience for Local and State Quality Systems Model for Quali- Experimerlis in Systems Manpower Organizations. Washing- ty Assurance in Design/Develop- Change (by J. ChynowQ.h, et al.). ton, D.C.: Center for Government ment, Production, Installation and Final report to The Ford Foundation Studies, 1973. Servicing, and Guidelines for Using and United Way of America. Wash- Hansen, LS. Preparing for the Work- Quail°, Assurance Principles in Ed- ington, D.C., 1992. place: Charting a Course for Post- ucation and Training. Milwaukee, Council of State Governments: Human secondary Training Policy. Policy WI, 1991. Services Integration: State Func- paper. Committee on Postsecondary Baj, J., R. Sheets, and C. Trott. "Im- tions in Implementation. Washing- Education and Training for the proving the Performance and Coor- ton, D.C., 1974. Workplace, National Research Council. Washington, D.C.: Nation- dination of Workforce Preparation Crawford, D., A. Johnson and A. Sum- al Academy Press, 1994. Programs for Youth Through the In- mers. Schools and Labor Market tegration of Federal-State Perfor- Outcomes. Philadelphia, PA: Na- Hoachlander, G. and M. Rahn. Perfor- mance Standards Systems," in In- tional Center on the Educational mance Measures and Standards for vesting in Youth. Washington, D.C.: Quality of the Workforce, Universi- Vocational Education: 1991 Survey National Governors' Association, ty of Pennsylvania, 1993. Results. Berkeley, CA: National 1992. Center for Research in Vocational DeWitt, J. Managing the Human Ser- Education (NCRVE), 1992. Barnow, B. "The Effects of Perfor- vice "System:" What Have We mance Standards on State and Local Learned from Services Integration? Ink, D. "The Origins and Thrusts of the Programs," in Evaluating Welfare Project SHARE, Human Services New Federalism," in The Adminis- and Training Programs edited by C. Monograph, Series 4, 1977. tration of the New Federalism: Ob- Manski and I. Garfinkel. Cambridge, fectives and Issues edited by L.H. Friedlander, D. Subgroup Impacts and MA: Harvard University Press, Grosoenick. Washington, D.C.: Performance Indicators for Selected 1992. American Society for Public Admin- Welfare Employment Programs. istration, 1973. "Thirty Years of Changing New York, NY: MDRC, 1988. Federal, State and Local Relation- InterstatuConference of Employment Gans, S.P. and G.T. Horton. Integra- ships in Employment and Training Security Agencies: Building An Ef- tion of Human Services, New York, Programs," in Publius, 23, Summer fective Workforce Development Sys- NY: Praeger Publishers, 1975. 1993. tem. A discussion paper. Washing- ton, D.C., April 1993,

Evaluation Forum lssul 11 Summor 1995 149 158 Resources State-Level Workforce Coordination Intration and Accountabir

Jarosik, D. and L. Phelps. Empowering National Association of Counties: Hu- Ruttenberg, S. H. and J. Gutchess. Accountability for Vocational-Tech- man Services Integration at the Manpower Challenge of the 1970s: nical Education. Berkeley, CA: Community Level: A Six County Re- Institutions and Social Change. Bal- NCRVE, 1992. port. Washington, D.C.: Research timore, MD: Johns Hopkins Press, Jennings, E.T. and N.S. Zank. Welfare Foundation, 1974. 1970. System Reform: Coordinating Fed- National Association of State Job Sanders, J. and the Joint Committee On eral, State and Local Public Assis- Training Coordinating Council Standards For Educational Evalua- tance Programs. Westport, CT: Chairs: Bring Down the Barriers: tion: The Program Evaluation Stan- Greenwood Press, 1993. Policy Recommendations. Washing- dards. Thousand Oaks, CA: Sage Kahn, A. J. Social Policy and Social ton, D.C., 1993. Publications, 1994. Services. New York, NY: Random National Commission for Employment Sheets, R. Building a National Skill House, 1973. Policy: Developing Effective .ITPA Standards and Certification System King, C. Cross-Cutting Performance Performance Standalls Incentive in the U.S.: The Role of Existing Management Issues in Human Re- Policies. Washington, D.C., March Public and Private Occupational source Program. Research report. 1990. Regulation Systems. Paper for Na- Washington, D.C.: National Com- National Governors Association: De- tional Center on Education and the mission for Employment Policy, signing Program Evaluations (by L. Economy. Washington, D.C., 1993. 1988. Bilheimer). Center for Policy Re- Smith, G. and D. Stevens. Beyond Ac- Levitan, S.A. and J.K. Zickler. The search, Washington, D.C., 1989. countability. Denver, CO: Commu- Quest for a Federal Manpower An Action Agenda to Re- nity College of Denver, 1994. Partnership. Cambridge, MA: Har- design State Government. Reports Spencer, L. M. "Planning and Organiz- vard University Press, 1974. of the State Management Task Force ing Human Services Delivery Sys- Lien, C. A. Design for a Multi-Service Strategy Groups. 1993. tems" in Proceedings of a Seminar on Human Services Integration. Delivery System. A report prepared Building State Workforce :74 for the Washington State Depart- Development Sptems Based on Pol- Denver, CO: Social Welfare Re- ment of Social and Health Services icy Coordination and Quality Assur- search Institute, 1973. by Stanford Research Institute, ance (by J. Baj, R. Sheets, and C. Stevens, D. and 3. Shi. Beyond Perfor- 1973. Trott). 1994. mance Standards. Berkeley, CA: Linn, L.E. "Organizing Human Servic- Performance Management National Center for Research in Vo- es in Florida," in Evaluation, Num- Project: Glossary of Project Terms. cational Education, 1994. ber 3,1976. Resource Papers series. July, 1994. Sundquist, J.L. Making Federalism Lucas, W. et al. The 1975 Census of Performance Management Work: A Study of Program Coordi- Local Services Integration Projects. Project: Federal Core Data Ele- nation at the Community Level. A paper prepared for the U.S. De- ments and Common Definitions.Re- Washington, D.C.: The Brookings partment of Health, Education and sourcePapers series. (Draft report) Institution, 1969. Welfare, 1975. July 1994. U.S. Congress, Office of Technology Lucas, W.A. Aggregating Organiza- Performance Management Assessment: Report Brief for No- tional Experience with Services Inte- Project: Texas' Process For Setting vember 1993, reviewing OTA' s gration: Feasibility and Design. Core Performance Measures. Re- "Making Government Work: Elec- Santa Monica, CA: Rand Corpora- source Papers series. July 1994. tronic Delivery of Federal Services." tion, 1975. Washington, D.C. National Institute for L.teracy: A Guide Mangum, G. L. The Emergence of to Strategic Planning For Perfor- Wage Record Information Manpower Policy. New York, NY: mance Measurement, Reporting And Systems. Background paper, May Holt, Rinehart, and Winston, 1969. Improvement Systems. Washington, 1994. Miller, R. et al. Developing Client Out- D.C., February 1994. comes Monitoring Systems. Wash- ington, D.C.: The Urban Institute, 1985.

160 Evaluation Forum 159 Issue 11 Summer 1995 Resources State-Level Workforce Coordination Int ration and Accountabili

U.S. Department of Education, Nation- U.S. Department of Labor, Office of Great Lakes States al Center on the Educational Quality Inspector General: Services Integra- Sheets, It. and C. Trott. Creating The of the Workforce: Crosswalk ofNa- tion: A Twenty-Year Retrospective. Guarantee: A Great Lakes Work- tional Data Sets Focusing on Work- Washington, D.C., January 1991. force Development Strategy. Center er Training and Statistical Compan- Employment and Training for Governmental Studies, Northern ions to the Crosswalk of National Administration: Coordination of Illinois University, 1993. Data Sets Focusing on Worker Housing and Job Training Services: Council of Great Lakes Governors: Training. Washington, D.C., June A Review of Best Practices in 12 New Directions in Great Lakes and April 1992. Cities. Washington, D.C., 1992. Workforce Development Policy. Au- Creating Responsible and Employment and Training gust 1993. Responsive Accountability Systems. Administration: Summary of State Report of the Office of Educational Reports on Coordination Between lowa Research and Information. Washing- Vocational Education and JTPA. Iowa Target Alliance: Integrated Em- ton, D.C.: U.S. Government Printing Washington, D.C., 1993. Office, 1988. ployment and Training Management Zank, N. Coordinating Federal Assis- Information System. November U.S. Department of Health and Human tance Programs for the Economical- 1992. Services: A Review of the Conceptu- ly Disadvantaged. Washington, al Foundations and Current Status D.C.: National Commission for Em- Indiana of Services Integration. Office of ployment Policy, 1991. Human Development Services, Indiana Workforce Legislation, 1993. 1983. Zornitsky, J., H. Bloom et al. Establish- ing a Postprogram Performance Indiana Department of Workforce De- .S. Department of Labor: An Assess- Management System for Employ- velopment: Workforce Development ment of the JTPA Role in State and ment and Training Programs. Cam- Summary, 1993. Local Coordination Activities. bridge, MA: Abt Associates, Inc., Washington, D.C., 1991. 1985. Maine U.S. General Accounting Office: Wel- Maine Human Resource Development fare Simplification: Projects to Co- References for the Coordination Efforts of Individual States Council: Second Annual Maine Hu- ordinate Services for Low-Income man Resource Development Plan. Families, 1986. Connecticut July 1989. Integrating Human Ser- Connecticut Employment and Training Report on the Status of the vices: Linking At-Risk Families With Commission: Connecticut Employ- Governor's Second Human Re- Services More Successful Than Sys- ment and Training Commission: source Development Plan, July-De- tem Reform Efforts. September People, Employment and Training, cember 1989. March 1990. 1992. 1992. U.S. Department of Health, Education Human Resources Invest- Massachusetts and Welfare: Integration of Human ment Goals for the 21st Century. National Commission for Employment Services in HEW: An Evaluation of July 1993. Policy: Assessing State-Level Job Services Integration Projects. Social Training Coordination: A Survey and Rehabilitation Services, 1972. Florida Design and Methodology Based U.S. Department of Housing and Urban Upon the Massachusetts Experience. Florida State Job Training Coordinat- Development: Coordinating Federal Washington, D.C., 1992. ing Council: Forum on Enhancing Assistance in the Community: Use of Florida's Economy Through Job National Governors' Association: Selected Mechanisms for Planning "Building the Capacity of Regional and Coordinating Federal Pro- Training Workforce 2000 Confer- ence. A White Paper, 1993. Employment Boards in Massachu- grams. Community Development Evaluation Series No. 8. Washing- setts" (by S. Spinner) in Labor Notes, 80, January 1993. ton, D.C.: Government Printing Of- fice, 1972.

Evaluaiion Forum Issue 11 Summor19g5 Resources State-Level Workforce Coordination intration and Accountabili

Michigan Oregon Issue Paper on Accountability. January, Michigan Job Training Coordinating Oregon Workforce Quality Council: 1994. Council: Creating A Human Invest- Shared Information System. May Patterns Emerging from the Research ment System, 1989. 1992. on Dislocated Worker Programs. Michigan Human Investment Fund and Information System: Sum- Spring 1994. Michigan Job Training Coordinating mary (by T. Lynch and J. Westine), Information from Research on Dislo- Council: Michigan Opportunity Sys- 1992. Oregon Department of Edu- cated Workers: Some Beginning tem: Report fi-om Mackinac Island. cation: Workforce Development in Policy Issues. Spring 1994. October 1989. Oregon 1988-1994. "State Efforts to Assure Accountability Michigan Human Investment Fund: in Comprehensive Workforce Sys- Michigan Human Investment Sys- Texas tems: With Insights from Other tem. Working paper. March 1989. National Governors' Association: Countries." Spring, 1994. "Quality Workforce Planning in New Jersey Texas: A Market-Driven Initiative" Washington Training and New Jersey State Employment and (by J. Boyd) in Labor Notes, 80, Education Coordinating Board: Training Commission: Annual Re- January 1993. Major Publications port, 1992,1993. References for Washhigton State Workforce Development Directory: A Unified State Plan for Coordination Efforts Workforce Preparation Programs New Jersey's Workforce Readiness and Services in Washington State. System, 1992. Washington Training and March 1993. Education Coordinating Board: Working Papers High Skills, High Wages: Washington's New York Comprehensive Plan for Workforce Some Beginning Ideas About Establish- Training and Education. September New York State Job Training Partner- ing Accountability: Oversight Issues 1994. ship Council: Creating a Vision: and Policies. Winter, 1994. The Workforce Preparation System of the Future. February 1990. Some Beginning Thoughts About What Office of the Governor of the Helps Define Accountability. Win- State of Washington: Office of New York State's Invest- ter, 1994. Financial Management ment in a Quality WorlOrce: A Cat- alog of Workforce Preparation and Thinking About Accountability: Making Evaluation of Training Programs. Related Programs, 1993. Assessments of Activities and Pro- Working paper. Forecasting Divi- grams. Winter, 1994. sion, 1993. Worltforce Preparation Evaluation Act. February 1990. Thinking About Accountability: Devel- oping Policy Questions to Anticipate Washington State Accountability Issues and Needs. Superintendent of Public North Carolina Winter 1994. Instruction: Assessment and Governor's Commission on Workforce k School Information Division Steps on the Way to Establishing Pro- Preparedness, 1992. Proposed Educational Outcomes and gram Accountability. Winter 1994. Measurement System, 1993.

152 161 Evaluation Forum !Me 11 Summer 1995 Resources Federal Research: An Update

Federal Research: An Update

..Deprtmont.of Labor

Reports Completed In 1994 Evaluation of the EDWAA Job Creation The School-to-Work/Youth Apprenticeship A Guide to Well-Developed Services for Demonstration. Demonstration: Preliminary Findings. Dislocated Workers. Evaluation of the Defense Conversion Ad- From the Farm to the Job Market: A Guide Employment and Training for America's justment Demonstration: Interim Report to Employment and Training Services Homeless: Report on the Job Training on Implementation. for Farmers and Ranchers. for the Homeless Demonstration Pro- gram. Service to Dislocated Farmers: An Evalua- Training and Employment Report of the tion of the EDWAA Farmers and Ranch- Secretary of Labor. Evaluation of the JTPA Title IV Migrant ers Demonstration. and Seasonal Farmworker Program.

Buro'n4'of 141)010 leticsf:Oftlee Edommlc:Research

Retirement in a Family Context: A Structur- Transitions from School-to-Work: A Survey al Model for Husbands and Wives. of Research Using the National Longitu- dinal Surveys.

Selected Projects In Progress

An examination of School-to-Work Transi- Education, Experience and Employer Incentive Pay, Earnings and Information: tion in the NLSY. Learning about Worker Productivity. Evidence from the NLSY. Wages, Fringe Benefits and Savings: Inter- High School Employment: Consumption of Technological Change and On-the-Job actions and Implications for the Determina- Investment. Training of Young Workers: The Intergen- erational Consequences of Family Structure tion of Labor Market Outcomes Analysis Labor Market Activity and Earnings: A on Wages and Educational Attainment. with the NLS. Comparison of NLSY and SIPP.

ureat; of Inte on ILaborfAffiltreFortgkRoo onilc ReS sic -"Taira o Av;*: Selected Recent Reports

The Labor Market Impacts of NAFTA. Employment Dimensions of Economic Re- Changer in the Industrial Structure ofJob structuring: A Review of Related Labor Displacement: Evidence from the Dis- U.S. Employment Effects of a North American Policies and Programs in Industrialized placed Worker Surveys. Free Trade Agreement: A Survey of Issues Countries. and Estimated Employment Effects. Labor Market Adjustment and Job Dis- Evaluations of Programs in Industrialized placement. Returns to Classroom Training of Dis- Countries to Assist Workers Displaced placed Workers. The Economic Consequences of Jobless by Structural Change. Duration for Displaced Workers. Is Job Stability Declining in the U.S. Econ- Structural Change and Labor Market Ad- omy? Guidelines for Assisting Trade Displaced justment: A U.S.-Japan Comparison. Workers. Analysis of the Dislocated Workers' Educa- Competition and Complementarily between Designing Trade Adjustment Policies tional Training Program. U.S. Imports from Development and U.S. Labor Market Adjustment Programs. Newly Industrializing Countries. Does Training Work for Displaced Work- ers? Trade-Sensitive U.S. Industries: Employ- The Wage Structure by Occupation: Skill ment Trends and Worker Characteris- Level and Skill. Labor Adjustment Policies and Practices in Europe. tics. The Impact of Technology, Trade and Out- The Quality of U.S. Jobs. sourcing on Employment and Labor Human Resources and Corporate Policy. Composition. Labor-Management Conflict and Coopera- tion.

163 Evaluation F0fUtn Issue 11 Summer 1025 162 Resources Federal Research: An U. ate

.S. Department of Health and Human Services

JOBS Evaluation: Early Lessons from Sev- Saturation Work Initiative Model in San Di- Summary of Final Evaluation Findings en Sites. ego: A Five-Year Follow-Up Study. from FY 1988-89 Demonstration Part- Welfare Reform in 1993: State JOBS Pro- Working Toward Jobs: A Series of State nership Program Projects. grams and Waivers. Case Studies. Responsibility, Work and Pride: The Values Underclass Behaviors in the U.S.: Measure- of Welfare Reform. ment and Analysis of Determinants. Families on Welfare: Focus on Teenage Assessing JOBS Participants: Issues and Mothers Could Enhance Welfare Re- Trade-Offe. form.

Selected Work/Welfare Waiver Projects Approved in 1994/1995

Arkansas Welfare Demonstration Indiana Indiana Manpower Place- Oregon JOBS Plus Program Project ment and Comprehensive Training Program PennsylvaniaPathways to Independence California Work Pays Demonstration Project Michigan To Strengthen Michigan Families South CarolinaSelf-Sufficiency and Paren- Colorado Colorado Personal Respon- tal Responsibility Program sibility and Employment Mississippi A New Direction Demon- Program stration Progam South DakotaStrengthening of South Da- kota Families Initiative Connecticut A Fair Chance New York Jobs First Demonstration Wisconsin AFDC Benefit Cap Dem- Florida Family Transition Program North DakotaEarly Intervention Program onstration Project Hawaii Creating Work Opportuni- Oklahoma Learnfare Program ties for JOBS Families

-A project to develop a skill-based labor strategies that provide economic and job A project to determine the feasibility of de- market information system reflecting the creation benefits and meet sustainability ob- veloping a plan to utilize private industry occupational restructuring underway and jectives. councils to attract high-growth high-wage changing job requirements. A project to stimulate technological innova- industries to distressed areas. A project to develop information cr :.,itiat- tion in small businesses. ing and executing economic development

.National Commissionfor Employment policY

An investigation of labor market trends be- An analysis of the Panel Survey of Income An exploration of employment prospects tween 1979 and 1994. Dynamics on the labor market status of Af- for low-income youth and young adults, in rican-American workers. A study of racial inequality in employment relation to lessons from the 1980s. and earnings among young men between A study of the employment effects of the A review of employment and training pro- 1979 and 1989. Administration's programs re: Technology grams for youth and young adults. for a Sustainable Future.

National Commission list continues.

.0.0.41.31MP.1 .1111NleaMEMIN111 154 Evaluation Forum issue 11e Surnmor 1995 163 Resources it Federal Research: An Update

: NatilatiOl CjznatX4001istc0AtirtUed .' ...':; , - , Selected Recent Projects

An analysis ofJTPA Title 11 training pro- A series of studies on the relationship be- A review and commentary on evaluation re- grams for in-school and out-of-school tween UI and dislocated worker programs ports produced by the U.S. DOL on con- youth, including the effect of local labor in terms of measuring performance in temporary employment and training pro- market characteristics on the JTPA post- JTPA, JOBS, and Secondary Vocational grams for an array of target groups. program outcomes of out-of-school male Education. An assessment c:the ability of three deliv- 7.f youth. An evaluation of the impact of Title III on ery systems to meet workforce devzioprnent A study of successful and promising earnings and employment prospects, using needs: education, government job training school-based dropout prevention programs UI data. and placement programs; and employer- sponsored education and training efforts. and strategies. A study of the impact of alternative eligibil- A series of work/welfare studies to explore ity requirements for training stipends in the A number of studies concerning earnings the implications of time-limited welfare re- Reemployment Act of 1994. inequality; history, causes, strategies to ad- dress. form initiatives on employment and training A study of the kinds of training workers un- systems; employers' roles in welfare re- cle:take. A study of the changing role of labor force form; the role of JTPA; patterns of partici- statistics for economic policies in the pation in programs; and other issues. An historical review of comprehensive 1990s. planning and programming in employment and training in the U.S.

Testing and Assessment in Vocational Edu- Peiformance Standards for the Food Stamp Technology and Work-Based Learning. cation. Employment and Training Program. Wage Record Information Systems. Technology, Jobs, and Productivity in a American Industry and the Environment: Multinationals and the U.S. Technology Service Economy. Implications for Trade and U.S. Com- Base. petitiveness.

Evicluailon Forum Issue 11 Summar 1995 155 164 In Memoriam

At the end of this issue of Evalua- work of community learning centers He made extremely important contri- tion Forum we honor the memory of to skill our youth, or a few thousand butions to policy understanding in a Sar A. Levitan, who died in May 1994. public service jobs to move people broad range of areas. He believed in Sar was a unique and remarkably influ- from welfare to work. But since these evaluations, learning by experience ential contributor to the national dia- are not an offer, I am honored to ac- and experimentation. I think one of logue on employment, training and an- cept this award. If right and rationali- the important lessons that he tried to tipoverty policy in the U.S. over his ty continue to be recognized, these teach everyone was never to assume long career as an activist economist. critical education, employment and that what you are doing will work au- Author of fifty books and hundreds of earnings policies may eventually tomatically, because if everything articles on employment issues, Sar was come to pass, to realize this nation's you tried to do worked it probably the recipient of tbe Secretary of La- promise of greatness. was not worth doing. .. . He had a bor 's Lifetime Achievement Award, A memorial service was held for Sar strong belief in democratic institu- presented to him by Secretary Robert tions, in the ability of informed opin- B. Reich. in June 1994 in the Education and La- bor Committee Hearing Room of the ion to get things done.... He be- The text of this coveted award read: U.S. House of Representatives. Those lieved that there was a role for "In recognition of your numerous con- commenting about his life, career and competition in politics as in other tributions to the Department of Labor achievements were Ray Marshall things, and that the federal govern- over several decades, you have set the (former Secretary of Labor), Gordon ment could in fact do a great deal to highest standards in your dedication to Berlin (formerly with The Ford Foun- improve the lives of people. And the work, the American worker, and the dation and now with MDRC), Marion reason he thought that is because it is fight against poverty. Your scholarship, Pines (The Johns Hopkins University), true. wisdom, wit and kindness have in- Jon Weintraub (House Committee on Sar actively supported the develop- spired us all." Education and Labor), Isaac Shapiro ment of Evaluation Forum as an educa- In accepting the award, Sar replied, (Office of the Secretary, U.S. Depart- tional tool for informing professionals ment of Labor), Cliff Johnson (The in the field about the results of major My scribblings over the years have Children's Defense Fund), Andrew policy analyses and research projects in aimed to combine right and rationali- Sum (Northeastern University), and employment and training, and willingly ty to improve social welfare. This Garth Mangum (University of Utah). offered many useful suggestions that award, from those who share this The following excerpts are from Ray improved its quality and utility and goal, is gratifying recognition of Marshall's comments at the memorial helped direct its further evolution. these labors. I would gladly trade it service: for an indexed increase in the mini- mum wage to reduce hardship, a net-

* U.S. GOVERNMENT PRINTING OFFICE: 1095 - 405085 i 44377

156 Ev,sluatIon Forum wIssue II if 165 Summer Mg I

History of the Journal The New Evaluation Forum Please Note Evaluation Forum wee devel- The new Evaluation Forum, of Each issue :-iddresses a general oped as part of a five-year national which this is the tnird issue, is theme in its Features section, but demonstration project, The JTPA being sponsored and funded by the also iaavers a range of subjects un- Evaluation Design Project, funded Office of Policy and Research, der the section Evaluation Issues between 1986 and 1991 by the Na- Employment and Training Adminis- and Activities. Therefore, readers tional Commission for Employment tration, U.S. Department of Labor. are encouraged to send materials Policy, the U.S. Department of La- its purpose is to disseminate useful to the editor on a wide variety of is- bor, the IBM Corporation and The information on evaluation issues, sues for possible review in the jour- Ford Foundation. The purpose of activities and results to those re- nal. the project was to increase the in- sponsible for administering, plan- Please send materials and sugges- terest, knowledge and sophistica- ning, managing and overseeing tions to: tion of employment and training employment and training programs profescionals at the state and local across the United States. Again, Ann Blalock Editor and Reviewer level regarding program evaluation: each issue of the journal will be Admiralty Inlet Consulting its scientific principles and meth- theme-oriented, and will follow the PO Box 409 ods, its practical applications in bet- same organizational format as in 40536 Skunk Bay Road ter understanding program imple- the past. However, each issue will Hansville, WA 98340 mentation and impact, and its utility feature reviews by the editor and 206-638-2159 as a practical tool for adjusting poli- guest reviewers rather than articles The editor would appreciate your cies and improving programs. The written by practitioners and re- searchers. providing the following information project's series of evaluatien guides with your materials: name, address, can be obtained through the ERIC Reader Participation phone, organizational affiliation, po- system in public libraries, or in book sition in the organization, and major The content of the new version form from the W. E. Upjohn Institute interests re: evaluation issues and/ for Employment Research. of the journal will cover the same or activities. kinds of materials as in the past: Evaluation Forum is a national commentaries, policy analyses; The Features of the next issue journal growing out of the five-year program reviews; reports on evalu- of Evaluation Forum will focus on project and developed specifically ation planning, reports on analyses issues surrounding youth employ- for state and local program practi- of monitoring data; research reports ment and training. tioners in the employment and on program evaluations or policy training field.Its major objective is research; books and articles on to communicate information on policy issues, programs and re- policy issues and research activi- search; and other materials related ties at the national, state and local to evaluation that would be of inter- level which can inform judgments est to practitioners. about the design and effectiveness of various employment and training policies and strategies. Eight theme-oriented issues of the journal were published as part of the dem- onstration project.

BEST COPY AVAILABLE