Evaluation Forum

Economic Competition, Restructuring and Worker Dislocation

U.S. Department of Labor Employment and Training Administration

Issue 11, Summer 1995 Evaluation Forum

Economic Competition, Restructuring and Worker Dislocation

Issue 11, Summer 1995

U.S. Department of Labor Robert B. Reich, Secretary

Employment and Training Administration Tim Barnicle, Assistant Secretary

Office of Policy and Research Gerard F. Fiala, Administrator

A review journal for employment and training professionals focusing on policy analysis and evaluation research in occupational training, education, work and welfare, and economic development.

Evaluation Forum n Issue 11 n Summer 1995 i ii Evaluation Forum n Issue 11 n Summer 1995 Table of Contents

Features Economic Change and Displacement 2 Presidential Perspectives on Economic Change and Restructuring: An Interpretation President Clinton’s view of economic change and related issues, based on an anthology of his speeches. 7 Understanding and Adjusting to Economic Change A review of The Urban Institute’s Rethinking Employment Policy. 11 A Challenge in Restructuring: Efficiency Vs. Equity A review of two informative books on economic and employment policy in the 1990s published by The Brookings Institution. 15 Understanding the Context of Dislocation A brief overview of four major books on technological change, economic restructuring and employment. 16 Worker Dislocation: A Review Article A guest review of three W.E. Upjohn Institute books on worker displacement, by labor economist Ernst W. Stromsdorfer.

National Studies of Dislocated Worker Programs 21 Evaluating Dislocated Worker Programs: U.S. Department of Labor Studies in the 1990’s A summary and commentary on U.S. Department of Labor process and net impact studies of programs designed to assist dislocated workers. 29 Other National-Level Studies in the 1980s and Early 1990s Relevant to Dislocation A review of analyses and evaluations of efforts to retrain and reemploy dislocated workers, conducted by the U.S. General Accounting Office, the Office of Technology Assessment, the Congressional Budget Office, and other sources. 33 Self-Employment As An Employment Strategy for Dislocated Workers A discussion of self-employment programs in the U.S. and Europe, and a review of evaluations of the major U.S. self-employment projects.

Defense Conversion and Dislocation 37 Defense Downsizing: Some Implications Contextual information about defense conversion, and a brief review of several Office of Technology Assessment reports on conversion issues. 40 Studying Defense Conversion in Affected Communities A summary and analysis of a major study of defense adjustment and conversion, sponsored jointly by the U.S. Departments of Commerce and Defense. 43 Programs to Ease the Adjustment to Defense Cutbacks An overview and discussion of the plethora of assistance programs for workers and communities impacted by defense downsizing.

Evaluation Forum n Issue 11 n Summer 1995 iii 46 Findings from the Defense Conversion Adjustment Demonstration Project A guest review of the U.S. Department of Labor’s evaluation of the demonstration, by Mary G. Visher, director of the evaluation, and Deborah Kogan, its co-principal investigator.

53 A Commentary by Secretary of Labor, Robert B. Reich Commentary A commentary on the status of the American economy, on employment and dislocation, and on income inequality, with implications for policy.

57 An Interview With Salvatore Zecchini Interviews An interview with internationally-respected economist, Salvatore Zecchini, Assistant Secretary-General of the Organization for Economic Cooperation and Development in Paris, on economic restructuring in Central and Eastern Europe and its policy implications.

62 An Interview With John Dryer of The Boeing Company, and Gary Jackson of the International Association of Machinists and Aerospace Workers An interview with the prime movers in developing a remarkably successful corporation-based partnership model for assisting dislocated workers, The Boeing Reemployment Program.

Evaluation Issues At the International Level Economic Reform in the European Union and Activities 68 The OECD’s Employment Study A review of OECD’s Job Study, its implications for employment policy, and its policy recommendations.

72 The Persistence of Unemployment in the Post-Industrial Era An overview of two analyses by OECD economists on the reasons for long- term unemployment in the EU.

76 Labor Issues in European Consolidation A review of a Brookings Institution book by three labor economists on the potential effects of EU consolidation on labor forces and labor markets.

77 Political Economy of “Borrowing” A review and analysis of a provocative book about the transnational borrowing of employment and training ideas between the U.S. and Great Britain, by a Brookings Institution economist and a British political scientist now at the Rand Corporation.

iv Evaluation Forum n Issue 11 n Summer 1995 Economic Reform in the Former Soviet Union 83 A Selective Overview of Soviet Reform Policies A context for understanding current economic restructuring in Russia, a review of Edward Hewitt’s classic, Reforming the Soviet Economy. 88 Gorbachev’s Perestroika An analysis of Gorbachev’s reform ideas. 90 The U.S.S.R. in Transition An overview of a fascinating book published by the OECD, Structural Change in Central and Eastern Europe.

At the National Level 94 Some Thinking About Employment and Training at the National Level A selected history of human service coordination/integration efforts over the past few decades. 97 Measuring Performance in Programs and Coordinated Systems A review of two informative monographs — by economists at the University of Baltimore on performance standards, and by the Office of Technology Assessment on the use of UI wage records in evaluation. 103 More About Performance Management A review of several useful reports on “accountability” produced by the Council of Governors’ Policy Advisors and the National Institute for Literacy. 105 Using Performance Standards in Establishing Workforce System Accountability Review of economist Burt Barnow’s excellent analysis of the pros and cons of using performance standards, in Evaluating Welfare and Training Programs edited by Charles Manski and Irwin Garfinkel. 109 Performance Management Systems: Evaluation Principles and Standards A summary of key elements of the American Evaluation Association’s new standards for evaluation practice, in James Sanders’ The Program Evaluation Standards published by Sage.

At the State and Local Level 115 A Brief, Selected History of Coordination Efforts A discussion of state and local efforts to coordinate their employment and training programs in the 1970s through 1990s. 118 Establishing Comprehensive Workforce Development Systems: State- Level Efforts in the 1990s 120 Contemporary Coordination Efforts: The Core Data Elements Project An overview of the joint efforts of the U.S. Departments of Education, Labor, Health and Human Services and Agriculture to develop a core set of variables, operational definitions, and quantitative measures cutting across multiple employment and training programs.

Evaluation Forum n Issue 11 n Summer 1995 v 123 Comprehensive State Workforce Development Systems A summary of ideas in Northern Illinois University’s Building State Workforce Development Systems Based on Policy Coordination and Quality Assurance, the major analytic resource for NGA’s Performance Management Project. 127 The National Governors Association’s Performance Management Project An in-depth summary and perspective on the NGA project.

142 References on Economic Change, Restructuring and Working Dislocation Resources Books, articles and reports on the above issues, and relevant international references. 153 Federal Research A selected update on employment and training-related research in the federal agencies.

A Memorial to Sar Levitan 156

vi Evaluation Forum n Issue 11 n Summer 1995 Features

Editorial Introduction n The changed international trade re- nesses operate in the global economic alities. marketplace. The terms “economic efficiency, revi- n The end of the Cold War. Illustrations talization, restructuring and reform” have now become well accepted within n The development of advanced high- Specific examples of such changes the postindustrial vocabulary — in fact, techology products and markets. are everywhere apparent. We have warmly embraced. We freely insert seen unprecedented partnerships es- n The new alliances between devel- these concepts into our narrative piec- tablished among some of the cable es as if they had been commonplace oped and developing countries. television industry’s largest companies, through the centuries and there is an Distinctions which will give customers a new range international consensus about their of telecommunications products and meaning. But do they have that consis- In understanding what is meant by options. These partnerships will not tent a history and is there agreement “restructuring” we need, of course, to only affect wireless communications but on how they are defined? The easiest distinguish between cyclical and non- also one-stop shopping for many ser- answer is “No.” cyclical causes for economic reorgani- vices. zation. Reduced domestic demand for With many international economists, certain products, trade deficits, and the The rise of Hewlett-Packard in the “economic reform” has been summarily shrinkage of market share in interna- computer printer market has been a replaced by “economic restructuring”, tional competition are largely cyclical dramatic story. The company now holds and subsequently adopted by the rest phenomena. Firms are pushed to re- approximately 55% of the world market of the world. However, the meaning structure by forces over which they may for inkjet printers, and has retrieved the continues to vary. This eminently fash- have minimal or no control. The other U.S. lead in related technologies. H-P’s ionable phrase may describe an array restructuring scenario is influenced success has made it one of the two of postindustrial phenomena: more by the efforts of management fastest-growing major U.S. multination- als. The company has attributed its suc- n The creation of joint ventures. and/or labor to increase economic effi- ciency and productivity within business- ess to its heavy investment in laboratory n The merging of previously separate es and industries. breakthroughs, its drive to capture mass businesses and industries. markets and sustain rapid product inno- It is important to recognize also that vation and price cutting, its trust in local n The consolidation of product devel- most restructuring efforts are motivated creativity and accountability, and the cut- opment through new business part- primarily by economic preservation, ting of its internal operating costs. nerships. which has the effect of staving off the Military technology has been applied n The development of “niche” markets. mass unemployment that would occur with the complete failure of businesses successfully, in the absence of the Cold n The expansion or diversification of — even though it frequently involves War, to domestic and international mar- products by a single multinational. significant levels of unemployment and kets. Scientists who helped the Navy worker dislocation at the time, and diffi- create a sophisticated submarine-track- n The downsizing of corporate work- ing computer system are now applying forces. cult readjustments for many of the workers affected. similar advanced digital processing Causes technology to the identification of trou- The reality is that economic competi- ble spots in jet engines, nuclear reactor Some of these activities have been tion is no longer contained within na- coolant pumps, and electricity-generat- characteristic of the economies of in- tional or regional boundaries. New eco- ing steam turbines. This translation from dustrialized nations for some time. nomic powers have emerged within this military to civilian markets has made new They have fueled economic growth and altered environment, to compete with economic efficiencies possible. created jobs. However, the postindustri- the U.S. for international customers, al period has brought with it new chal- whose needs and desires have also In the Northwest, we see Oregon’s lenges due to heightened international changed. American businesses and in- restructuring of the timber industry. Tra- economic interdependence. The basis dustries have had to rise to this occasion, ditionally, Oregon has been the top tim- for these challenges lies in a number of if they were to support general economic ber-producing state. The drastic curtail- developments: health, and the life quality Americans ment of logging in federal forests have attained since World War II. represented a major challenge to the n The movement of postindustrial na- state’s economy. But Oregon managed tions from manufacturing to services. A variety of “restructuring” strate- to produce five billion board feet each gies had to be designed and tested in n The revolution in information and year using wood from tree farms rather this context. Some have been success- telecommunications. than the 300-year-old trees from public ful in preserving economic progress land, and the mills are producing more n The new political positions affecting while protecting the gains of the Ameri- products out of this timber by using social spending and trade relations. can labor force, and others have not. parts that were discarded previously. In But these new economic methods have n The increased concern about the the restructuring process, Oregon lost clearly revised the way American busi- planet’s physical environment.

Evaluation Forum n Issue 11 n Summer 1995 1 Features n Economic Change and Displacement

nearly 15,000 jobs in forest products permanent job loss. Many of the work- ganize and finance new products. Mat- over the past five years, but gained ers laid off are skilled high-wage work- tell Inc. recently announced a restruc- nearly 20,000 in high technology busi- ers but their occupational expertise pro- turing plan that would eliminate 4.5% of nesses. vides few if any opportunities to its workforce, which was expected to Dislocation transition to other occupational areas. reduce Mattell’s expenses by $25 mil- Many companies have tried to ease the lion in 1995. In 1994, Kmart planned to Despite restructuring successes, it is impact of downsizing, but markets and close 110 stores and eliminate 6,000 clear that the postindustrial period has profit margins in emerging economic ar- jobs, including 10% of its managerial had its downside. Although we are en- eas inevitably influence the extent of positions, related to its loss of market joying general economic growth, there downsizing. And in some cases mass share to competitors. The Boeing Com- is a long-term increase in income in- layoffs have been inevitable if compa- pany’s own series of layoffs was due equality in the U.S. The trend toward a nies are to survive and prosper. Some both to reduced customer demand and greater gap between the wealthy and dislocated workers can be retrained restructuring in the interest of retaining the poor is attributed by many experts successfully for other occupational a competitive edge in world aerospace to wage stagnation and erosion, which niches, but frequently at lower wages. markets. And these are but a few of the in turn is correlated with the shift to a And some fall into poverty. many examples of companies that have service economy, increased interna- The recent IBM layoffs are an exam- had to include layoffs in their efforts to tional competition, the decline of union- reorganize and compete. ization, and the decline of the minimum ple of the dislocation often involved in wage. corporate restructuring. Cray Research In this issue of Evaluation Forum, we Inc., a supercomputer maker, recently explore some of the issues raised in And certainly the downsizing of laid off 1,000 workers to cut its manu- this introduction. In Part I we look at workforces is a major restructuring is- facturing costs. American Express economic change as a context for fo- sue. Worker dislocation unquestionably planned to eliminate as many as 6,000 cusing on workforce issues. In Parts II represents another reason for rising in- jobs in an effort to recover from several and III we look specifically at worker equality. Cost containment in the inter- lean years, and to save $500 million in dislocation and what is being done to est of economic efficiency has led to expenses which could be used to reor- address that contemporary problem.

Economic Change and Displacement Editorial Introduction the international economic environment mists specializing in restructuring and on that development, and the impact of dislocation issues. We conclude Part I The purpose of Part I of the Features economic change on workers affected with a guest review of three important Section is to provide some background by it. In serving this purpose, we pro- books that focus more exclusively on information on the recent evolution of vide a number of perspectives, from the worker dislocation. the American economy, the influence of President’s to those of various econo- nn The knowledge and understanding a municating key themes with the intent to Presidential president possesses about the economy reflect as accurately as possible what ideas Perspectives on and its larger context, and his policy were conveyed through that medium. positions and perspectives on economic Economic Change issues, influence policy development in Economic History and Restructuring: a uniquely powerful way within the In remarks at the signing of the NAF- An Interpretation framework of a tri-partite governing TA side agreements in September, structure. This president has paid un- 1993, President Clinton commented that usual attention both to economic prob- Perhaps the most significant public after the second World War we refused lems and economic development possi- policy issue of the 1990s has been gov- to let history repeat itself. Rather the ernment’s response to economic bilities, to the role of the U.S. in the United States and other Western powers global economy, to domestic workforce change — to the cyclical changes we came together to shape a new era. The development, and to ways in which have come to accept and tolerate as an American economy was rebuilt. NATO inevitable characteristic of a free mar- government and the private sector can was established. We carried out the work together to assist workers placed ket economy, and the economic restruc- Marshall Plan to reconstruct war-rav- at risk in a new economic environment. turing that has been occurring in this aged nations abroad. New institutions period of international competition. Therefore we begin this section with were formed to foster peace and pros- The former has produced variations in insights drawn from an anthology of perity — such as the United Nations, the employment. Frequently the latter has President Clinton’s speeches prepared for International Monetary Fund, the World resulted in permanent job loss and users by the White House Press Office, Bank, and the General Agreement on worker dislocation. excerpting from that anthology and com- Tariffs and Trade.

2 Evaluation Forum n Issue 11 n Summer 1995 Features n Economic Change and Displacement

These actions, the President said, of change which economics, technolo- made in America were subject to com- helped to usher in four decades of robust gy and information flow have imposed. petition at home or abroad from foreign economic growth and collective security. producers and foreign providers of ser- When you live in a time of change, Yet the Cold War was draining — we vices. Whether we saw it or not, our he remarked, the only way to recover had devoted trillions of dollars to it. your security and to broaden your hori- daily lives were touched everywhere by Now the Soviet Union had disintegrated. the flows of commerce that crossed na- zon was to embrace change, adapt to it, The nuclear shadow was receding with tional borders. Capital clearly had be- and move forward. The only way we Start I and II. Democratic governments could recover the fortunes of the mid- come global. Services had become glo- were emerging around the world. bal. Information had become global. dle class, so that people who worked This was his message. The President reminded his audience harder and smarter could at least pros- that at the end of World War II an un- per more, he believed, was to adapt to His position was that the economy challenged America was protected by the many changes that were occurring. was now so globalized and change so the oceans and by our technological su- He encouraged the audience to rec- institutionalized that no government of periority. We chose, then, he said, to any nation could promise to protect ognize how far we had moved from an help create a world of free trade sup- people from the changes in the world industrial age built on gears and sweat ported by institutions that would facili- to embrace an information age demand- economy. So if change was inevitable, tate it. Global trade subsequently grew we had to help people view change as ing skills, learning and flexibility. In from $200 billion in 1950 to $800 bil- our friend rather than our enemy, a this process, record numbers of Ameri- lion in 1980. As a result, jobs were cre- cans were succeeding in the new global source of security rather than a source ated. That decision to create a system of insecurity. And we had to do that in a economy. The economy had generated of expanded, freer global trade played a way that promoted those institutions of almost six million new jobs since 1993. major role, he believed, in producing We had the lowest combined rate of society that were most important to us, the prosperity experienced by the U.S. principally our families and communities. unemployment and inflation in twenty- middle class after World War II. five years. Businesses were more pro- The changing nature of work, he This new period of history repre- ductive. We had brought the deficit said, was placing enormous demands sented a world America helped bring down to expand trade and give citizens on working people. The average work- into being, he suggested — through the more of the tools they needed to obtain er today in every kind of work had to period of the Civil War, two world an education and to rebuild their com- be able to work with more information, wars, the Cold War, the establishment munities. to be more creative, to solve more of global economic and trading mis- problems on his or her own initiative. sions, and the efforts to build a family Workers, Families and Therefore, the President said, we had to of nations and establish other instru- Communities in the transfer more responsibility to workers ments of peace and harmony, of New Economic Era at the grass roots level. They had to progress and democracy. However, the President also stated learn more skills, and process and ana- lyze more information than ever before. But the President also acknowledged in his remarks at the NAFTA signing that Americans were continuing to raise that while the nation as a whole had en- Since every American had to face these forces, and every American family did, central questions about our place and joyed peace and prosperity in this era, he felt the job of government ought to our prospects in this new world we had too many people were working harder done so much to produce. They were for less. While businesses were restruc- be to empower people to make the most of such forces. asking questions such as “Will we and turing and growing more productive our children really have good jobs, and competitive, too many people still When the family is under economic first-class opportunities, a world-class could not be sure they had a job next stress, he proposed that workers within education, quality affordable health care, year or even next month. And far more families either had to learn more and safe streets?” After having fully defend- than our material riches were threat- become more productive, get a better ed freedom, he realized, they want to ened, he insisted — our children, our job, or work harder for less, trying to know if they will share its bounty. families, our values were at risk. The be as sturdy and resilient as the times common bonds of community which For the last two decades, he said, the are. The latter was what had happened had been the great strength of our to millions of American families for the middle class that was created and en- country from its beginning had become last twenty years, he said — working larged by these policies had been under badly frayed. stress due to changes in the global en- harder for less. The average working family was spending more hours at vironment, the growth of technology, Speaking at American University in work today than twenty-five years ago, and increasing economic competition. the winter of 1993, the President point- If we had learned anything from the ed out that we were woven now inextri- for about the same hourly wages, ad- justed for inflation. collapse of the Berlin Wall and the fall cably within the fabric of a global of the governments in Eastern Europe economy. Imports and exports, which The President saw 1973 as the wa- and the former Soviet Union, he com- accounted for about one in ten dollars tershed year in American life. From the mented, it was that even a totally con- now represented one dollar in every end of World War II until 1973 family trolled society cannot resist the winds five. Nearly three-quarters of the things income doubled. We lived in an era of

Evaluation Forum n Issue 11 n Summer 1995 3 Features n Economic Change and Displacement

prosperity that we almost came to take And into that vacuum had rushed job loss. Businesses do not choose lo- for granted. The middle class grew ever gangs, drugs and violence. cations based solely on wages, he said. larger and more secure. People just as- They choose them on the basis of the Acknowledging that it was difficult sumed that they could obtain jobs they skills and productivity of workforces, to maintain a sense of security and opti- would hold for a lifetime, that they mism that a country like ours needed in on government attitudes, on roads and would always do better every year than railroads to deliver products, on the order to lead the world into the future, they had the year before, that they availability of markets for those prod- he identified a plethora of needs: would be able to afford to send their ucts. Those locations were our strength, children to college, have a comfortable n The need for pro-family policies, he believed. retirement, own their own house, and like the family and medical leave Some of the specific provisions in take care of their parents. But the reali- policy enacted into law. NAFTA, he said, were important in that ty was, he said, that since 1973 most n respect — for example, provisions that Americans had worked increasingly The need to pass humane welfare re- removed some of the current incentives harder for the same or lower incomes. form that encouraged people to be successful parents and workers. for people to move their jobs across the He had other disturbing information border, making it harder than before for to convey. Between 1972 and 1992, n The need for a national network of businesses to relocate solely on the ba- while the work year became longer for manufacturing extension centers to sis of very low wages or lax environ- most Americans, wages stagnated. The help small firms accomodate to new mental rules. The environmental agree- 75% of people who did not have col- challenges and acquire new technol- ment applied trade sanctions against lege degrees felt it profoundly. Those ogies. any of the countries that failed to en- who began but did not finish college force their own environmental laws. saw their wages fall by 9% just since n The need for telecommunications re- The agreements also ensured that Mex- 1979. For those who did not go on to forms that would help the U.S. de- ico would enforce its laws in areas such college, wages fell 17%. And although velop the information superhighway. as worker health and safety, child labor he pointed out that the U.S. traditional- and the minimum wage. In addition, n ly generates new jobs through interna- The need to reform U.S. job training they protected our industries against tional trade, he also recognized that programs, especially the unemploy- unforeseen surges in exports from ei- some were lost each year to competi- ment system. The President remind- ther one of our trading partners. tion from countries with lower wages ed his audience that eighty percent In his view, we had now created a or higher quality, or sometimes unfair of the people losing their jobs today trade agreement that moved beyond the trade practices. do not get called back to their old traditional notions of free trade, putting Median family income today, he jobs. We had become locked into a the environment and human resource added, was only $1,000 higher than it 1950’s system when we needed one considerations at the center of this and was twenty years ago. And more signif- for the 21st century that encouraged future trade agreements. At the same icant, the growth in income inequality continuous retraining and place- time, the President felt such trade between those who were educated and ment in the work force. agreements imposed new obligations those who were not had escalated dra- on American government. Government matically. Even though there were 50% International Trade was obligated to make sure that U.S. more people in the work force of mi- and the Economy workers were the best prepared and nority origin with four years of college trained in the world. The President also commented that or more, the aggregate racial gaps in in- open and competitive commerce would We now know, he said, that the av- come were deeper because the educa- enrich us, spur us to innovate, force us erage 18-year-old American will tion gap had grown so great and be- to compete, connect us with new cus- change jobs up to eight times in a life- cause inequalities in income had tomers. His perspective was that trade time. Over the past ten years, for the escalated in the last several years. Once promotes global growth without which first time in history workers losing their Americans looked forward to doubling no rich country could hope to grow jobs often found they could not return their living standards every 25 years. wealthier. It enabled our producers to to their old job and had to find a new At present productivity rates, it would prosper, who were themselves consum- occupational niche. Consequently gov- take 100 years to double living stan- ers of services and raw materials. ernment needed to develop a new reem- dards, he predicted — until our grand- He understood that many Americans ployment system to replace the tradi- children’s grandchildren are born. tional unemployment system. We had were worried that expanded trade Our society, he said, had suffered to be able to tell that subset of workers would move jobs out of the U.S. be- unbelievable stresses, as broken homes cause of the difference in wage rates. who became dislocated by more open and unwed mothers had become more trade policies that there was a job train- They had seen this happen. But the commonplace. In places devastated by ing and reemployment program avail- President pointed to the eighteen out of poverty and despair, we had seen the nineteen serious economic studies by able to them. absolute collapse of families and work both liberals and conservatives that had Once we had put our own economic itself, and of the sense of community. concluded that there would be no net house in order, he said, we must make

4 Evaluation Forum n Issue 11 n Summer 1995 Features n Economic Change and Displacement

trade a priority element of American ing business beyond our borders. For the first time in a decade we had security, part of an integrated economic nine months of manufacturing job n program. Then we must do our best to Increase our investment in education growth. For the first time in nine years exercise leadership among the major fi- and training — and in technology the annual vote of international econo- nancial powers to improve our coordi- and defense conversion. mists was that the U.S., not Japan, was nation on behalf of global economic n Bring the benefits of free enterprise to the most productive economy in the growth. areas which have been isolated from world. Also, he proposed, we must promote it, in our inner cities and rural areas. But the President also believed we the steady expansion of growth in the were dealing with some long-term eco- n Make the federal government more developing world, not only because it nomic problems. The average hourly was in our interest but because it would efficient and effective. wage in the U.S. actually peaked about help these countries as well. These di- Implementing these strategies, he twenty years ago, and working people rections, he said, constituted an agenda believed, would require a much more had been losing their health insurance for American action in a global econo- aggressive, innovative partnership with steadily for about ten years, the only my, as well as for our own prosperity. the private sector. Government does advanced country in the world where Therefore the best investment we could not have the capacity to “save” the U.S. that was the case. So there were genu- make, he was convinced, was in the economy in an economic environment ine reasons why a lot of hard-working one resource rooted within our own that is global in scope, he said. But gov- Americans did not feel more secure or borders — in the education, skills, rea- ernment can create the right climate, the happier with the good statistics and soning capacity and creativity of Amer- right conditions, and can empower peo- growth rates. They were still not sure icans. ple so that they can compete by taking re- that these numbers guaranteed them a sponsibility for themselves and their fam- good future and a good job, the ability American Economic Progress ilies. He felt that increasing changes in to keep their children’s health insur- In February, 1993, speaking to a the world made this imperative. ance or put aside money for their col- Joint Session of Congress, the Presi- lege education. They were not sure that The President indicated we were not the society was going to be able to dent explained that a number of condi- alone in facing economic problems in tions over the 1980s had brought us to solve the social problems that violated this new era. Every advanced country our values and conscience. a point of decision: two decades of low was experiencing a common phenome- productivity, low growth and stagnant non: even as economies grew, there And we had growing poverty. We wages; persistent unemployment and were pockets within them that had not had to find a way, through both public underemployment; years of huge gov- been benefited by economic recovery, and private sources, to enhance the ernment deficits and declining investment where investment had not taken hold, lives of the American people who lived in our future; exploding health care costs where jobs had not materialized. This in poverty, by bringing investment and and lack of coverage for millions of was particularly true for inner cities jobs to poverty areas. We could not, he Americans; legions of poor children; and and isolated rural areas.The first thing said, repair the American family until education and job training opportunities we needed to do in the U.S., he sug- we provided the structure, the values, inadequate to the demands of an unusu- gested, was to change the job mix, in- the discipline and the reward that work ally competitive global economy. creasing the proportion of high-skill, provided. The economic plan he recommended high-wage jobs. And we needed to pro- We needed to recognize also, he for turning this around involved four vide special incentives for people to in- said, that well over half the women fundamental components: a shift of em- vest in isolated areas, using empower- who were mothers in this country were phasis from consumption to invest- ment zones, the enterprise community in the work force. We had to decide ment; honoring work and family in ev- concept. that as long as the economy mandated ery part of our public decisionmaking; In 1994, according to the President, this, and the economic pressures of the reducing the federal deficit honestly we found ourselves in the midst of the time did, we had to find ways for peo- and credibly, and earning the trust of first investment-led, low-inflation, pro- ple to be both successful parents and the American people by paying for this ductivity-driven economic expansion in successful workers, and ways of orga- with cuts in government waste and in- three decades. Something fundamental nizing ourselves differently with regard efficiency, in government spending, was beginning to happen, he was con- to child care, health care, family leave, and with fairness to those bearing the vinced — more than half the new jobs and the income of people who had chil- burdens of these changes. created by our economy in 1994 were dren and who worked but still did not He felt the key elements of this ap- above the average wage. This was more make enough money to support them. proach were these: high-wage jobs in one year than in the previous five combined. Investment in Education and Workforce n Reduce the deficit. new equipment was eight times what it Preparation in a Postindustrial Economy n Expand trade and intensify U.S. had been in the previous four years. For government efforts to become a the first time since 1979 the U.S. led Speaking at the University of North partner with the private sector in do- the world in the sale of automobiles. Carolina in 1993, the President pro-

Evaluation Forum n Issue 11 n Summer 1995 5 Features n Economic Change and Displacement

posed that economic security could no with a common love for this country Goals 2000 set national skill standards longer be found in a particular job. It and the values of freedom and mutual as well, to ensure that our workers were must be rooted in a continuing capacity respect. He felt we had to find a way in better trained. It gave states the flexi- to learn new things. That condition re- this age and time to restore that kind of bility to meet educational and skill quired a system of life-long learning, discourse and that kind of respect. We standards according to their own needs beginning with higher standards for could not afford to engage in the citi- and ideas, stimulating well-planned ex- schools. Almost two decades ago we zenship of division, distraction and de- perimentation. He felt strongly that in- began education reform efforts that in- struction. We had a future to build, novations needed to be encouraged and cluded higher standards, he said. Now through education. rewarded, evaluated for their efficiency we were trying to adopt a whole new and effectiveness, and if successful The President had a special opportu- approach in a national effort to raise nity to talk about the meaning of edu- adapted for use elsewhere. This would educational standards. This, he felt, be a departure from our past. There had cation for American young people at a was an American imperative. At the never been any national standards. junior high school in 1994. He ex- same time he believed we must insure plained that the Goals 2000 proposal There had never been a formal way to that every young person had the oppor- measure whether or not educational and embodied ideas gleaned from many tunity to obtain a college education. skills acquisition goals were being met. years of work on educational issues. It And for the three-fourths of young peo- established what kind of education every In the past, he said, everyone who ple who did not obtain a four-year col- child needed in every school in America. was willing to work could find work. lege degree, we needed to merge the It set out some goals that were expected Today, more than ever before, whether world of learning with the world of to guarantee that young people would be you had a job or not, how much you work, to offer young people classroom prepared to compete successfully in the could earn in a job, and what your fu- and on-the-job training. 21st century — wherever they were, and ture was, depended on how much you In a talk at Rutgers University in whether they came from poor, middle could learn — not just what you knew 1993, the President reminded the audi- class, or wealthy families. but how much you could learn. Re- ence that in the midst of World War II One of those goals for the year 2000 search was indicating that people who President Roosevelt proposed the GI graduated from college made twice as was for every school in America to of- Bill of Rights, which offered returning much as those who did not. The world, fer a disciplined environment condu- veterans the opportunity for education cive to learning. Another was to in- he said, was increasingly smaller, in return for their service to our country changing very fast. People were con- crease the high school graduation rate in war. Thanks to the GI Bill, more nected financially and by communica- by 90% — an international standard. than eight million veterans received an Still another was to make sure that ev- tion networks that were unheard of ear- advanced education. Half a century lat- lier. Therefore we had to be able to ery adult possessed the knowledge and er, the enduring legacy of the GI Bill learn new things continually. skills needed to obtain and retain a was the strongest economy in the good job, a job as good as people had The President’s perspective on our world, and the broadest, biggest middle in other postindustrial countries. future was that we must view ourselves class that any nation had enjoyed. At the signing ceremony for the more as an extended family, a large The national service plan, he point- community, and look at all the people Goals 2000: Educate America Act in ed out, was constructed on the same in this society as an enormous resource. March of 1994, the President remarked principles, namely that when people that this legislation set world-class edu- If we did so, we would reinvigorate our give something of invaluable merit to educational and training systems. He cational standards for what every child their country they ought to be rewarded said we needed to make sure that our in every American school should know with the opportunity to further their ed- — for reading and writing, math and movement into the 21st century was ucation. The national service plan en- grounded in an old idea, that all Ameri- science, history, geography, foreign abled American students to borrow the cans have not only a right but a respon- languages, civics and economics, and money they needed for college and pay the arts. It also said something about sibility to rise as far as their talents and it back as a small percentage of their determination can take them, and to teachers. We had to prepare them bet- own income over time. The President give something back to their communi- ter, enable them to continue to learn. considered this especially important af- And it said something about parents. In ties and their country. We needed a ter a decade in which the cost of a col- new set of understandings, he thought, a world in which families were under lege education had gone up even more about how we can equip people to meet increasing stress, the President said we rapidly than the cost of health care. must find ways to help them become the challenges of the new economy, how we can change the way govern- At the 75th anniversary of the more involved in their children’s edu- ment works to fit a different time, and founding of UCLA in May, 1994, the cation. And the legislation acknowl- President asked graduating seniors to edged that we needed more research and how to repair the damaged bonds in so- ciety. The future was not an inherit- look back at the history of the Constitu- innovation so we could generalize the use ance, he said. Rather it was an opportu- tional Convention. To look at how peo- of successful models and strategies. ple assembled wildly different points of nity and an obligation, something every Besides academic standards, the generation had to create. view and argued heatedly, but always President reminded the audience that

6 Evaluation Forum n Issue 11 n Summer 1995 Features n Economic Change and Displacement nn Understanding and and addressing the problems of disad- technology and service jobs of the vantaged workers, the structurally un- future. Adjusting to employed, and those experiencing dis- crimination in the labor market. n Changes in our approach to poverty, Economic Change given growing income inequality. There was a significant expansion of Economists’ perspectives on the programs to train and retrain workers, These changes provide the context economy, how it works, how it is increase UI benefits, create public ser- for the perspectives of other economists changing, differ with their theoretical vice jobs, and ensure equal opportuni- in Rethinking Employment Policy. training and orientation. Although this ty. Even though the problems that intellectual attribute confuses us, it also prompted this policy change have not Ray Marshall’s Perspective expands our minds. In this article we faded away, the times have changed, Former Secretary of Labor, Ray look at perspectives embodied in sever- they point out. In the 1980s, the combi- Marshall, concentrated his chapter on al chapters of a book published by the nation of a conservative president, fis- “The Implications of Internationaliza- Urban Institute, Rethinking Employ- cal pressures to cut budgets, and nega- tion for Labor Market Institutions and ment Policy, edited by D. Lee Bawden tive public perceptions of certain Industrial Relations Systems.” It is and Felicity Skidmore. The book is one programs shriveled employment policy. Marshall’s thesis that the international- in a series of publications sponsored by The editors predict prophetically in this ization of economies and the globaliza- the Institute’s Project on the Federal introduction that the American public tion of markets between 1950 and 1980 Social Role funded by a number of pri- is not likely to support a more aggres- have been the most important economic vate foundations. Although the book sive and expansive policy, if based on trends since World War II. He supports was written in 1989, prior to additional the full array of past strategies. this contention with adequate evidence. changes in the American economy and Meanwhile, they insist that some of By 1987, he says, over half of U.S. cor- the role of the federal government, its the troublesome problems of the past porate profits derived from overseas. insights nevertheless apply usefully to International trade accounted for one- the 1990s. remain to be solved. In fact there is more concentrated structural unem- third of all U.S. cropland, one-fourth of The book takes a retrospective look ployment, expanded underemployment, farm income, and one-sixth of all jobs. at what is known and remains to be un- an increase in worker dislocation, and Nearly 70% of all goods manufactured derstood about dealing with domestic continuing discrimination in the work- in the U.S. was competing with im- and international issues facing the U.S. force. They propose that the solutions ports. Close to 25% of the growth in labor market in this decade. The chal- to these problems must be crafted in the the U.S. workforce during the 1970s in- lenge the editors pose at the beginning context of important social, economic volved immigrants and refugees. of the book is to find a method for and demographic changes that have These changes, he says, have altered 1) maximizing the potential of our taken place since the 1960s, and must the effectiveness of traditional macro- workforce without creating inflation, take advantage of what has been economic policies. The integration of and 2) targeting job training and em- learned over the past several decades. markets globally has transformed labor ployment programs to those who can The changes they claim we must re- market institutions and industrial rela- benefit the most from them, so that our spond to are these: tions systems, as well as influencing resources are used efficiently. n Changes in the composition of the management practices. Bawden and Skidmore identify two workforce — that is, more older major themes that guide the book. One The Evolution of Economies workers and women — requiring the Since World War II is that U.S. employment policy has not adjustment of retirement policies, met the goals envisioned in the early wage structures, working hours, and Marshall explains that after the 1960s as achievable. The other is that services such as child care. WWII, governments viewed an open, we now know enough from analyses expanding trading system as the best and research that shaping a more effec- n Changes in equal opportunity and way to encourage the growth of world tive future employment policy is possi- affirmative action policies respon- economies. Many experts had linked ble and should be done. sive to less overt, more subtle forms protectionism and trade restrictions of racial and sexual discrimination. with the Great Depression and the war Employment Policy History itself. As a preventive measure, the n Changes in our educational system, In their introductory chapter, the ed- U.S. and other countries worked to- itors suggest that the employment and given that over one-fourth of the gether to form institutions, policies and training policies of the 1960s and early adult population lacks a high school procedures to encourage and maintain 1970s, which represented a policy education, and may lack the basic international trade and finance. This change as well as increased funding, skills that support training and re- free-trade system became known as the generated a new interest in identifying training for the expanding high- Bretton Woods system.

Evaluation Forum n Issue 11 n Summer 1995 7 Features n Economic Change and Displacement

It was organized around three basic accompanying this development. Inter- ment of the American economy — that principles: 1) free trade, 2) fixed ex- national economic rules that worked for business and industry’s ability to com- change rates, and 3) autonomy in do- the 1950s and 1960s no longer applied. pete successfully in world markets had mestic economic policy. The world’s been declining for over twenty years, Countries are not as effective in di- welfare was considered to be increased recting their policies concerning the in- even in high technology sectors. Con- by a competitive, free-trade, open-mar- clusions from other analyses elaborated ternational economy, Marshall says, ket system in which each nation would on this assessment. Marshall cites two and the effects of domestic economic concentrate on producing things which policies have changed. The impact of studies in the mid-1980s by the Brook- would give it the greatest or the least ings Institution and the New York economic stimuli in a period of eco- advantage economically. This idea was Stock Exchange that reach other con- nomic depression, for example, limits referred to as “the doctrine of compara- domestic economic expansion by in- clusions. These were the major find- tive advantage.” ings: creasing foreign imports — a serious Alongside this doctrine, a critical in- problem for the U.S. during the 1981- n An overvalued dollar, due to huge fluence was the power of the U.S., ac- 1982 recession when foreign imports U.S. budget deficits, is the main cording to Marshall. Our domestic cut into increased demand. And the cause of our economic problems. economy had been strengthened by consequences of U.S. budget deficits Given this scenario, the analysts World War II, having benefited from are not under as much control. This concluded that the economic boom technological development during the lowers U.S. living standards and can of the 1950s and 1960s was “abnor- war, which supported an enormous raise inflation, he says. It also forces growth in productivity and total output. other countries to keep their interest mal” and the economy would return This economic dominance translated to rates high. These destabilizing effects to “normal” as soon as the budget policy dominance. It was therefore in have brought pressure on policymakers was balanced. our national interest to advocate for a to reconsider the Bretton Woods sys- n Trade was the factor most responsi- relatively open and expanding world tem. ble for a net increase in jobs in the economy. The reality has strayed from this sys- 1970s — we gained more employ- But even as the Bretton Woods sys- tem’s theoretical underpinnings. Com- ment from trade than we lost to it. tem encouraged growth in the interna- petition, Marshal says, does not direct tional economy through the 1960s, cer- domestic markets, therefore countries n Trade was not the main reason for tain events were to begin to tear it do experience unemployment. There- declining employment even in the apart, Marshall explains. These were fore many industrial countries have industries experiencing the greatest some of those events: now supported strategies that both cre- declines. ate comparative advantage and improve n The abandonment of the fixed ex- n U.S. industrial performance was the industry mix. And Marshall claims change rate system. that economic activity is now based on strong relative to other countries — first or second in the world in 23 out n The massive increase in foreign in- national and enterprise strategies, not vestment and related flows of capi- simply on the interaction between of 40 manufacturing industries in tal. short-run market forces and the maxi- 1982. mization of profits. n Most job losses were concentrated in n The growth of the Eurodollar market Japan, for example, is more interest- four key depressed industries which (dollars held and traded abroad). ed in strengthening national power, accounted for only 20% of total n A large increase in U.S. debt and the productive capacity, and market share, manufacturing output. The 36 indus- debt of non-oil-producing develop- he says, than in “short-run profit maxi- tries that dominated manufacturing ing nations. mizing.” Also, economic change is not actually added jobs between 1977 gradual, Marshall points out. Trade is and 1982. n Large-scale movements of workers no longer limited to “goods at the mar- among countries in response to em- gin.” Whole technologies are exported, n The growth in productivity had been ployment and income opportunities. and countries share the production of slowing since the 1960s, but in man- products. The result , he says, is poten- ufacturing it had been particularly Some Implications of tial job loss, wage suppression, and less strong. Internationalization satisfactory working conditions. In this generally more optimistic Marshall sees both positives and The Potential Loss of the perspective, poor production perfor- negatives in this internationalization of U.S. “Competitive Edge” mance was seen to be correlated with markets. The positives are increased ef- energy price shocks, inflation and eco- ficiency and expanding knowledge, Marshall reports that the President’s nomic uncertainty, none of which were which raised living standards. The neg- 1985 Commission on Industrial Com- thought to interfere with performance. atives are the destablizing influences petitiveness conveyed a dour assess-

8 Evaluation Forum n Issue 11 n Summer 1995 Features n Economic Change and Displacement

The analysts pointed to the expansion Marshall’s Viewpoint ed macroeconomic policies and strate- of real plant and equipment spending, Marshall draws from both perspec- gies that avoid over-valuing the dollar. increased funds for research and devel- And finally we need to reform our in- tives, acknowledging that the U.S. is opment, increased efficiency due to the dustrial relations system, to improve still the strongest world economy and accumulation of work experience by likely has the highest average produc- economic stability and employment and baby boomers, a better educated work- reduce unemployment. tivity levels, even though we appear to force, and a significant improvement in be losing that advantage rapidly. He be- Most of the features of the Bretton labor-management relations tied to the lieves that macroeconomic policies and Woods system were characteristic of all need to become more competitive. the expensive dollar are important fac- industrial nations, and some of these But there was a remaining residue of tors. But he thinks the pessimists are features persist. However, Marshall pessimism, exemplified in the Harvard correct in being concerned about main- teases out certain elements that have Business School’s analysis. The main taining market positions “without sacri- been unique to the U.S. and must be conclusions, according to Marshall, ficing return on assets, or real wages or considered in developing a new indus- were the following: incomes” — that is, he feels our living trial relations system: standards are going down. His position n A more realistically-valued dollar is n American unions were considered is that analysts are on target in taking not the answer, because the structure seriously how we are doing vis-a-vis the bargaining agent for all employ- of trade has changed. The economic countries like Japan. Productivity ees, whether union members or not. recovery from the 1981-1982 reces- growth is the key to maintaining com- However, the legal right of workers sion has led to slower growth in ex- petitiveness, in Marshall’s mind, and to vote for or against union member- ports and faster growth in imports therefore for sustaining high wages and ship created competition for em- than in previous recoveries. The re- profits. ployees’ loyalty between union and turn of business assets has failed to He also sees the composition of in- non-union sectors, and between keep pace with the rising cost of dustry as an important influence. Stud- unions and employers. capital. ies that treat all industries the same are n Collective bargaining was decentral- n Energy prices were an important missing critical information, he says — ized, emphasizing wages, hours and factor in trade deficits in the late “there are dynamic and symbiotic rela- working conditions in particular tionships between production activities 1970s and 1980s, but there was also firms, industries and labor markets. and technology transfer.” Also, he re- a decline in non-energy trade bal- minds us that the linkages between eco- n U.S. employers were generally hos- ances as well. nomic activities and technology are cu- tile to unions, despite the affinity of n Market share in constant dollars is mulative. A lack of competitiveness in the American labor movement for not the only, or the most significant key industrial sectors may lead to their the capitalist system and its flexibili- loss, which then removes from remain- measure of economic performance. ty in responding to the interests of ing businesses the technological sup- Performance is determined also by employers. ports, the learning accumulated, and the the relative values of exports and networks developed that are required n A “scientific management system” imports. Our share of total world ex- for them to stay competitive. was used by employers which was ports has declined. generally authoritarian and adversar- Marshall’s Recommendations u The U.S.’s share of high-technol- ial, allowed only minimal participa- ogy exports has fallen also. Having established that the Bretton tion in decisionmaking by workers, Woods system no longer works in the and provided minimal job security. u U.S. productivity growth has postindustrial era, what does Marshall been far below our competitors, suggest? One answer, he says, is to re- n Union politics were not class-con- even though their economies strict the growth of the international scious, rather they focused on jobs have remained depressed. trading system — an answer he rejects and economic security rather than on out of hand. Trade restrictions, he says, an organizational or political base The pessimists have discounted can worsen economic problems be- many of the optimists’ tenets, indicat- for the working class. cause it produces retaliatory action. ing that the evidence shows that in n Management felt they had a right to manufacturing, productivity growth has The right answer, in Marshall’s lay off workers during economic deteriorated “despite improvements in view, is to promote an open and ex- downturns since the expectation was both capital formation and capital-labor panding trading system guided by 1) that business cycles would keep lay- costs.” internationally acceptable rules that can offs temporary, and the UI system be enforced, 2) international economic policies, and 3) international economic would provide adequate temporary institutions. Also needed are coordinat- assistance.

Evaluation Forum n Issue 11 n Summer 1995 9 Features n Economic Change and Displacement

n The American system was more n Mechanisms within firms that help economists, is generally supported by flexible than most European indus- absorb demand shocks, such as bo- the Administration’s economic and in- trial relations systems in adjusting to nus compensation systems, sharing dustrial relations system reform ideas, change, due to: of production with Third World and has been taken seriously by the countries, subcontracting, and the larger corporations. However, the im- u The principle of exclusive bar- pact of these ideas on reducing disloca- use of temporary workers. gaining. tion has yet to be determined. n The ability to shift resources from u The more decentralized bargain- Another Perspective: Goals for ing system. non-competitive to more competi- tive industries. National Employment Policy u The lower degree of unionization. n An emphasis on lifetime employ- Isabell V. Sawhill’s perspective on economic issues is also enlightening. u The greater ease with which em- ment, continuing education, the con- She defines four goals for employment ployers could close plants and lay centration of collective bargaining at policy: off workers. the enterprise level rather than at the n The provision of jobs — i.e. the re- u The greater openess to immigra- industry or sectoral level, and the duction of unemployment. tion and imports. annual adjustment of wages (pre- cluding internal competition on the n The creation of more good jobs — u The greater displacement of la- part of employers and unions.) i.e. the improvement of productivity bor, resulting in pools of under- and earnings. employed workers. n Heavy emphasis on consensus- building at every level, rather than n The provision of assistance to the These elements supported a long pe- on detailed regulations. riod of relatively high growth in pro- disadvantaged — i.e. making the duction and output, and moved most n A bonus wage payment system that poor more self-sufficient. union members into the middle class. keeps wage increases in bounds. n The improvement of the functioning But the system tended to be inflation- n A flexible job assignment and train- of labor markets using means that ary. The bargaining approach tended to ing process that increases workers’ guarantee economic efficiency and raise wages by playing one employer or the fair treatment of workers. union against another. Long-term union job security and employers’ willing- contracts with cost-of-living adjust- ness to finance long-term education She views these goals as serving ments and annual increases in benefits and training for their employees. dual purposes: human welfare in soci- tended to inflate the compensation ety, and the economy’s long-term But Marshall does not leave us here. base. UI and other income maintenance growth and productivity. The strategies He points out that the U.S. has higher programs reduced the impact of labor overall productivity and living stan- she recommends for achieving the four supply on wages, just as multiple wage- goals are these: 1) macroeconomic pol- dards than Japan, and some sectors of earner families did. Full employment icies; 2) job creation policies, such as the Japanese economy, he says, are not policies reduced employers’ incentives particularly productive or efficient. subsidized employment in the public to avoid wage increases and unions’ and private sector; 3) education; 4) in- Nevertheless, the Japanese system pro- willingness to hold them down. As a come transfers; and 5) government reg- vides rising real incomes and much result, Marshall says, insufficient atten- more job security. And it is more egali- ulation. She considers the public educa- tion was focused on the level of pro- tion system the strategy of choice for tarian in terms of income differentials. ductivity and the degree of economic creating better jobs through improved Beyond that, Japanese workers see a efficiency. strong relationship between improve- productivity. In providing jobs overall, macroeconomic policies are primary, Marshall looks to the Japanese sys- ments in productivity and the employ- she believes. Income transfers are im- tem for answers. These are the major ment benefits they receive, which gen- characteristics that capture his atten- erates a considerable incentive. The portant in assisting the disadvantaged, even though concern about work disin- tion: cooperation between public and private centives and long-term dependence re- sectors makes it possible for businesses n Highly integrated, consensus-based to be satisfied with lower rates of re- main. economic policies emphasizing up- turn, which then encourages long-term With respect to the goal of improv- grading of the Japanese industry planning. ing the functioning of labor markets, mix. she gives attention to worker protec- Marshall’s diagnosis of our prob- tions and benefits — that is, to protec- n A management system supportive of lems and possibilities has garnered labor-management cooperation and wide support from other respected tive labor laws; anti-discrimination laws; job security; retirement, disability participation. and unemployment insurance; pension

10 Evaluation Forum n Issue 11 n Summer 1995 Features n Economic Change and Displacement

nn rights; and notification of layoffs. More jobs can be created if Americans While the protections and benefits are would accept lower wages, she points A Challenge in of critical importance to workers, she out. Alternatively, wage levels could be Restructuring: says, they may also reduce economic increased at the risk of greater unem- efficiency. Employers find it more dif- ployment. The choice is more disturb- Efficiency Vs. Equity ficult to adjust to changing market de- ing, she suggests, in an economy that is In this article we review ideas in two mand. This is the perennial tension be- vulnerable to increased foreign compe- books whose topics are relevant to the tween equity and efficiency that tition, since our standard of living can perennial dilemma posed by efforts to economists have been struggling with be greatly influenced by that choice. increase economic efficiency without since Adam Smith. We could, she says, compensate for jeopardizing economic equity. Resolv- choosing higher wages through a com- ing this dilemma has important impli- Resolving Problems bination of 1) trade protectionism, 2) a cations for worker dislocation, since in Meeting Goals stimulative fiscal and monetary policy dislocation is frequently a product of What Sawhill suggests is that we find that reemployed displaced workers, or efforts to increase economic efficiency more flexible and less costly methods 3) a decline in the value of the dollar but often leads to increased income dis- for resolving the tension, weighing — but these solutions would, she be- parity. costs vs. benefits more carefully. The lieves, lead to an erosion of the pur- chasing power of higher wages and de- Both books are Brookings Institu- current labor market and industrial re- tion publications. We first give atten- lations system, Sawhill says, “evolved teriorate the standard of living. tion to Growth With Equity: Economic as a requirement of a modern economy Whatever the strategy, Sawhill feels Policymaking for the Next Century, built on the principles of specialization we must make special efforts to assist written by Martin N. Baily, Gary Burt- and large-scale production.” These the disadvantaged. Her perspective on less, and Robert E. Litan in 1993. And principles served us well, she recogniz- programs to reduce youth unemploy- then we turn to A Future of Lousy es, greatly increasing productivity and ment and welfare dependency is that Jobs? The Changing Structure of U.S. America’s standard of living. adverse economic and demographic Wages, edited by Gary Burtless in However, little attention was paid in forces are more responsible for slow 1990. this system to workers’ rights and pro- progress than the programs developed tections, which led to unionization and for that purpose — and that some of Themes in Growth With Equity to an array of protections and benefits. these programs have been, in fact, quite successful. The authors’ position is that the And the industrial system that initially American economy exhibits two worri- gave us prosperity was ill-suited, she A Brief Commentary some trends: slow improvement in pro- claims, for a world in which the labor ductivity growth, and growing income force is educated, workers make de- These selected perspectives on the inequality. Short-term fluctuations in mands, services and information re- economy provide important lessons the business cycle and in unemploy- place standardized goods, and capital about potential relationships between ment are not their primary concerns in and technology are widely available. economic policies, unemployment and this book — it is long-term patterns These changes have created a climate dislocation, despite the differences that concern them: for workplace reform in the direction of among labor economists in the nature worker satisfaction and participation, of the evidence they interpret, their n What will economic recovery mean? and new styles of corporate leadership. conclusions from this information, and n How can we boost the rate of long- How successful this new direction will their recommendations for change. On term growth? be, Sawhill says, is yet to be estab- one point there is agreement among lished. them: it seems clear that the industrial n How can income disparities be re- relations system of the 1950s and duced? Some Conclusions 1960s is no longer viable in the 1990s and must be revised substantially in the These questions frame a fascinating One of the most difficult trade-offs book that provides insights based on an in achieving the goals Sawhill proposes context of postindustrial international competition. increasingly large body of evidence is between employment and wages. about long-term economic performance and its implications for equity issues. The authors’ approach, they want us to understand, is “eclectic” and “cau- tious.” No single policy is envisioned as the magic bullet. They view their recommendations as modest, in the sense that they are designed to increase the probability of achieving progress in resolving productivity and income ine- quality problems.

Evaluation Forum n Issue 11 n Summer 1995 11 Features n Economic Change and Displacement

A premise is that appropriate poli- By faster growth, the authors mean n How strong the evidence is from re- cies for increasing economic growth both a more rapid rate of advance in to- search and expert opinion in sup- and bringing greater equity to the dis- tal output and a faster rate of increase porting the wisdom of the proposal. tribution of income are inevitably inter- in output — “holding constant the related. The economy, they say, can amount of inputs used to produce that Of these proposed policies, labor grow faster if the low-skilled and low- output.” A labor force that works more policies are considered the most sup- incomed can produce more and higher intelligently to produce goods and de- portive of both growth and equity. quality goods and services per hour liver additional services with the same The full complement of recommended worked. Skilled workers, they contend, amount of work effort is at an advan- policies is presented in Figure 1. have greater flexibility and mobility, tage. Advantage is achieved by raising The authors’ proposals are designed which means they can move from de- labor productivity. to speed economic change by increas- clining to expanding industries with Economic growth refers to the ing productivity growth, while easing shorter periods of unemployment fol- growth of potential output, “the output the burdens of change for workers and lowing job loss. If unemployment is so that can be attained when labor and their families. But the authors remain reduced, the economy can be pushed to capital are fully employed and inflation humble about the impact of such pro- lower levels of unemployment without is not accelerating.” Economic growth posals, hoping for substantial improve- serious levels of inflation. At the same is spurred, the authors say, by advances ments if not dramatic results. They are time, the problems we associate with in technology, by improvements in convinced it is time to develop a com- poverty will not be sapping the econo- workers’ skills, and by progressive prehensive set of interelated policies as my if its resources as much. management practices. When the au- well as for realism. Inaction, they say, With respect to income inequality, thors talk about income inequality, they is an unacceptable strategy. the authors state unequivocably that are referring to inequality in pre-tax in- their interest is not limited to strategies comes as well as after-tax incomes — The Issue of Earnings Inequality that reduce poverty, rather they are in market incomes, particularly wages. Gary Burtless’ edited volume, A concerned about changing the condi- In commenting on their comprehen- Future of Lousy Jobs? speaks directly tions that lead to low earned income sive policy model, they acknowledge to the equity issue. In the introductory among the least skilled. However, they that the research evidence does not sup- chapter, Burtless identifies the prob- recognize that some policies that speed port all of their recommendations lem. Over the last two decades, the long-run growth do not support equity, equally well. But this does not deter share of all income obtained by the and policies that are consistent with eq- them. Using a medical analogy, they richest fifth of families in the U.S. has uity do not necessarily increase eco- point out that one cannot always expect increased by 3.3% to 43.7%. The share nomic growth. existing scientific information to pro- received by the poorest one-fifth has Their main contribution to thinking vide conclusive answers about alterna- fallen 1% to 4.6%. Incomes of the about these complex issues is a com- tive treatments, but this is hardly a rea- group of families just below the richest prehensive framework and strategy —a son to abandon the search for evidence rose slightly, while the incomes of mix of policies and programs — for en- or to avoid making policy suggestions. those groups of families just above the couraging long-term economic growth And so they do make them! poor fell. These shifts, Burtless con- while maintaining equity. This policy cludes, have reversed the trend over model is based on assumptions about An Agenda for Growth much of the postwar period, a trend the underlying causes of slow growth and Equity that saw this gap shrinking. and increased inequality, such as: Three broad themes run through the Causes of Inequality n The impact of oil price shocks over large set of proposals in the authors’ Since the main source of income for the last two decades. model: increasing and reallocating in- vestment; assisting workers; making most Americans is wages, one cause n Reduced innovation. hard choices;and implementing for rising income inequality is thought to be the decline in middle-class jobs n An increase in regulation. change. The proposals subsumed under these themes are organized into several with high to moderate wages. Not all n Deteriorating educational perfor- policy categories: innovation, labor, in- economists agree that this is the major mance. vestment, and trade and foreign invest- cause, but most agree that average ment policies. For each proposal in the earnings growth among production n Stronger foreign competition. model, there are qualifications: workers has stopped in recent years af- n A stagnant minimum wage. ter rising strongly over the early post- n Whether or not the policy contrib- war period. This sharp decline since n Declining unionization. utes to economic growth or to eco- 1973 is related, Burtless says, to the nomic equity, or to both. slowing of productivity growth, but The model also depends on one’s again economists are not of one mind definition of growth and productivity. n How substantial the proposal’s ben- about why this change has occurred. efits may be in resolving one or the other, or both.

12 Evaluation Forum n Issue 11 n Summer 1995 Features n Economic Change and Displacement

One problem is defining and measuring among women. Given this result, one tion or aggregate level of labor supply, productivity and wage inequality. For could propose that middle-class jobs and 2) the changing pattern of demand example, Burtless proposes that an ide- are declining for men, but not for wom- for U.S. workers. al measure of worker compensation en. Composition and Demand Shifts would include both wages and fringe Between 1973 and 1987, the earn- benefits per hour or per forty-hour ings gaps between well-paid and mod- The rise in the number of poorly- workweek, but the Census does not col- erately-paid workers, and between paid jobs might be related to an in- lect enough information to calculate moderately-paid and poorly-paid work- crease in employers’ use of less skilled this. ers have grown over time, particularly labor — but Burtless says this is con- Another way to measure inequality in the 1980s. The number of men with trary to the evidence. If employers were is to look at the distribution of wages earnings close to the median wage level hiring less skilled workers, the wages of and calculate the trend in earnings for has shrunk. Jobs near the middle of the these workers would be elevated relative workers at selected points in the earn- male distribution have disappeared. to those received by better skilled, more ings distribution. This method indicates Burtless points to two reasons for these educated workers. However, the opposite that inequality is rising among men but changes that have been explored by pattern occurred in the 1980s. Also, job- falling or remaining fairly stable economists: 1) shifts in the composi- lessness among the less skilled should in-

Figure 1 n Policies Recommended by the Authors Innovation Policies

n Strengthen basic science research: to provide peer review. u Institute tax penalties for paying di- u Authorize oversight on the part of u Fund precompetitive research. rectors in cash. the National Science Foundation. n Promote commercial R&D by dou- n Reduce risks of innovation: u Re-examine the allocation of the bling the R&D tax credit. u Adopt modest “tort” reforms, ac- current federal budget. n Lengthen business time horizons: companied by strengthened R&D and enforcement by regulatory n Strengthen precompetitive R&D: u Encourage equity ownership by agencies. u Fund only projects with private bankholding companies. u Continue to allow joint R&D. sector financial support. u Institute tax penalties on executive u Fund a broad diversity of projects. director stock options with short ex- u Create a Civilian Technology Board ercise periods.

Labor Policies

n Improve skill training of schools. u Establish public-private labor au- Corps and JTPA. u Encourage schools to provide high thority to develop tests and creden- n Encourage firms to link worker pay school transcripts to prospective tials for non-college occupations. with division of company-level perfor- employers. u Institute pay-or-play training re- mance. u Require national standardized quirement for non-college-educated n Cushion economic shocks: workers. tests. u Upgrade national and state employ- u Tie compensation and promotion of u Establish national apprenticeship ment services. teachers and principals to perfor- program for non-college-educated workers. u Establish a non-cause-related pro- mance of students on national gram of earnings insurance. tests. n Establish improved R&D programs n Improve skills of current labor force: for such public training efforts as Job

Investment Policies

n Raise investment share of total out- n Enact a permanent 5% income tax n Allow banks to meet requirements put: credit for equipment investment. through investments in community u Reduce, and ultimately eliminate, n Spend somewhat more on infrastruc- development banks. federal budget deficit. ture, concentrated on upgrading ex- n Encourage particular types of invest- isting facilities. ment with spillover benefits to the n Encourage regional industry “clus- rest of the economy. ters” by states or localities.

Trade and Foreign Investment Policies

n Encourage further multilateral liber- n Use liberalized and more rational practice” status as carrot to obtain alization of trade and foreign invest- treatment under U.S. “unfair trade better access to foreign markets. ment.

Evaluation Forum n Issue 11 n Summer 1995 13 Features n Economic Change and Displacement

crease under this theory, yet unemploy- ative to the number in the worst-paid n It is extremely unlikely that part- ment rates for the less skilled have risen trade and service sectors. Between 1967 time jobs or welfare state programs relative to those more experienced. and 1977 this pattern was similar for men have contributed to changes in the and women. From 1977 through 1987, The declining wages of less skilled wage structure. men may be due to a variety of shifts in the loss of goods-producing jobs contin- ued, but was more rapid for men than for n The direct and indirect effects of the structure of labor demand, Burtless female workers. This may explain, in supply-side demographic trends indicates. Demand for less skilled workers could have decreased due to part, the diverging patterns of wage gain have contributed in a significant for men and women. collapsing demand for the goods or ser- way to wage developments. vices produced by industries in which The declining importance of labor n The long-term trend toward higher less skilled labor is concentrated. Un- unions may be another contributor to joblessness, particularly among skilled workers displaced by this shift the patterns observed in the data. prime-aged men, has contributed, to would have been pushed to seek em- Unions assist workers in raising the some extent, to earnings inequality. ployment in other industries, forcing earnings of moderately-paid workers down the wages paid to unskilled labor and frequently reduce wage inequality. n Changes in the industrial structure and in those industries. Changes in the tech- Also, the erosion of the value of the the decline in unionization have con- nology of production may have de- minimum wage has affected the relative tributed importantly to inequality. The pressed the demand for less skilled labor earnings of people at the lower tail of deteriorating quality of American in selected industries or across industries. the wage distribution. schools has left a generation of work- Or the decline in average wages re- Burtless says a striking trend in the ers poorly prepared to deal with the ceived by unskilled workers could be wage structure in recent years is the challenges of modern employment. due to sharp employment losses in in- continuing and pronounced drop in dustries such as mining and durable earnings among less educated men. In n The changing structure of demand manufacturing that normally paid high an era in which an increased proportion for skilled and unskilled male work- wages to less skilled workers. Fewer of the workforce has some schooling ers within industries has contributed unskilled workers will be employed in beyond high school, workers who have to inequality. high-wage industries. And by directing had no more than a high school educa- Burtless concludes that if the de- displaced workers into low-paying in- tion appear to face growing handicaps mand for unskilled labor has dropped, dustries, the average wage received by in the labor market. This is due, Burt- the obvious policy response is to im- low-skilled workers in those low-wage less reports, both to shifts in labor de- prove the qualifications of the less industries could be depressed. mand and in labor supply. The most skilled to match the developing re- important factors in increasing wage Changes in the labor supply, accord- quirements of the job market. “If the ing to Burtless, can have two kinds of ef- differentials between less skilled and nation has too many unskilled workers, more skilled workers appear to be: fects on the wage structure. An increase rather than too many bad jobs,” says in the number of less skilled workers will n Changes in industrial structure. Burtless,” both efficiency and equity raise the proportion of all workers who will be served by improving the skills n Declining rates of unionization. receive low wages. Increased competition of workers now lodged at the bottom.” for less skilled jobs may depress wages n The fact that the number of college- relative to those received in more skilled educated young men entering the A Commentary on the occupations. Both of these effects have, workforce failed to keep pace with Brookings Books in fact, occurred. Some of the differences the rising demand for highly educat- These two books are useful compan- between men and women may reflect sex ed workers. ion pieces in gaining a better under- segregation in the labor market and dif- standing of the trade-offs between eco- ferent trends in labor demand. If moder- Some economists have proposed nomic efficiency and equity. Clearly ately-paid men are concentrated in de- that the rise in part-time employment in there are substantial differences in the clining industries, Burtless points out, the U.S., and the effect of government perspectives of different economists, in “middle-class jobs could be disappearing income maintenance has contributed to terms of theories, the assumptions driv- for men even as the distribution of wages wage inequality. However, Burtless re- ing those theories, the interpretation of among women remains virtually un- ports that the research does not support research findings, and strategies to re- changed.” this contention. solve efficiency and equity problems. The phenomenon of dislocation is Major Patterns Burtless’ Conclusions caught in this web of information. The Meanwhile, the pattern of employ- The evidence from research is quite data on reemployment wage rates for ment gains and losses emerging in sta- persuasive, Burtless believes, on sever- increasing numbers of dislocated tistical analyses generally confirms the al points: workers, which typically involve a de- proposition that the number of jobs in pression of their previous wages, must n Wage inequality has risen among better-paid industries has declined rel- be figured into these analyses and the men, but not among women. patterns emerging from them.

14 Evaluation Forum n Issue 11 n Summer 1995 Features n Economic Change and Displacement nn Understanding the mental changes in economic systems about collaborative restructuring ef- in the past two decades. Dislocation has forts — that is, from the transnation- Context of been a major legacy, requiring new al borrowing and adaptation of roles for workers, management, labor ideas. Dislocation and government in economic develop- ment — and a coherent economic re- The essays in the book identify both The purpose of this short article is to the challenges and the opportunities encourage you to read further in the lit- structuring strategy. that exist in developing alliances that erature on restructuring and disloca- The book examines three main is- can have economic value. The authors tion. Four books worth the effort are sues: 1) the extent to which work orga- seem to agree that the best role for the the following: nization and market share, or competi- federal government is to facilitate and n Technology and Global Industry: tive position, are influenced by coordinate activities at the regional, technical innovation; 2) how these Companies and Nations in the state and local level, and provide an in- changes are conditioned by government World Economy, edited by Bruce R. formation clearinghouse on best prac- policies, and 3) what role collective tices. The most important conclusion Guile and Harvey Brooks and pub- bargaining relationships between lished in 1987 by the National Acad- seems to be that without some kind of unions and companies play in the eco- institutional base for supporting eco- emy Press. nomic restructuring process. A series nomic restructuring, short-term politi- n Technology and Employment: Inno- of case studies of cooperative restruc- cal pressures will overwhelm the long- vation and Growth in the U.S. Econ- turing efforts, both in Europe and the er-term needs of industrial economies. U.S., are used to extract lessons for omy, edited by Richard M. Cyert American policymakers. and David D. Mowery, also pub- Technology and Employment lished by the National Academy A number of themes permeate the The Cyert book grew out of the Press in 1987. book. An interest in team-based work Committee on Science, Engineering organization is one, emphasizing the and Public Policy of the National n Innovation and the Productivity Cri- participation of workers in areas tradi- Academy of Engineering, prompted by sis, by Martin Neil Baily and Alok tionally reserved for managerial deci- the Committee’s interest in the impact K. Chakrabarti and published by sionmaking. Interfirm linkages is an- of technological change on employ- The Brookings Institution in 1988. other interest — among companies, ment opportunities, productivity and their suppliers, and their clients. Locat- life quality. The theme is that employ- n Economic Restructuring and Emerg- ing the development of industrial poli- ment losses due to a decline in interna- ing Patterns of Industrial Relations, cies at the subnational or regional level tional competitiveness are likely to be edited by Stephen R. Sleigh and is another emphasis, encouraging a mu- greater than any that result from tech- published by the W.E. Upjohn Insti- tually-reinforcing network of key ac- nological change, and that such change tute for Employment Research in tors. is essential to the preservation and ex- 1993. And these are some of the lessons: pansion of U.S. employment and wag- es. Themes and Recommendations n In the absence of a national industri- al policy, the efforts of state govern- The book contributes a significant We merely provide glimpses of the ments have created a common direc- array of policy options and recommen- wealth of information and insights in dations in three areas: public policies tion that supports an economic these books, as an incentive to readers to assist worker adjustment to techno- restructuring process integrating so- to look for more. logical change; public policies to sup- cial goal-setting while retaining the port the development and application Economic Restructuring and efficiency of market competition. of advanced technologies; and im- Emerging Patterns of n Governments cannot influence eco- provements in private sector manageri- Industrial Relations nomic development positively with- al expertise in evaluating and imple- menting new technologies. The Sleigh book was based on a out sponsoring joint efforts among year-long series of seminars sponsored business, workers and unions. With respect to worker adjustment, by the City University of New York, the recommendations deal mainly with ending in a major national conference n Transnational economic alliances changes in Title III of JTPA, particular- in 1990. The purpose of the book was and investment have blurred the na- ly to broaden the range of employment to summarize and comment on innova- tional identity of corporations and services, including job counseling, tive economic restructuring jointly un- made it more difficult to develop a skills diagnosis, job search assistance, dertaken by corporations, unions and national industrial policy. and placement services. There is strong government. Sleigh says that the eco- support for increasing funds for basic n A national industrial policy needs to nomic shocks of technological change and job-related skills, and for expand- and global competition created funda- be developed, however, based on a ing income support during retraining. synthesis of what has been learned Instituting a program of federally-pro-

Evaluation Forum n Issue 11 n Summer 1995 15 Features n Economic Change and Displacement

nn vided direct loans and loan guarantees Referring to an OECD study, how- administered by states or local areas is ever, the authors propose that diffusing Worker Dislocation: felt to be a critical resource for retrain- and refining already-developed tech- A Review Article ing or relocation. Also recommended is nology is the key to high productivity a series of demonstration projects to growth.They therefore argue for con- test program designs, which are rigor- tinued support of basic science, for Ernst W. Stromsdorfer ously evaluated. “middle-ground” and commercial re- Department of Economics search, and an improved educational Washington State University Innovation and the system that appreciates the knowledge Productivity Crisis and skills required for technological This Brookings book is one in a se- innovation. There are important mes- sages here for workforce development. ries of publications on productivity is- “You should learn a trade. They can sues. Funded in part by the National Technology and never take that away from you.” Science Foundation, it suggests that the Global Industry — Anonymous factory worker, slow pace of innovation may be influ- ential in slowing productivity growth. Brooks and Guile begin the book by in advice to his son. Unlike some other economists, the au- identifying two characteristics of global thors propose that the size of the col- economic change: 1) the U.S. no long- As the above quote suggests, work- lapse of productivity growth in the U.S. er dominates world economic affairs ing men and women are in fear of los- should define this as a national crisis. but rather is only one element, even ing their jobs due to economic change, They claim that had the growth in out- though significant, is an increasingly whether it be from technological inno- put per hour after 1965 equaled the av- global economy, and 2) the growth of vation, foreign competition, shifts in erage rate from 1870 to 1965, output other national economies has permitted demand, government regulation, or oth- would have been more than 20% higher production and distribution to become er sources. There are epic tales of in 1985 than it actually was. Unless increasingly transnational. Technologi- highly skilled workers attempting to there is an increase in productivity cal advances have played an important outdistance technological change growth, they say, Americans will suffer role in emerging economic and techno- through increased effort and other stagnant or falling living standards. logical integration, increasing the ca- means, with the demise of the glass pacity of transnational systems that car- bottle blowers at the turn of the century In addition to labor productivity, two other indices of production perfor- ry information, goods and people. coming to mind most sharply. There are tales of violent resistance—the ac- mance are given attention: output per Focusing on the activities of multi- tion of workers themselves creating unit of capital used, and multi-factor national companies and the policies of production. The owners of capital did industrialized nations, the book seeks evocative new words for our lan- guage—sabotage and Luddites. There poorly after 1965. And the collapse of to answer these questions: is the modern image of the “Rust multi-factor productivity growth, in the n What are the effects of changing aggregate, has been startling, with no Bowl”, characterizing the upper indus- technologies on the production and trial Midwest. clear message that it will increase. The authors place this in context by refer- distribution of goods and services in Scientific interest in the impact of ring to the data on other industrialized the global economy? economic change on worker displace- countries, which reveal a similar and n How do technological advances con- ment began early in this century, fol- sometimes more severe slowdown. tribute to shifts in the relative com- lowed by seminal articles by Haber (1925) and Myers (1929) in the 1920s. Although the authors focus on the petitive advantage of nations, re- Case study monographs of worker ad- role of innovation, they find that infla- gions and firms? tion, recessions, oil price fluctuations, justment to plant shutdown began oc- n curring in significant numbers soon af- and other economic interferences in the How do governments and enterpris- ter the end of World War II, but 1970s also affected economic perfor- es respond to the dynamics of tech- mance. They suggest that there are two nological advance in a global econo- systematic empirical research on the problem of worker displacement did kinds of strategies for utilizing technol- my? not begin until the 1960s in response to ogy for economic growth: one expands n What are the likely consequences of the technological frontier, developing waves of restructuring that began in the their efforts? post World War II era to hit auto manu- leading-edge industries (characteristic facturing, meat packing, the railroads, of the U.S.); the other diffuses technol- Although the book directs our atten- ogy that has already been developed (as coal mining, and other industries (See tion to the management side of eco- Borus, 1964; Schultz and Weber, 1966; in Europe). The Japanese are now using nomic issues, there are keen insights to Somers, 1968). Even then, the issues both strategies. be culled for workforce preparation and dislocation between the lines.

16 Evaluation Forum n Issue 11 n Summer 1995 Features n Economic Change and Displacement

were not fully understood, whether one velopment Act of 1959 became the first cash flows to the program of one bil- was concerned with the measurement national legislation to provide for inter- lion dollars a year and historical expen- of the displacement loss to the individ- vention in the worker readjustment pro- ditures since 1959 that exceed many ual and society; the time and money cess. This legislation focused on poli- tens of billions in current dollars! costs of finding new employment; is- cies to aid in the economic adjustment sues of social compensation to losers process. No direct cash compensation The Dimension of the Problem from change; whom to train in new for loss of human capital was contem- Perhaps part of the historical reason skills, how to train them, and what plated. The first effort at direct cash for this tolerance has been our failure skills to train; or, how to provide reli- compensation for displacement came until recently to comprehend the full able data to inform social policy on any with the trade adjustment assistance dimension of the problem. The studies of these issues. program in the Trade Expansion Act of by Leigh and Jacobson et al. indicate 1962. Compensation was to be paid to The three volumes discussed in this that the numbers affected are large and those workers damaged by liberalized essay carefully and thoroughly assem- the losses per worker are severe. ble much of the methods of analysis international trade. The intent was, in Working with different data sets and part, to reduce political resistance to re- and knowledge that we possess today slightly different assumptions, both duction in trade restrictions; however, concerning the above issues. The Costs Leigh and Jacobson et al. estimate sim- of Worker Dislocation by Jacobson, the program concept had strong theo- ilar annual totals of workers hit by dis- retical support in the compensation LaLonde and Sullivan (1993) provides location. A key assumption in such es- principle of theoretical welfare eco- one of the most conceptually compre- timates is to specify a minimum length hensive and empirically complete stud- nomics. Trade adjustment assistance of employment in a firm in order for a has gone through several modifica- ies of the pattern and quantity of wage laid off worker to be counted as a dislo- tions—first by simplifying eligibility loss due to economic change and dislo- cated worker. For one estimate, Leigh cation. Duane Leigh’s study, Assisting for compensation in 1974 and then hav- sets a minimum of at least five years of ing eligibility and payments tightened Displaced Workers, is a comprehensive employment at a firm prior to disloca- up in 1981. review as of 1989 of the policies de- tion. This assumption yields a total for signed to ease the burden to workers of As pointed out by Leigh, a major displaced workers of 3.2 million a year economic change and presents the most problem with this legislation is the dif- over the time period 1979 to 1984. For recent evidence as to their relative ef- ficulty of identifying who the losers are the period 1980 to 1986, assuming a six fectiveness. Back to Work, by Howard from liberalized international trade (or year minimum job tenure, the estimate Bloom, reports with thorough method- any other structural change to the econ- of Jacobson et al. is 2.6 million. Tak- ological detail on one of the very few omy) and how much they lose. Current ing the midpoint of this range gives an classical random tests of programs de- policy is now summarized in the Eco- estimate of approximately two per cent signed to aid displaced and disadvan- nomic Dislocation and Worker Adjust- of the economy’s labor force each taged workers. These studies progress ment Assistance Act of 1988 (Leigh, year—more than 20% in ten years— from the most to the least general, and pp. 93-94). This latest revision was damaged by dislocation. are discussed in that order. motivated by evidence that prime age Leigh’s estimate is that the typical males were having difficulty adjusting displaced worker loses about 20% of Reasons for Concern to economic dislocation (Jacobson et his or her pre-displacement wages. Policy makers and analysts are inter- al.). During fiscal year 1991 the federal More conceptually sophisticated analy- ested in who gets damaged by econom- government spent nearly one billion sis using longitudinal data that incorpo- ic change, what the time pattern of eco- dollars to help displaced workers under rates a comparison group suggests that nomic loss is, and what the total this act. the initial annual loss is closer to 40% present value of that loss is. A key rea- Yet, to anticipate what is to come, (Jacobson et al.). Moreover, the aver- son for this interest is to determine very little is known about the broad age total present value of losses whether to intervene in the economy outcomes of various forms of labor amounts to about $80,000. This is a and assist the dislocated. The decision market interventions to aid displaced private loss to the individual but it is to intervene is in part a function of the workers other than job search assis- also a real loss of economic welfare to total earnings lost per dislocated work- tance. As Leigh’s thorough review of society as well. As Jacobson et al. er and the distribution of that loss the literature shows, even less is known point out “displacement is a major set- across groups in the labor force. Sec- about the fine detail of optimal pro- back for experienced workers.” ond, if society wishes to compensate gram design for different types and Using the Jacobson et al. estimates, the losers to any economic change, it mixes of treatments relative to their tar- the total annual cost of displacement to needs to know who the losers are and get populations. It is difficult to com- society is a present value of $208 bil- how much they have lost. prehend this long-standing tolerance of lion (2.6 million workers a year times After extensive Congressional hear- partial knowledge and outright igno- $80,000 per worker). What if these es- ings in the late 1950s, the Area Rede- rance of these policy outcomes. Yet, timates are four times too large? The such tolerance exists, even with recent

Evaluation Forum n Issue 11 n Summer 1995 17 Features n Economic Change and Displacement

loss is still $52 billion a year! Hopeful- Policies to Mitigate vouchers, to encourage firms to hire ly, the economic changes that cause Earnings Loss displaced workers. this dislocation provide society as a With the possible exception of some whole with benefits that equal or ex- 3. Job search assistance programs, in- of the wage premiums due to unioniza- cluding the components of: out- ceed the losses. However, ignoring tion, unanticipated1 economic disloca- Food Stamps and other transfer pay- reach; orientation; assessment and tion that reduces or eliminates any of ments, the direct compensation that is testing; job search work shops; the four sources of wage premia above paid by society’s beneficiaries to this result in a loss of human capital or situ- counseling; and job development change is only about a billion dollars a ational productivity. This in turn is and job matching. year, as noted above. translated into a loss of earnings power. 4. Advance notice and on site delivery Why Such Losses Occur These losses of productive capacity can of services. be seen as property right losses, and, The above estimates are large and hence, compensation is justified (some 5. Economic alternatives to plant shut- dramatic. What is their source? Con- would say required) on the grounds downs, including: subsidies of vari- ceptually, does the total sum represent that no one be made worse off by a giv- ous kinds to employers; assistance to a potentially compensable loss? Jacob- en economic change and at least one enable employee buyouts of firms; son et al. draw a careful taxonomy of person be made better off. The point is and subsidies to workers to start up the source of these losses. Wage pre- that for society to benefit, there must be their own enterprises. mia can derive from the following a net gain after all compensation for sources or combination of sources: costs is accomplished. The first four policies are intended to aid the transition of dislocated workers 1. Some workers gain wage premia due Leigh presents a complete taxonomy to new jobs and replace lost human to a collective bargaining agree- and critique of the policies that are in- capital. The final group of policies rep- ment. tended to replace lost earnings power resent efforts at direct job creation. due to dislocation. He makes the point The evidence as to the relative effec- 2. Some workers may have firm-, in- that while the most widely publicized dustry-, or occupation-specific hu- tiveness of each of these policy choices group of displaced workers come from is limited to say the least; however, man capital. unionized, high wage jobs in the manu- there is better evidence on some initia- 3. Some firms may offer higher wages facturing sector, in fact displaced work- tives than on others. ers overall are a very diverse group, and be able to select higher produc- differing by gender, ethnic origin, in- Occupational Training tivity workers as a result in certain dustry, occupation and labor market lo- situations. cation. These characteristics imply dif- Occupational training programs pro- 4. Some firms and certain sectors of ferent types and mixes of policies for vided at the state level include a variety of innovative components, including the labor market employ promotion each subgroup of displaced worker (Leigh). vouchers; active participation of em- practices that preclude lateral entry. ployers to design training, choose train- While he focuses on program poli- Leigh suggests that wage premia due ees, and hire the trained workers; per- cies that have been implemented and to collective bargaining agreements formance-based contracts; and the may not represent a component that evaluated in the several states, the fol- diversion of unemployment insurance lowing program policies represent the justifies compensation. However, there funds to pay for training. Leigh points full range of alternatives in today’s pol- is evidence that unionized workers out that the net impacts of these pro- work more intensely and are more pro- icy toolkit: grams largely have to be inferred from ductive than similar non-unionized 1. Formal classroom occupational evidence based on evaluation of federal workers and that, in response to higher training, with remedial or adult basic programs since the state initiatives gen- unit wage costs, firms will also select education where also needed. erally have not built in evaluation com- more productive workers as a result. ponents that incorporate comparison To the extent that this is true, then 2. On-the-job training, with direct or groups. An exception to this is the union wage premia lost due to disloca- indirect wage subsidies, including Texas initiatives discussed by Bloom. tion are compensable. Real productivi- There is now evidence with strong ty has been destroyed, not just union- 1There is a strong theoretical argument that internal validity based on the National induced wage rents. In sum, if any of only displacement losses due to unanticipated JTPA Evaluation conducted by Abt As- the four conditions are negatively af- economic change be compensated. One can ar- sociates Inc. in cooperation with the fected due to displacement, both short gue that where change is predictable the market will provide a compensating wage differential in Manpower Development Research Cor- term and long term losses can result. the form of a wage premium to balance out the poration that retraining has a positive expected capital loss. See Hamermesh, Cordes return for adult men and adult women. and Goldfarb, 1987. Sixteen Service Delivery Area (SDA) sites were judgmentally chosen with

18 Evaluation Forum n Issue 11 n Summer 1995 Features n Economic Change and Displacement

random assignment to treatment and sonable to stage the process of provid- workers and no necessary costs to em- control within each site. As with the ing reemployment services in order to ployers. Thus, advance notice appears sites evaluated by Bloom, adult women gain the most effective use of social re- to be a useful social policy. fared better than men. Other, non-ex- sources. The notion is to present all perimental evaluations of training re- workers with job search assistance Job Preservation through ported by Leigh, such as the Downriver first, withholding initially any wage Subsidies and Employee Community Conference Economic Re- subsidy, training voucher or other Buyouts adjustment Program, are too method- training services. Workers with fewer How appropriate are policy inter- ologically flawed to allow any defini- disabilities due to the dislocation will ventions designed to keep threatened tive judgment as to program outcomes be able to find jobs first. This strategy firms operating? Leigh discusses three (Leigh). was followed in the Downriver project alternatives: mentioned above and positive returns Bloom’s (1990) evaluation of three to job placement services were identi- 1. “Assistance to allow current owners Texas sites providing reemployment to keep operating or to attract new services for displaced and disadvan- fied, within the (severe) methodological limitations of that analysis. Leigh reports owners from outside the firm. taged workers demonstrates conclu- that in general job search assistance ser- sively that with the appropriate politi- 2. Assistance to encourage employee vices appear to pay off socially. cal will and careful attention to buyouts of existing facilities. experimental design and execution of Related to job search assistance is that design in an on-going program, it the reemployment bonus initiative 3. Assistance to enable displaced is possible to properly evaluate such in- wherein a cash bonus is paid to workers workers to start up their own small itiatives. His exhaustive account and to induce them to search more diligent- businesses.” record of design, program and evalua- ly for work. Experimental results for Evidence on all these initiatives is tion management, and analysis provide Illinois suggest such a policy does limited to a few qualitative studies. a blueprint for federal and state legisla- work, but non-experimental results in The first initiative is generally sub- tors who wish to discover whether pub- New Jersey suggest otherwise. The sumed under programs to establish “en- lic investments of this type are effec- weight of the evidence should go to the terprise zones.” The notion is to stimu- tive. We agree with his statement that experimental results, at this time, con- late economic development by the Texas evaluation “...demonstrated ditional on new evidence developing. reducing government regulation and the feasibility of conducting a high- lowering taxes in the zones. The ap- quality, randomized field experiment at Advance Notice parent reality of state enterprise zones, several sites simultaneously, within a Early intervention in the job reloca- though, is that total but not marginal modest budget and limited time frame” tion process makes intuitive sense. tax rates are reduced, and the apparent (Bloom). Workers are helped thereby to maintain benefits of increased government activ- Of course, the problem at the federal their self-esteem and job search ener- ity within the zones are reduced by the level is not one of inducing an individ- gies, and firms can be induced to coop- cost of complying to regulations in or- ual program site to agree to an experi- erate in any job development and re- der to receive any assistance. mental evaluation. The National JTPA training efforts more effectively. These The second strategy relies for its ef- Evaluation has demonstrated this. The concepts are incorporated in the Work- er Adjustment and Retraining Notifica- ficacy on the increased productivity in- problem at the federal level is to mar- duced in workers who now own the shall the political focus and energy to tion Act of 1988. Leigh points out that firm. For a variety of reasons, such a bring about nationally representative early notice does appear to reduce the private costs of adjustment to workers, consistent successful result has yet to experimental evaluations within the be established in practice. And, there is context of on-going programs. Given largely due to the fact that some work- the further downside that the workers the billions of dollars that have been ers quit before the plant closing as a re- sult of locating a new job prior to sev- typically have taken their diverse port- spent delivering subsidized employ- folio of retirement investments and ment and training programs since the erance from the threatened job. converted its capital value to a single Area Redevelopment Act was initiated However, it is possible that advance egg in a single basket—their threatened 36 years ago, it is not unreasonable to notice can disrupt the production effi- firm. Such an action simply is not pru- expect such an initiative. ciency of a firm in a variety of ways, so dent, especially if the original owners that gains to workers are then offset by Job Search Assistance are not willing to maintain a significant losses to stockholders. Plausible argu- stake in the firm. As Leigh carefully points out, it is ments in the evaluation literature exist difficult to identify initially which as to both positive and negative effects The third strategy has several prob- lems with it. First, if the new business- workers laid off due to economic on worker productivity and firm costs es would have started up anyway, change have suffered losses of human due to advance notice. However, the capital. This being the case, it is rea- weight of evidence suggests gains to which has to be a possibility, those re- ceiving the subsidy get a windfall gain.

Evaluation Forum n Issue 11 n Summer 1995 19 Features n Economic Change and Displacement

No net social benefit results. Second, education should be offered, yet states chinery in the Men’s Clothing In- since such a start-up firm would be have slighted this initiative in favor of dustry.” The Journal of Political subsidized, it would have lower costs economic development activities. Fi- Economy. April. and could conceivably displace an ex- nally, there simply is not enough evi- isting, otherwise efficient, firm or dence at this time to offer such subsi- Hamermesh, Daniel S., Joseph J. Cord- firms. dies to firms in an effort to maintain or es, and Robert S. Goldfarb. 1987. increase jobs. And workers who can As to the third point, few additional “Compensating Displaced Work- jobs may be generated and, indeed, in- succeed in difficult entrepreneurial en- ers—What, How Much, How?” in deavors are least likely to need the ser- ducing inexperienced individuals to at- Labor Market Adjustments in the vices tendered to truly dislocated work- tempt to be entrepreneurs may just ex- Pacific Basin. Peter T. Chinloy and ers. acerbate failure. Ernst W. Stromsdorfer. Editors. As a concluding point, given the Boston, MA: Klewer-Nijhoff. An Overall Judgment very large real losses to workers as well as the large amount of public resources Jacobson, Louis, Robert LaLonde and Convincing and detailed evidence Daniel Sullivan. 1993. The Costs of now exists on the extent and severity of devoted to assisting these workers, it is Worker Dislocation. Kalamazoo, earnings loss due to economic changes reasonable to expect more systematic of all types. The social costs are very evaluation of program effects, which MI: The W.E. Upjohn Institute for large indeed and warrant a systematic includes more use of experimental de- Employment Research. sign, but at the very least, the use of development and valid testing of poli- Leigh, Duane E. 1989. Assisting Dis- cies to deal with the problem. state-of-the-art quasi-experimental de- sign. placed Workers: Do the States Have The survey by Leigh reveals a very a Better Idea.” Kalamazoo, MI: rich and diverse set of policy options, The W.E. Upjohn Institute for Em- only the highlights of which are pre- ployment Research. sented in this survey. But the fact is Bibliography Myers, Robert J. 1929. “Occupational that but for a few of the policies, we Bloom, Howard S. 1990. Back to know little of their efficacy. Job Adjustment of Displaced Skilled Work: Testing Reemployment Ser- search assistance should be the core Workers.” The Journal of Political readjustment policy. It is relatively low vices for Displaced Workers. Economy. August. cost and avoids the problem of having Kalamazoo, MI: The W.E. Upjohn Schultz, George P. and Arnold R. We- to distinguish truly displaced workers Institute for Employment Research. ber. 1966. Strategies for the Dis- from those unemployed due to season- Borus, Michael E. 1964. The Econom- al, cyclical or short term frictional placed Worker. New York, NY: ic Effectiveness of Retraining the causes. The role of retraining is less Harper & Row. Unemployed. New Haven, CT: The clear. It seems clear that employers in Somers, Gerald G. Editor. 1968. Re- local labor markets must be involved in Yale University Press training the Unemployed. Madison, the training process. Training should Haber, William. 1925. “Workers’ WI: The University of Wisconsin be offered only for well specified needs Rights and the Introduction of Ma- and in the types and quantities of work- Press. ers needed. Apparently, more remedial

20 Evaluation Forum n Issue 11 n Summer 1995 Features n National Studies of Dislocated Worker Programs

National Studies of Dislocated Worker Programs Editorial Introduction numbers of plant closures and mass adequate, sufficient responses. layoffs increased concern about dislo- The authors discuss a variety of pro- In a comprehensive paper on dislo- cation. In 1982, Title III of JTPA re- grams and pilot projects spawned by cation by Robert W. Bednarzik, a senior sponded to that concern. And states anxiety over increasing dislocation. economist at the U.S. Department of began to finance their own retraining Five demonstration projects had been Labor, and Malcom Lovell, Director of programs for dislocated workers. Title undertaken under Title III over an eight- the Labor Management Institute at III was followed by the Economic Dislo- year period beginning in 1980. All were George Washington University, the au- cation and Worker Adjustment Assis- evaluated. Bednarzik and Lovell’s sum- thors discuss the problem of dislocation tance Act (EDWAA), which was to in- mary of the results for OJT and class- and its major causes, and review na- crease the E&T system’s provision of room training in the five projects shows tional projects seeking to reduce this assistance to the dislocated. a mixture of success and failure for the emerging characteristic of postindustrial There was reason to worry about same interventions in different projects. societies. dislocation, the authors contend. From A consistent finding, however, was that Written for the Competitiveness Poli- 1985 to 1989, 4.3 million workers had training, per se, did improve the earn- cy Council in 1992, the paper provides become dislocated. Most were in man- ings of dislocated workers over time, an excellent summary of expert opinion ufacturing, but dislocation was spilling and job search assistance seemed to and research regarding dislocation is- over into the service sector. EDWAA improve both employment and earn- sues in the 1980s, and the programs and TAA were serving about 150,000 ings. developed to address those issues in dislocated workers a year out of over A study of ten exemplary dislocated the U.S. and Europe. The authors place 800,000. In 1986, the Secretary of La- worker projects under Title III provided retraining and reemployment strategies bor’s Task Force on Worker Dislocation important insights about service strate- in the context of the need to make sub- took a serious look at the problem gies and service delivery processes, to stantial improvements in the quality of again, as well as the programs devel- which the authors give appropriate at- the American workforce to remain inter- oped to resolve it. Not surprisingly, they tention. But by 1989 EDWAA had re- nationally competitive. found that the workers at highest risk of placed Title III, emphasizing “rapid re- They locate the beginning of adjust- dislocation were those in high risk in- sponse,” labor-management ment assistance programs in the area dustries — operators, fabricators and cooperation, a more powerful role for of public transportation. The govern- laborers in mining, construction and substate areas, linkages with UI, the ment had concluded in the aftermath of manufacturing. Minorities and youth Employment Service and TAA, and consolidating certain transportation sys- were at particular risk, Bednarzik and longer-term retraining. There the paper tems that workers losing jobs in the Lovell point out, because of their con- ends, and our series of articles on na- public sector should be compensated centration in import-sensitive industries. tional evaluations begins. The well-con- by receiving assistance in finding new These workers experienced severe ceived and implemented studies re- ones. Similarly, changes causing job structural barriers to reemployment. viewed in these articles tell us much loss in the area of trade were followed Later in the 1980s we began to see about an ongoing national dislocated by the Trade Adjustment Assistance middle managers and other white collar worker program, EDWAA, about ED- program (TAA). workers in the ranks of the dislocated. WAA demonstration projects, and about The surge in unemployment in the Public policymakers began to ask DOL studies of self-employment. early 1980s related to unprecedented themselves if EDWAA and TAA were nn JTPA was accompanied by formal gov- One result was the conducting of Evaluating ernment interest in performance man- some of the most sophisticated and sci- Dislocated Worker agement and evaluation, likely related entifically viable evaluations of ongo- in part to the decentralization and ing and demonstration programs in the Programs: U.S. privatization of employment and train- employment and training field that had Department of Labor ing programs to states and local areas. ever been undertaken. Most of the pro- Among other purposes, federal moni- grams and projects evaluated have fo- Studies in the 1990s toring and evaluation guidelines and cused on the economically disadvan- standards were to compensate for this taged. However, in the 1990s the U.S. In the 1990s, the issue of worker devolution of power by maintaining Department of Labor (DOL) launched a displacement and dislocation became a more serious concern, growing more quality, consistency and equity. They number of evaluations of Title III pro- were to assure that a transfer of authori- grams and demonstrations, as well as problematic with increasing economic ty and administration to states would pilot tests of new strategies addressing competition and risk. Title III of JTPA was created to respond to this fluoresc- not result in a redirection of programs dislocation. toward state-specific agendas and away ing phenomenon in postindustrial life. Historically, dislocated workers from Congressional intent. were a new target group for services,

Evaluation Forum n Issue 11 n Summer 1995 21 Features n National Studies of Dislocated Worker Programs

encompassing workers of all socio-eco- delivery systems. All but one of the government has been sensitive to dis- nomic levels who were experiencing outcome studies utilized either random placement due to trade conditions as permanent job loss in the occupations, assignment to produce equivalent treat- early as the 1930s. The Trade Expan- industries and businesses that had sus- ed and control groups, or a constructed sion Act of 1962 began the recent his- tained them over time. Employment comparison group of individuals con- tory of compensating workers for trade- and training policymakers, planners sidered similar to those receiving pro- induced job loss thorough “trade and program designers faced a new gram services. adjustment assistance.” Although the challenge. And evaluations now began There is no question that measure- TAA program put in motion by this to investigate the nature and effects of legislation was reduced drastically in ment is a special challenge in process programs responsive to the middle the 1980s, legislation in 1986 reempha- studies. Many of the variables describ- class. ing program implementation that are sized adjustment services, 1988 legisla- tion expanded the use of training to likely to exert the greatest influence on The Status of Evaluation speed readjustment, and reemployment Research program outcomes are difficult to mea- sure. The variability in these measures services were subsequently given more attention. With the advent of EDWAA This article briefly reviews a subset across process evaluations reduces in 1988, coordination between TAA of the DOL-funded evaluations of pro- one’s ability to compare studies. grams for dislocated workers. The and the new program was mandated. The measurement of outcomes has 1990s witnessed increased government This linked TAA with a more disci- commitment to the use of experimental been limited traditionally to a small set plined and sophisticated service deliv- of easily quantifiable employment and research designs for estimating the ef- ery system. Clearly the legislative evo- earnings variables, to which measures fectiveness (return on investment) of lution of TAA reflected the growth in ongoing programs and demonstration have been added to provide informa- the population of dislocated workers in tion on interim outcomes such as the projects. At the same time there was this society. completion of educational and skill new support for learning more about The Process Evaluation the influence of organizational struc- competencies, and longer-term out- comes such as job retention. of TAA tures, functions and processes in pro- grams, and service delivery characteris- However, even these standard out- The purpose of the evaluation was to tics, on the outcomes of these programs come variables are not always defined learn more about 1) the certification, — that is, a growing interest in process identically. And less tangible and more notification and application processes, evaluations, or studies of program im- inclusive sets of outcomes for individu- and 2) the delivery of reemployment plementation. als, such as an increase in work motiva- services. Conclusions and recommen- tion and productivity, greater self-es- dations are presented in Figure 2. Where possible, the DOL encour- aged a “methodological partnership” teem in the workplace, or less stress in one’s family life, are rarely given atten- The Evaluations of the between process and net impact evalua- tion, even though these outcomes may Implementation of EDWAA tions when studying programs, which permitted evaluators to offer better ex- be highly significant. The design of EDWAA programs planations of the potential causes of Conceptualization differences also was dependent largely on the planning program outcomes. The process and net reduce the comparability of evalua- preferences and expertise of the states. impact evaluations of the WA State tions, even though studies may be fo- Consequently it was important for DOL SEED program are an example, as are cused on similar issues and target to know, from a public policy stand- the twin process/net impact evaluations groups. The underlying assumptions point, what designs seemed most ap- of the Trade Adjustment Assistance evaluators make in defining and fram- propriate to the needs of dislocated program. In other cases, process evalu- ing the research, and the paradigms workers and what configurations ap- ations preceded potential outcome stud- (propositions) that flow from these as- peared to produce the outcomes intend- ies, or were considered most appropri- sumptions subtly influence the con- ed by the Congress. Conclusions from ate when the focus was mainly on struction of research questions. the EDWAA evaluation are presented in Figure 3 and recommendations are implementation issues. These caveats should be kept in presented in Figure 4. Figure 1 provides a quick overview mind when looking across the conclu- of the kinds of evaluations conducted sions and recommendations of evalua- The Phase II report on EDWAA im- by DOL on dislocated worker pro- tors regarding these programs for dislo- plementation was reviewed in an earlier grams, the major reseach questions for cated workers. issue of Evaluation Forum and so is which answers were sought, and the not given detailed attention in this arti- methodologies used. The Evaluation of the Trade cle. The methodology of the evaluation Adjustment Assistance is quite unique. The research was guid- The Results of the Studies Program (TAA) ed by an innovative, thoughtful and useful model describing the elements of DOL’s process studies emphasized Although our concern about the dis- similar aspects of programs: their orga- located in the 1990s has been more dra- “a responsive service delivery system” for dislocated workers. Information nizational dimensions and their service matic than in earlier periods, the federal

22 Evaluation Forum n Issue 11 n Summer 1995 Features n National Studies of Dislocated Worker Programs

Figure 1 n An Overview of DOL Evaluations

Benus, Jacob and Rhonda Byrnes The St. Louis Metropolitan Re-Employment Project: An Impact Evalu- ation. Research and Evaluation Report Series, Employment and Training Administration, 1993.

Type / Design Main Research Questions Methods Net impact What are the net impacts of the services re- Data Collection evaluation / ceived by participants on their earnings, receipt n UI wage record data quasi-experimental of UI, and their reemployment over time? n Telephone interviews design n Program records Data Analysis n Treated group was compared with random- ly-selected sample of St. Louis residents fil- ing for UI.

Corson, Walter, Phillip Gleason, and Johanna Patch Evaluation of the Trade Adjustment Assistance Program: Process Report. Office of Strategic Planning and Policy Development, Employment and Training Administration, 1992. Type / Design Main Research Questions Methods

Process evaluation / n What is the nature of services provided? Data Collection Descriptive design n To what extent did 1988 amendments im- n Aggregate quarterly quantitative data on prove program? payment and service activity under each pe- tition. n What were “best practices” in terms of service delivery? n Unstructured on-site interviews with central office staff and some local staff. n How does TAA interact with other dislocated worker programs? n Participant survey. Data Analysis n Comparison of delivery of services over time, among states, by petitions.

Corson, Walter, Paul Decker, Phillip Gleason, and Walter Nicholson International Trade and Worker Dislocation: Evaluation of the Trade Adjustment Assistance Program. Office of Strategic Planning and Policy Development, Employment and Training Administration, 1993. Type / Design Main Research Questions Methods

Net impact n What are the differences in the characteris- Data Collection evaluation / tics of TAA caseloads and other dislocated n Participant survey quasi-experimental workers? n Telephone interviews with random sample design n What kinds of services were provided by of recipients comparison with participants in other pro- n Labor market data grams? Data Analysis n What effect did TAA services have on the re- n Comparison of client characteristics, servic- employment experiences of TAA participants, es received, and client outcomes for TAA as compared with other dislocated workers? treatment group with matched comparison n What effects did the 1988 amendments and group of UI claimants. changes in Title III have on TAA?

Figure 1 continues

Evaluation Forum n Issue 11 n Summer 1995 23 Features n National Studies of Dislocated Worker Programs

Figure 1 n Continued

Dickinson, Katherine, Deborah Kogan, Kevin Rogers, and Mary Visher Study of the Implementation of the Economic Dislocation and Worker Adjustment Assistance Act. Research and Evaluation Report Series, Employment and Training Administration, 1992. Type / Design Main Research Questions Methods

Process evaluation/ n What is the nature of EDWAA policies and Data Collection descriptive design organizational practices? n Telephone interviews n What are the operational links between n Site visits: discussions and interviews with 1) state and substate functions and 2) de- state, substate, and service providers sign and delivery of services? Data Analysis n Cross-site analysis of qualitative and quantita- tive data from program MIS and other sources.

Dickinson, Katherine, Deborah Kogan, Kevin Rogers, and David Drury Study of the Implementation of the Economic Dislocation and Worker Adjustment Assistance Act: Phase II, Responsiveness of Services. Research and Evaluation Report Series, Employment and Training Administration, 1993. Type / Design Main Research Questions Methods

Process evaluation / n What types of services are provided? Data Collection descriptive design n To what extent do these services respond to n Site visits. clients’ needs? n Discussion with state-level staff. n What are “best practices” regarding program n Interviews with substate staff. designs and service practices? n On-site observation by evaluation team. n What problems are occurring re: service de- n Discussion with participants. livery policies and practices? n Review of client files. Data Analysis n Comparison of collected information with an evaluator-constructed ideal-type “Conceptu- al Model of Responsive Dislocated Worker Services” to judge extent to which programs fit this model.

Dickinson, Katherine, Suzanne Kreutzer, Deborah Kogan, and Richard West Study of the Implementation of EDWAA: Report on the Survey of Substate Areas. Research and Evaluation Report Series, Employment and Training Administration, 1993. Type / Design Main Research Questions Methods

Process evaluation/ n What is the nature of the organization and Data Collection descriptive design operation re: EDWAA services such as type n Worker Adjustment Annual Program Report: of clients targeted, characteristics of services for types of clients served, recency of layoff, provided and their possible effects on out- educational level, previous wage level, barri- comes? ers to employment, program outcomes. n What is the orientation of programs and ex- n Survey data. tent to which they are consistent with the n Review of state plans. themes of their legislation, and what is the n Labor market information. possible effect on outcomes? n To what extent is EDWAA coordinated with Data Analysis Title IIA of JTPA, with what effects? n Survey results and other data were analyzed against the conceptual framework of proposi- n What is the probable effect on outcomes of tions regarding ways substate areas organized the level of responsiveness to client needs? their programs, the types of clients served, the types of responsiveness to needs, and the out- comes achieved, using statistical methods.

24 Evaluation Forum n Issue 11 n Summer 1995 Features n National Studies of Dislocated Worker Programs

Figure 1 n Continued

Drury, David, Stephen Walsh, and Marlene Strong Evaluation of the EDWAA Job Creation Demonstration. Research and Evaluation Report Series, Employment and Training Administration, 1994. Type / Design Main Research Questions Methods Process evaluation / How effective are Community Development Data Collection descriptive design Corporations in expanding employment oppor- n Site visits. tunities for dislocated workers through linkages n Interviews with staff from 11 substate pro- between entrepreneurial training and other eco- grams in eight states. nomic development activities? n Interviews with state-level EDWAA staff in five states. n Conferences. n Telephone interviews. Data Analysis n Analysis of an array of quantitative and qualitative information.

Visher, Mary, Stephen Walsh, and Ronald D’Amico Serving Dislocated Farmers: An Evaluation of the EDWAA Farmers and Ranchers Demonstrations. Research and Evaluation Report Series, Employment and Training Administration, 1994. Type / Design Main Research Questions Methods

Process/outcome n How effective are the service categories used Data Collection evaluation, descrip- in the demonstration? n DOL WAPR data. tive design n On-site interviews and discussions. Data Analysis n Services received compared with those re- ceived by other EDWAA participants. n Relationship analyzed between services and participants’ short- and long-term outcomes.

Wandner, Stephen (ed.). Self-Employment Programs for Unemployed Workers. Parts 1-4. Unemployment Insurance Occasional Papers, Employment and Training Administration, 1992.* Type / Design Main Research Questions Methods

Process evaluation/ n How viable is self-employment as reemploy- Data Collection descriptive design ment option for the unemployed? n Quantitative data on clients and services via n What is the relationship between 1) early pro- DOL’s Participant Tracking System. vision of business assistance services and n On-site discussions and interviews with self-employment allowances to UI claimants, SEED personnel using standard protocols, and 2) successful outcomes in self-employ- via monitoring visits. ment? n On-site group interviews with SEED participants. n On-site observation by evaluation team. Data Analysis n Cross-site analysis of quantitative/qualitative data on organizational arrangements, service delivery. Net impact evalua- n How effective are SEED services in terms of Data Collection tion/experimental self-employment and earnings? n UI wage records; telephone survey. design; cost/benefit Data Analysis study n The self-employment and earnings out- comes of SEED participants (offered busi- *1, From Unemployed to Self-Employed , Stephen Wandner and Jon Messenger; ness start-up training, a waiver of the UI 2, Washington Self-Employment Enterprise Development (SEED) Demonstration, Terry Johnson work requirements, and periodic payments and Janice Leonard; 3, Massachusetts Unemployment Insurance Self-Employment Demonstra- equal to UI benefits) were compared with a tion, Jacob Benus, Michelle Wood, Chris Napierala, and Terry Johnson; 4, Self-Employment control group that received only UI payments Programs for the Unemployed: An Analysis of Program Evaluation and Operations Research in Europe and North America, Douglas Scott. and services.

Evaluation Forum n Issue 11 n Summer 1995 25 Features n National Studies of Dislocated Worker Programs

about EDWAA implementation was Figure 2 n TAA Evaluation Conclusions and Recommendations then judged against this model. The major conclusion was that there was Issue Area substantial variation across the study Certification of groups of workers and notification of workers about TAA sites in the level of responsiveness of Conclusions Recommendations EDWAA to the needs of this target Most workers in certified groups did n States should obtain lists and con- group. Areas experiencing high levels receive notices about TAA but states tact workers who are laid off follow- of dislocation, and special projects lacked a systematic approach to iden- ing certification, and ask firms to tended to be the most responsive. tifying and notifying potentially eligible notify state agencies of subsequent workers. layoffs, following up with employers The third report in the series on ED- to assure that this occurs. WAA was based on a survey of sub- n States should use UI wage record state areas delivering EDWAA servic- and other data as a basis for identi- es. It addresses both process and fying potential eligibles. outcome issues. The results regarding n States should disseminate informa- service integation were particularly tion about TAA through brochures useful. Those substate areas with the and the media. greatest integration of services with JTPA IIA tended to have these charac- Issue Area teristics: Application to TAA n They served workers whose profiles Conclusions Recommendations were similar to the economically Just as certification and notification n Group orientations to TAA would disadvantaged. were occurring earlier in workers’ strengthen movement toward the spells of unemployment, so the appli- use of TAA services. n They utilized OJT as the primary cation process had been moved for- training option, but provided less re- ward. sponsive services. Issue Area n They provided weaker early inter- Delivery of Reemployment Services vention services. Conclusions Recommendations At this stage in EDWAA evolution, Substantial numbers of TAA clients re- n Staff needs to assist workers in incentive/sanction systems were in ceived long-term occupational training viewing reemployment options more place in several states, and the evalua- at a variety of public/private institu- realistically. tion indicated that performance incen- tions. Nearly all clients received Em- n Employment Service staff should tives appeared to increase substate ef- ployment Service job search services. become more informed about local/ forts to obtain positive outcomes. Also, A small number received payments to regional employment opportunities finance job search outside local ar- and appropriate training. eight of the study states provided finan- eas, and some received relocation al- cial incentives for providing long-term n TAA programs should work more lowances. Clients rarely used avail- closely with unions. training, which had the effect of in- able remedial education or OJT. creasing the intensity of that training. n States should offer placement ser- The main recommendations based on Some workers strongly resisted re- vices as well as job search servic- ceiving training. Staff sometimes es. the survey focused on developing re- lacked a detailed knowledge of labor sponsive services, organizing services n States should consider consolidat- market conditions and skill needs, and ing TAA and EDWAA. more effectively, and improving state therefore were vulnerable to making policies. These recommendations mir- inappropriate decisions about training rored many of those summarized in the options. review here of the first study. Most states placed little emphasis on post-training placement services. Evaluation of the EDWAA Mandated coordination between TAA Farmers and Ranchers and EDWAA was working well al- Demonstration though few sites actually integrated A major demonstration project fo- service delivery. cusing on rural dislocation, this pilot project involved four states. The evalu- ation was based on intensive site visits and produced insightful case studies. The demonstration was critical in that little has been known about the

26 Evaluation Forum n Issue 11 n Summer 1995 Features n National Studies of Dislocated Worker Programs

Figure 3 n EDWAA Evaluation Conclusions

Issue Area n Implementation Problems Economic development funds were sometimes used to re- Funds were underused given the extent of the needs, and train workers to stave off layoffs but not to serve already-dis- the services were received by only a small proportion of dis- located workers. Sometimes EDWAA funds were used to located workers. Job search assistance and short-term train- support economic development projects. ing were overemphasized and rapid response capacity was often weak. The requirement for matching funds had unin- Issue Area n Coordination with Other Programs tended effects on the type of services provided and the kind Coordination occurred mainly through joint efforts to deliver of workers served. services, particularly training. Sometimes EDWAA and feder- al vocational education funds, or EDWAA and federal finan- Issue Area n EDWAA Resource Use cial aid funds were used together. Title IIA programs and ED- Most states selected Service Delivery Areas (SDAs) as the WAA were frequently co-located administratively. Linkages organizations to receive substate EDWAA funds. States with other human service agencies were considered weaker, lacked adequate data for allocating funds — that is, data on such as with welfare agencies. plant closings and mass layoffs, declining industries, and farmer/rancher economic hardship. Only half the states felt Issue Area n Basic Readjustment Services resource allocation was consistent with the needs of the dis- Pre-layoff services, rapid response and basic readjustment located. services were often coordinated by a single agency. Some substate areas did not provide basic readjustment services if Issue Area n Targeting dislocated workers were not receiving training. Local areas While most states used inclusive definitions of eligibility, tar- were struggling to develop appropriate assessment strate- geting priorities and methods varied considerably across gies for this target group. Only one-third of the substate ar- states. eas were using a case management system. Classroom training was often perceived by dislocated work- Issue Area n Service Delivery Systems Most states supported the federal guidelines regarding ex- ers as offering better reemployment opportunities than OJT. penditure of 50% of funds on retraining, but few states en- However, classroom training was highly localized and diver- couraged substate areas to request waivers for training or to sified. Community and technical colleges were the most com- reward the provision of long-term training. mon providers. OJT positions had mixed benefits, sometimes more oriented to economic development goals than to the Issue Area n Local Program Organization and Design needs of workers. Few training options involved workplace- Some local areas partially integrated EDWAA with Title IIA based learning. service systems. Half of substate areas had focused on laid- off workers, one-third on the long-term unemployed. The ex- Issue Area n State Role tent of dislocation had directed these priorities. Most sub- The requirement that EDWAA funding be passed through state areas complied with the federal emphasis on retrain- states to substate areas tended to weaken the state’s leader- ing; about half (those with high levels of dislocation) had ship role in setting program parameters and assuring ac- emphasized long-term training. countability. In retaining 40% funds, however, states frag- mented the EDWAA delivery system, leading to policy confu- However, some administrators felt this emphasis denied the sion and a lack of policy coordination. importance of basic readjustment services, and led to an overuse of OJT. Over half of the local areas emphasized Issue Area n Program Accountability classroom training; however, about one-third emphasized Most states did not provide much technical assistance to OJT. Most offered only limited supportive services. Even substate areas. Reviews of reports to determine compliance fewer provided needs-related payments. Most states en- with regulations focused on procedural rather than program- couraged participation by labor and management but only a matic issues. All states had fiscal expenditure reporting sys- third had created formal labor-management committees. tems to insure compliance with federal fiscal rules, but states were not consistent in recapturing funds from delinquent Issue Area n Coordination with TAA, ES and UI SDAs. Most states had nonfinancial agreements with ES and UI, usually negotiated at the local level. Nearly two-thirds of States tended to retain a strong role in designing and admin- states provided EDWAA 40% funds to ES/UI for rapid re- istering 40% activities, budgeting for a variety of activities. sponse activities. Over half used these funds to contract for specific services to dislocated workers statewide. Issue Area n Coordination with Economic Development In terms of programmatic accountability, EDWAA perfor- Coordination tended to be limited to information sharing re: mance standards had little meaning, since no incentive or potential and actual layoffs. A few states used 40% funds for sanction system had been developed by states. Few had special coordination activities. Even so, companies’ layoff even developed criteria for reviewing and approving substate plans were frequently too far in advance of WARN notices to plans. allow for effective layoff prevention efforts.

Evaluation Forum n Issue 11 n Summer 1995 27 Features n National Studies of Dislocated Worker Programs

Figure 4 n EDWAA Evaluation Recommendations program than in other decentralized na- tional E&T programs to respond to na- n Substate capacity building should design of rapid response, on realiz- tional guidelines intended to maintain be supported. ing the benefits of reduced labor/ the program’s consistency, comparabil- n Resource management practices management tensions, and on im- ity and accountability nation-wide. should be improved. proving the range and availability of While EDWAA’s funding and organi- services that are responsive to dis- n Rapid response must be assured. zational framework allowed more ex- located workers’ needs. n Labor and management should join perimentation, this experimentation did government in the design and im- n States need to create a more coor- not always translate to progress in deal- plementation of service delivery. dinated statewide program, linking ing with dislocation. rapid response, recruitment, and n A coordinated use of program and early intervention services, and co- The current renewed interest in fed- other resources should be encour- ordinating plant-specific projects eral block grants consolidating federal- aged. with ongoing local service systems ly-sponsored E&T programs, and the n The coverage of the program to produce a more seamless deliv- accompanying emphasis on states’ and should be extended. ery of services. local level rights in the human services, n Longer-term retraining and reem- n States need to improve efforts to has not yet benefited from a serious ployment plans should be included identify the needs of dislocated in service options. consideration of consistency, compara- workers and tailor services to those bility and accountability issues sur- n EDWAA administrators and service needs, sharing “best practices mod- providers should concentrate on the els” with other states. rounding this proposal. It is possible that in attempting to resolve the issue of duplication and program prolifera- needs of this target group, despite the n The four projects varied greatly in tion in the federal system, we may un- wittingly produce a new form of human dramatic decline in American farms the way funds were used and in the service system fragmentation at subna- over the 1980s. Ten percent of the na- average cost per client. tion’s farms had disappeared within a tional levels. Under the block grant few years. Historically, urbanization n Half of all participants were still policy suggested as of the writing of and the consolidation of farming activi- farming three months after program this article, states, substate areas and lo- ties had led to a loss of more than four termination; two-thirds were en- cal communities will be encouraged to million farms over the past sixty years. gaged in off-farm employment, design and implement their own ver- sions of an E&T system, assisted only But the farm crisis of the 1980s was ex- many having had this kind of em- ceptional in that the movement out of by very general federal policy and ployment prior to program participa- oversight guidelines. This is likely to agriculture in this decade was no longer tion. mainly voluntary. This unprecedented result in programs whose operation and dislocation was having severe effects n Participation appeared to increase results cannot be compared appropri- on agriculture-dependent communities. employment rates, hours worked and ately with those of others, thus denying The demonstration was to identify the hourly wages, but only modestly — the Congress the basis for improving employment and training needs of this and there was no evidence that any national E&T policies. group of workers and determine how project was any more effective than Meanwhile, there are important les- best to meet them. another in delivering services or in sons from the EDWAA evaluations. The main findings of the evaluation of the outcomes. We suggest a few: the demonstration were these: n n Case management systems seemed Programs focused on dislocated n Participants tended to be in severe well-suited to this group, and re- workers must be unusually sophisti- financial distress despite high levels training appeared to be as appropri- cated and professional in their re- of education and off-farm jobs. ate for this group as for other dislo- cruitment and selection process. n Recruitment goals were met despite cated workers. n They should try to intervene prior to the reluctance of farmers to partici- or immediately after layoffs. Commentary pate in employment and training n They should provide a wide range of programs. The evaluations of EDWAA reveal income support and service options. the substantial variation across states n Outreach efforts appeared to in- and across substate areas in the way na- n They should construct individual crease enrollment. tional programs may be implemented. service plans with clients that are n Over 80% of the participants re- Since EDWAA funding policy gave based on competent client profiling, greatest authority and discretion to sub- ceived retraining. on a thorough knowledge of avail- state areas, it was more difficult in this able education, training and related

28 Evaluation Forum n Issue 11 n Summer 1995 Features n National Studies of Dislocated Worker Programs

nn services and how to access them, on Other National-Level namic economy pressured by interna- the most up-to-date and comprehen- tional competition. sive labor market information avail- Studies In the 1980s The Office of Technology Assess- able, on the key results of research and Early 1990s ment of the U.S. Congress produced a on dislocation, and on clients’ own revealing report in 1987 on one of the strong preferences for assistance. Relevant to U.S. Department of Labor’s traditional programs for workers dislocated due to n Dislocation They should have vigorous case trade, Trade Adjustment Assistance: management systems to provide on- The problem of dislocation is, his- New Ideas for an Old Program. Re- going support to clients as they take torically speaking, a recent one. The in- viewed intensively in an earlier issue of advantage of the program’s services crease in international economic com- Evaluation Forum, this report provided and benefits. petition and rapid changes in insights for revising the TAA program These suggestions require a highly technology have led to unprecedented that contributed directly to later interest business and industrial restructuring in consolidating TAA and EDWAA. trained staff with access to critical data- with long-term effects for workers. Lit- The authors praised TAA’s conceptual bases, service networks, and other sources of information relevant to the erally hundreds of analyses and evalua- design for its income-supported long- tions related to the issue of worker dis- term training, and EDWAA’s adminis- functions required of them. They must placement/dislocation due both to trative flexibility and broader coverage be able to invest employers and unions in becoming an integral part of the de- ordinary business cycles and to restruc- as well as its quicker and more compre- turing have been sponsored over the hensive service response, and suggest- sign and implementation of these pro- past decade, testifying to the serious- ed that the best features of the two pro- grams. They need to be generally fa- miliar with the characteristics of ness and importance of this phenome- grams be combined in a new one. non in all postindustrial economies. evaluation research so that they are Often overlooked, the report alerted This article alerts us to a few of these competent consumers and contractors policymakers to another aspect of TAA of research on their own programs. studies and reports. — it is virtually the sole federal program to provide sustained, intensive technical With regard to service options in Selected U.S. General dislocated worker programs, the DOL Accounting Office (GAO) Studies assistance to small and medium-sized manufacturers. Interestingly, the OTA re- monograph A Guide to Well-Developed A series of studies relevant to dislo- port viewed the negative results of a pre- Services for Dislocated Workers is a cation was commissioned by the U.S. veritable gold mine. This guide was in- vious evaluation of TAA by DOL’s In- Congress between the late 1980s and spector General’s Office as a less than tended for EDWAA practitioners at all the present. Figure 1 provides a brief fair judgment of its overall effectiveness. levels of government. Information for overview of their purposes and results, the guide was obtained from 70 sub- Mathematica Policy Research pro- and the GAO’s main recommendations. state areas and 10 special projects in 24 duced an informative report for the states, and focuses on the design, orga- Selected Studies in the 1980s U.S. Department of Labor and the New nization and provision of services to Jersey Department of Labor in 1989, dislocated workers between Program Stephen Baldwin’s report, Technol- The New Jersey Unemployment Insur- Years 1989 through 1992. The mono- ogy and Employment Policy, commis- ance Reemployment Demonstration graph provides detailed information on sioned by the National Commission for Project. This project examined the use the following service options: Employment Policy in 1987, was a of the UI system for identifying dis- careful exposé of the effects of techno- placed workers early in their spell of n Early intervention services. logical change on jobs and workers. As unemployment, and the provision of n Services to help clients develop ap- a form of structural change, or restruc- early intervention services to speed re- propriate reemployment plans. turing, technology effects are manifest- employment. The net effects of three ed, Baldwin demonstrated, in worker “treatments” were studied: 1) job n Basic readjustment services. displacement, retraining, and organiza- search assistance only; 2) job search as- n Retraining services. tional changes internal to firms for the sistance combined with training and re- purpose of taking better advantage of location assistance; and 3) job search Although this extensive guide was new markets and production processes. assistance combined with a cash bonus designed for program operators, it is an The benefits of technological change, for early reemployment. All three strat- excellent resource for policymakers, Baldwin pointed out, are not evenly egies resulted in net benefits to both in- planners, oversight personnel, and eval- distributed across workers, firms and dividuals and the society at large. uators. communities. Therefore a major policy implication of such change is the need Selected Studies in the 1990s for effective training and retraining Another useful report by OTA was programs that provide workers with the made available in 1990, Worker Train- skills needed to be productive in a dy- ing: Competing in the New Internation-

Evaluation Forum n Issue 11 n Summer 1995 29 Features n National Studies of Dislocated Worker Programs

Figure 1 n Selected GAO Reports

Dislocated Workers: Comparison of Assistance Programs (September 1992) Purpose and Type of Study Process study: to obtain information on services provided to dislocated workers by TAA and EDWAA in three states.

Results Major Recommendations

n TAA was serving a larger proportion of hard-to-place n A better information system for both programs should workers than EDWAA. be developed, to enable policymakers to determine n TAA participants received more income support and what services are received and what outcomes are were more likely to enter long-term training. achieved. n Both programs provided classroom training and OJT, but n TAA and EDWAA should be consolidated and serve all EDWAA placed more emphasis on OJT. dislocated workers, irrespective of the source of dislo- cation. n Both programs lacked comprehensive information on participant outcomes, but particularly TAA. n Income support should be provided with retraining. n Early intervention is essential. n There was only limited coordination between the two programs, partly due to complex TAA regulations. n A larger menu of assistance options should be avail- able.

Dislocated Workers: Improvements Made in the TAA Certification Process (October 1992) Purpose and Type of Study Process study: to explore the implications of the TAA certi- fication (eligibility determination) process in a random sam- ple of petitions. Results Major Recommendations

n Flaws were found in 63% of the petitions filed. Major n Certification process should be made more accurate, flaws were: and information more adequate for determining w incompleteness and inaccuracy of data collection from elgibility. companies. n States need to be more responsive to workers filing w Inadequacy of analyses of trade statistics. petitions. w Inadequate customer surveys. n TAA and EDWAA should be consolidated, and the need w Variations in volume of petition filings related to level for certification eliminated. of outreach and assistance provided to workers wish- ing to file petitions.

Dislocated Workers: Worker Adjustment and Retraining Notification Act (WARN) Not Meeting Its Goals (October, 1992) Purpose and Type of Study Process study: to investigate how well the WARN system was working in 11 states.

Results Major Recommendations

n While workers were more likely to receive prior notice of n DOL should be given the responsibility and authority to closures and layoffs, nearly half of the employers with vigorously enforce the WARN provisions. 100+ workers were not required to give notice. n Some employers appeared to be confused about WARN provisions, even though DOL brochures and seminars sought to educate them. n Many employers believed WARN benefited dislocated workers in terms of their reemployment but decreased company productivity. Figure 1 continues

30 Evaluation Forum n Issue 11 n Summer 1995 Features n National Studies of Dislocated Worker Programs

Figure 1 n Continued

Dislocated Workers: TAA Program Flawed (October 1993, Testimony) Purpose and Type of Study Outcome study: testimony summarizing DOL studies, such as Inspector General’s and Mathematica’s, and their impli- cations for policy. Results Major Recommendations

n TAA was not achieving its key goals: n Dislocated workers need to be assisted systematically w Benefits not equally accessible. and consistently irrespective of the source of disloca- w Slow program response. tion — i.e., whether dislocated under TAA, EDWAA, De- fense Conversion/Diversification, Clean Air, Transition w Services limited and not well tailored to needs. Assistance Program, Demonstration Centers for Re- w Not enough counseling. training Dislocated Workers, etc. w Waiver provision led to only half of TAA recipients n Programs need to be consolidated. receiving training. w Limited job placement assistance after training. w Inadequate information system for determining perfor- mance. n Separate funding sources have led to separate rules/ regulations which act as barriers to quick, equitable, comprehensive service provisions.

Dislocated Workers: Reemployment Assistance Program (November 1993) Purpose and Type of Study Process study: to explore the implications of the new re- employment program initiatives that specifically target dis- located workers (in an amendment to TAA considered a “bridging” program prior to legislation consolidating dislo- cated worker programs).

Results Major Recommendations

n The program had many of TAA’s shortcomings. n Processes should be developed that ensure that the certification process is quick and fair. n Activities in addition to classroom training, such as job search assistance, should be allowable in considering eligibility for income support.

Dislocated Workers: A Look Back at the Redwood Employment Training Programs (December 1993) Purpose and Type of Study Process/limited outcome study: to learn more about the im- plementation and outcomes of assistance programs relat- ed to the dislocation of timber workers in northern Califor- nia in the late 1970s. Results Major Recommendations

n Many workers received wage replacement benefits or n Receipt of benefits should be closely tied to participa- severance pay, but few received training. tion in training. n Service providers were reluctant to develop retraining n New job opportunities must be created in communities programs. affected by major layoffs, in addition to providing re- n Data on participant outcomes were not adequate enough training and income support. to determine what happened to workers after receiving services. n Data on outcomes for communities were not sufficient to sort out the effects of the programs. Figure 1 continues

Evaluation Forum n Issue 11 n Summer 1995 31 Features n National Studies of Dislocated Worker Programs

Figure 1 n Continued

Dislocated Workers: An Early Look at the NAFTA Transitional Adjustment Assistance Program (November 1994) Purpose and Type of Study Process study: to review in four states DOL’s effort to make changes in this transitional program (modeled on TAA) to 1) shorten the certification process, 2) include states in that process, 3) broaden eligibility regulations, 4) tie income support more closely with retraining, and 5) eliminate waivers. Results Major Recommendations

n DOL had successfully addressed a number of shortcom- n More consistent and comprehensive assistance should ings. be assured to all eligible workers. n Secondary workers (those indirectly affected by NAFTA) n A better performance monitoring system should be de- still faced problems in accessing benefits. veloped for participants’ services and outcomes. n Some workers were still receiving incomplete assess- ments and remedial assistance, and a limited mix of ser- vices. n Performance management remained inadequate. al Economy. Also reviewed in more de- placed Workers for the association’s to firms that were members of AMA, tail in an earlier issue of the journal, corporate members. The report indicat- the results were particularly useful this OTA study examined sixteen op- ed the following: since the study involved one of the few tions for training policy regarding the employer-based surveys conducted in n Forty-six percent of the firms sur- needs of the economically disadvan- the 1990s. taged, displaced workers, workers with veyed reported downsizing within The importance of an adequate and special needs, and the general popula- the past year. comprehensive occupational informa- tion of employed workers. These op- n The cuts made by these firms in- tion system was emphasized in David tions were organized within four issue creased to an average of 10.4% of Stevens’ 1993 monograph for the Na- areas: 1) reducing barriers to company their workforces. tional Occupational Information Coor- training; 2) upgrading workers’ skills; dinating Committee, Occupational In- 3) providing training and technology n Sixty-six percent of the downsizing formation: The “Blue Highways” of assistance; and 4) enhancing the quality was due to companies’ concerns the Labor Market. The paper pointed to and effectiveness of training. The re- about future business downturns. the utility of occupational information port thoroughly analyzed these options, in helping users become more discrimi- referring to policy analyses and re- For the first time, a majority of the jobs eliminated by firms in the survey nating and skillful in selecting and search conducted about them, and rec- translating published statistics to meet ommended certain options for policy involved supervisors, middle managers, specific policy, planning, management consideration. professionals and technicians. The number of firms offering outplacement and oversight needs, and in heightening A study conducted in 1993 by the services to dislocated workers was at a users’ sensitivity to workplace changes Congressional Budget Office examined seven-year peak. The report confirmed they can see are occurring but about lessons learned in obtaining informa- that workforce reductions were occur- which they do not yet have confirming tion about the actual experiences of dis- ring more frequently. Fewer than half statistics. located workers. The report, Displaced of the companies downsizing had in- Workers: Trends in the 1980s and Im- creased their profits or their worker Some Comments on the Studies plications for the Future, detailed the productivity as a result. The larger The recommendations in the reports number, characteristics and experiences companies were more likely to trim referred to involve something of a con- of a large sample of dislocated workers, jobs than smaller ones, but small busi- sensus on the desirability of: 1) weld- and the implications of this information nesses cut more deeply when they did ing income support to longer-term re- for federal policy. downsize. Firms that had downsized training, 2) providing a comprehensive Another study in 1993 that provided since 1989 (70% of the survey sample!) set of service options, 3) consolidating insights about dislocation was spon- were cautious about hiring new work- programs so that all dislocated workers sored by the American Management ers and had tended to lengthen hours are eligible for a unified program, irre- Association (AMA). AMA conducted a and/or overtime as an alternative. Few- spective of the source of their dislocation, survey regarding workforce reductions, er than half of all the firms surveyed and 4) increasing administrative flexibili- producing the 1993 AMA Survey on had increased their workforce since ty by removing regulatory barriers to ef- Downsizing and Assistance to Dis- 1988. Although the survey was limited fective program implementation.

32 Evaluation Forum n Issue 11 n Summer 1995 Features n National Studies of Dislocated Worker Programs nn centrated their attention on small busi- tional UI-related projects testing the Self-Employment as ness development as an economic de- use of reemployment bonuses as incen- an Employment velopment tool, largely to the neglect tives for an early return to work for dis- Strategy for of microenterprise development. located workers in Washington State Rather than serving economic devel- and Pennsylvania. And in the late 1980s DOL funded the St. Louis Met- Dislocated Workers opment needs, self-employment pro- ropolitan Reemployment Project, The two major self-employment grams in other industrial nations have been designed to expand employment which was evaluated using a quasi-ex- projects funded by DOL in 1988, and perimental research design. opportunities. By the mid-1980s the reported on in 1992, in the monograph Self-Employment Programs for Unem- seventeen advanced industrial countries In 1987, the Department funded a who were members of OECD were of- project to test self-employment strate- ployed Workers, were targeted to UI fering formal self-employment pro- gies. Washington State was selected as claimants on permanent layoff. Howev- er, entrepreneurial self-employment grams for the unemployed. According the test site. In the same year, however, to the DOL report, by 1987 nearly 20% the Omnibus Budget Reconciliation strategies had been of major interest in of new business development in Great Act required the Secretary of Labor to Europe for some time, as an option for coping not only with cyclical unem- Britain was due to self-employment; al- test self-employment programs in a most 25% in France. multi-year project involving three ployment and underemployment but states. In this second effort to test self- also with dislocation due to restructur- At the time the DOL report was ing and downsizing of workforces. written two major self- employment employment approaches, DOL, the four states, and the research contractor joint- models had been developed in Europe: The Organization for Economic Co- ly designed the strategy to be tested and operation and Development, for exam- n The “French Model,” involving the evaluation plan. The design com- ple, housed key experts on programs lump sum self-employment allow- pleted in 1989 also drew heavily from throughout Europe that were utilizing ancs that were to provide part of the insights gained in a study of self-em- self-employment as an alternative to re- capital needed for a microenterprise. ployment projects in Great Britain, employment. OECD was also interested France and Sweden, funded by the Ger- in the distinctive challenge this option n The “British Model,” involving peri- man Marshall Fund. Concerned about involved for evaluators, sponsoring an odic payments as a form of income potential costs to states, two of the international seminar on the evaluation support while developing and oper- states withdrew from the project, leav- of “Self-Employment and Employment ating a new microenterprise, such as ing Washington and Massachusetts as Creation Schemes” in 1989. UI in the U.S. the leading test sites. The DOL report was very useful in Although the European effort was The Washington SEED Project suggesting how experimental evalua- not rigorously evaluated, descriptive tions can benefit policymakers and pro- information was suggesting that self- The Washington project followed gram designers in developing success- employment was a significant option the French model (lump sum payments) ful self-employment programs. It also for the unemployed and dislocated. while Massachusetts applied the British made an important contribution to the model (periodic payments). Both in- knowledge base regarding European U.S. Approaches volved evaluations using an experimen- entrepreneurial initiatives. tal research design requiring the ran- In 1985, DOL became interested in dom assignment of a pool of UI Self-Employment Programs testing new strategies for dislocated claimants into a group receiving self- in Europe workers, and using an experimental re- employment services and subsidies, search design to evaluate their effec- and a control group not receiving them. The DOL monograph distinguises tiveness. The major elements of these between “microenterprise develop- strategies were: 1) early identification In the Washington project, claimants ment,” which is used synonymously and profiling of dislocated workers us- interested in the project, and potentially with “self-employment,” and “small ing UI data, 2) referral of dislocated subject to random assignment, were re- business development,” which is im- workers to providers of reemployment quired first to attend an orientation ses- portant to the definition of the enter- services, and 3) the linking of UI and sion as a first step in the application prise strategy DOL was testing and reemployment services within a single process. Of the 44,456 UI claimants in- evaluating. Microenterprises were seamless delivery system for these vited to participate in the project, only viewed as sole proprietorships with workers. The set of reemployment ser- 7% attended the orientation. Of the only one or a few employees. Small vices of greatest interest were job group attending the orientation, 61% businesses were defined by the Small search assistance, training, relocation subsequently applied and were random- Business Administration as having less assistance, reemployment bonuses, and ly assigned. Consequently, the project than 500 employees. The DOL report self-employment allowances paired was testing the self- employment strate- reminded us that microenterprises as with business support services. New gy on a small, self-selected group of well as small businesses were growing Jersey was selected as the first test site. claimants. Although the evaluation of rapidly. But public policymakers at Later the Department funded two addi- SEED was featured in an earlier issue both the federal and state level had con-

Evaluation Forum n Issue 11 n Summer 1995 33 Features n National Studies of Dislocated Worker Programs

of Evaluation Forum, we repeat some during the demonstration period, n Participants interested in self-em- of the main findings here, which were and reduced total earnings from this ployment will more likely be male, reported in detail in DOL’s monograph, form of employment. white, middle-aged and educated First Impact Analysis of theWashington than most UI claimants, and from n Both gross business income and tax- State Self-Employment and Enterprise professional/technical/managerial Development Demonstration. es were significantly higher for the occupations. Women, however, are treatment group. Program Implementation: The likely to be overrepresented in this Process Study n The treatment group worked fewer select group of claimants. hours and obtained lower earnings n The dislocated workers in the study in UI-covered employment. The EDWAA Job Creation groups were older and more educat- Demonstration ed than the larger universe of UI n If self-employment and wage/salary With the decentralization of the bulk claimants, and more likely to have employment were considered to- of EDWAA funding to Service Deliv- been in higher-wage white collar oc- gether, the demonstration increased the likelihood of being employed ery Areas or local community organiza- cupations. Many had an employed tions, the role of Community Develop- and remaining employed, but had no spouse and substantial assets. ment Corporations (CDCs) and other significant effect on total earnings or n Clients achieving five interim out- community-based nonprofits in deliver- average monthly earnings. ing services to dislocated workers be- comes received a lump-sum pay- n came more important. DOL’s multi- ment equal to their remaining UI Although the demonstration reduced year Job Creation Demonstration was benefits. These outcomes were: the length of UI payments, it in- creased the amount of total benefits to study this role, and the evaluation 1) completion of the training pro- was to judge its effectiveness. gram, 2) development of an ap- when the lump sum payment is in- The emphasis in this project was on proved business plan, 3) the estab- cluded. job creation through self-employment. lishment of a business bank account, The evaluators also gave attention These were the main results of the eval- 4) compliance with licensing re- to job satisfaction and job creation for uation: quirements, and 5) the obtaining of other workers. They found that the adequate financing for their pro- demonstration netted 49 new jobs for n Participants’ characteristics varied posed business. nonparticipants, although these went considerably from one site to anoth- mainly to family members, and there er, but overall they were older and n The program treatments clients felt was no effect on job satisfaction. better educated than the typical dis- were most useful were business located worker population. training, the waiver of the UI work The Massachusetts Project search requirement, and the lump- n The selection of applicants also var- This self-employment demonstration ied greatly, ranging from selection sum payment. followed the British model. Clients inter- strategies that involved sophisticated ested in applying were required to attend Program Results: The Net a training seminar consisting of a series screening to self selection. The latter Impact Study of training workshops and counseling led to a high attrition rate; the former was quite successful. n The project increased the likelihood sessions. An additional condition was of being self-employed by about six that those receiving self-employment al- n Curriculum design using an “adult lowances would have to work full-time months, a bit more for women than learning approach,” extensive staff on microenterprise development. for men. and peer support, and substantial Only about 4% of the 26,170 UI technical assistance to individuals n Both treatment and control groups claimants invited to attend the seminar appeared to be critical to success. started primarily service sector busi- did so. Of this number, 63% applied nesses. and were randomly assigned. Again the n Staff qualifications that included practical business experience and a n Approximately a third of these busi- study group was small and self-selected. nesses failed in the first year, but the Although the DOL monograph on self- high degree of commitment were employment projects was written prior to demonstration did not appear to af- important. obtaining final impact results, the evalua- fect this outcome. tors did come to a few early conclusions: n Development of business plans was an essential component of training. n Average gross monthly earnings from n It is likely that self-employment is self-employment were increased by appropriate only for an estimated n Lack of access to revolving loan approximately $150/month. 5% of UI claimants. funds and other sources of capital was the most serious barrier to n The demonstration reduced slightly n Self-employment businesses will achieving demonstration goals. the likelihood that clients would par- likely be in the service sector. ticipate in wage/salary employment n Strategies for designing services and

34 Evaluation Forum n Issue 11 n Summer 1995 Features n National Studies of Dislocated Worker Programs

service delivery systems converged produced marketable skills even if Self-Employment in Europe across sites over the course of the they were not immediately applied to The section of the DOL monograph demonstration, but organizational starting a business. on self-employment directs our atten- problems interrupted services and n Earnings from self-employment tion to “European Experience with led to inappropriate hiring decisions, were generally disappointing. Earn- Self-Employment Programs,” by poor communication with partici- Stephen Wandner and colleagues. It ings varied widely from business to pants, and reduced efficiency. provides a useful if brief overview of business, and net business income European strategies. Although these n Coordination with EDWAA was averaged only about $1000/month strategies have undergone policy mixed, whereas the CDCs worked when measured at the six month fol- changes with increasing unemployment well with the UI system. However, low-up and $582/month at the one- and slimmer government human ser- the specter of UI sanctions affected year follow-up. These self-employ- vice budgets, it is important to review data collection for the evaluation, ment earnings were much lower than the strategies that characterized the late since some participants did not re- the $12 average hourly wage the par- 1980s. But we should notice that Cana- port their previous self-employment ticipants had earned prior to layoff. da and Australia have also been inter- experiences and understated their Pre-post studies of gross incomes can ested in self-employment approaches. earnings from businesses begun as be misleading. However, on average, These were some of the characteris- part of the demonstration. it appears clear that dislocated workers tics of European self-employment pro- tend to earn less in their new jobs. grams in 1988: n In the short term, the demonstrations were successful in launching new n The outcomes for demonstration n Their primary purpose was to pro- businesses and creating secondary participants who did not start busi- mote self-employment (microenter- employment — 45% of participants nesses of their own but rather sought prise development) among the un- started up businesses and 29% found reemployment were more positive, employed, to contribute to solutions wage and salary employment. matching those of the more tradi- to unemployment. Among the businesses begun, 76% tional EDWAA programs in the n The target groups of these programs were still in operation a year later. same areas — 79% entered unsubsi- usually involved those eligible for dized employment. n Comparing the Job Creation Dem- income support payments through onstration with the Washington The evaluators made a number of national unemployment insurance State and Massachusetts UI demon- recommendations. If immediate wage programs, but some included the strations and five other projects sug- replacement were to be a participant’s long-term unemployed and those re- gested that the demonstration’s primary goal, they suggested that self- ceiving social welfare benefits. gross outcomes, such as business employment did not appear to be a fea- sible alternative in the short term. If n The services ordinarily included fi- start-up and short-term survival participants and program sponsors were nancial assistance and supportive rates, were on a par with other mi- prepared to absorb a certain level of services. The definition of “support- croenterprise programs undertaken risk, microenterprise strategies were ive” was services designed to assist in the same general period. recommended as offering longer-term workers in planning, establishing n Service industries were the most benefits. and operating the businesses they prevalent type of start-up business- For a small subset of dislocated were creating. Financial assistance es, even though only 22% of the par- workers, who were aware of the risks was offered in 1) lump-sum pay- ticipants had been dislocated from and the effort required, were highly ments equal to all or part of the this sector. motivated and prepared to develop a funds remaining in an individual’s specific business concept, self-employ- insurance entitlement, which was to n Comparing employment outcomes at ment was recommended as a viable support initial capitalization of his follow-up with those of EDWAA strategy. But even workers with these or her business, or 2) periodic weekly substate areas indicated that the qualities needed access to capital as or bi-weekly payments from the insur- demonstration had not produced as well as training to be successful, the ance system to give a worker continu- beneficial employment outcomes as evaluators concluded. Since a self-em- ing income in the early stages of plan- were characteristic of EDWAA. But ployment strategy was distinctly differ- ning and operating a new business. when both placements in wage and ent from a reemployment approach, the The payments often lasted for a year. salary jobs and business start-ups were evaluators recommended that current considered, the six Job Creation sites performance management systems uti- n A comprehensive package of servic- lize a set of variables and measures had an average employment rate of es included a number of business-re- more responsive to the uniqueness of lated services, although the ingredi- 74% compared to EDWAA’s 67%. microenterprise development, in judg- The evaluators concluded that the ac- ing its outcomes. ents of the package varied greatly quisition of entrepreneurial training from country to country. These ser-

Evaluation Forum n Issue 11 n Summer 1995 35 Features n National Studies of Dislocated Worker Programs

vices encouraged business counsel- programs wherever found, based on an the JOBS program. ing, entrepreneurial training, techni- expert panel developed by the Organi- In considering what to propose as a zation for Economic Cooperation and cal assistance, exemptions from common methodology for evaluating Development (OECD) in Paris. certain business taxes and legal re- diverse self-employment programs, the quirements, and preferential access The conclusion of the authors, who authors turned to the OECD panel. This to business loans and grants outside remain unamed in the paper, is that group of experts identified six major is- the programs. most European programs have been sues that could and should be studied: studied using a descriptive research de- n n How important the provision of In some countries, microenterprise sign and survey methods, particularly government assistance for business development was limited to certain surveys of participants. Periodic sur- economic sectors, and to special cat- veys have formed a longitudinal data- start-up really is. egories of the unemployed. base for analyzing process issues and n To what extent business ventures Based on selective gross outcome gross outcomes. generated by programs would have studies, both the French and British In general, however, European E&T occurred in their absence. models appeared to have a positive ef- programs were not systematically eval- n Whether the methods used to target fect on employment and economic de- uated in the 1980s. Of the 17 OECD services to subgroups within the eli- velopment. Compared to the actual countries with operating self-employ- numbers of unemployed workers ment programs, 10 had not produced gible population are effective. served by American demonstration evaluation reports. Most of the evalua- n How long program effects seem to projects, these programs enrolled large tions conducted had been limited to last. numbers of the unemployed. In France, process issues, in the context of weak the Chomeurs Createurs program en- public sector support for and experi- n How cost-effective programs actual- rolled 72,000 in 1986. The British En- ence with impact evaluations. In the ly are. terprise Allowance Scheme exceeded E&T field, this may have been related n How programs affect potential fu- 100,000 participants in 1987. However, to the tendency for European universi- ture dependence on unemployment the prediction of the evaluators study- ties to produce many more excellent ing the U.S. demonstrations was con- theoretical economists than applied insurance and other income support sistent with the French and British ex- econometricians. programs. perience, namely that under 5% of the In their discussion, a major differ- Appreciative of these issues, the au- unemployed were appropriate for self- ence surfaces between contemporary thors of the paper suggest a very simple employment programs. American evaluation approaches to evaluation paradigm. Program elements Increasing familiarity with self-em- new program initiatives and European are classified either as “inputs” or “out- ployment efforts in Europe led to sub- orientations. With respect to self-em- puts,” and the evaluator’s task is to de- stantial transnational borrowing of self- ployment programs, for example, Euro- termine how the inputs affect the out- employment concepts. Enough so that pean governments have not been under puts, or at least what kinds of the National Governors’ Association pressure to test new ideas through dem- relationships exist between them. In- and the Corporation for Enterprise De- onstration projects. Programs there are puts are: 1) targeting services, 2) vet- velopment jointly sponsored a national more often than not the products of ting service (selecting among the busi- conference on self-employment strate- government agency policies that tend ness plans submitted), 3) financial gies in 1987. The conference brought not to be dependent on the results of services, and 4) support services (an ar- experts from Europe, Canada and else- evaluations. ray of technical business-related servic- where to address self-employment is- es). Outputs for participants are em- Under public pressure to establish sues at the time Americans were tour- ployment and business retention. accountability, aided and abetted by the Outputs related to economic develop- ing European programs under the elaboration of the evaluation research German Marshall Fund. The borrow- ment goals are the generation of eco- field, the pattern in the U.S., particu- ing, however, was mainly by Ameri- nomic development activities and the larly beginning in the 1980s, was to creation of new jobs. cans from Europeans. test innovations or reforms in well- planned small-scale pilot projects that Emphasis is placed, in this para- Evaluating Self-Employment digm, on 1) the adequacy of databases Programs in Europe could be evaluated using experimental or quasi-experimental research designs supporting evaluation, including infor- The Appendix of the DOL mono- prior to full-scale adoption. Exceptions mation on the background and charac- graph contains an interesting UI Occa- existed, of course, such as JTPA, al- teristics of clients, information about sional Paper on “Self-Employment Pro- though its design benefited indirectly the services received, and performance grams for the Unemployed: Program from previous E&T research on CETA indicators, and 2) the collection of data Operation and Policy Research in and other programs. And a large num- over time. The authors also suggest a Western Industrialized Countries.” Its ber of experimental evaluations of series of indicators which they classify intent is to propose a common method- work/welfare demonstrations preceded within these categories: 1) employment ology for evaluating self-employment and were fed directly into the design of for the proprietor, 2) enhanced business

36 Evaluation Forum n Issue 11 n Summer 1995 Features n Defense Conversion and Dislocation

survival rate, 3) contribution to the lo- main lessons are that self-employment lored to the individual and his or her cal and macro economy, and 4) new job is most appropriate for a very small, se- business, with ongoing case manage- creation. They appropriately recom- lect group of dislocated workers who ment that emphasizes technical assis- mend the use of measures of job dis- are highly motivated, disciplined and tance as well as other kinds of support. placement. They review problems with planning-oriented in the way they go This is a somewhat different concept descriptive and quasi-experimental re- about developing businesses supported of microenterprise development than in search designs and advocate for experi- by government income support and Eastern Europe and third world coun- mental approaches in determining pro- training. These program participants tries, where it is offered as a significant gram effects. must also be risk-takers of a sort, will- intervention for large groups of unem- ing to try something new with all the Commentary on the Wisdom of ployed workers. For example, in the attendant anxieties. The results from early 1980s the Chinese government Self-Employment Strategies for the final report may shed more light on stressed self-employment as a major di- Dislocated and Otherwise the effectiveness of self-employment Unemployed Workers mension of the liberalization of the strategies. economy, only to find that business We have learned a great deal from Therefore it looks as if effective re- failures outnumbered successes five the projects generated by the increase cruitment and screening strategies will years later. Carefully targeted self-em- in dislocation in the 1980s and the U.S. be essential to install at the front end of ployment programs, on the other hand, government’s strong response in ex- microenterprise development programs. appear to have contributed effectively ploring a range of service options using Ways to sustain motivation and disci- both to job creation and to economic state-of-the-art evaluation research pline will surely be important. And it is development. principles and methods. Some of the fairly clear that the training must be tai-

Defense Conversion and Dislocation Editorial Introduction to the resolution of pressing societal flicts, and about the impact of defense problems. But the changing budget for- restructuring on military personnel, de- Cutbacks in the defense budget re- tunes of the Department of Defense fense-dependent communities, and de- lated to the end of the Cold War have have not yielded all the positive effects fense-related businesses and indus- been heralded by the American public anticipated. Anxiety remains about de- tries. The following set of articles as a bonus, a symbol of a less danger- fense readiness in a world complicated explore some of the issues involved in ous world, a chance to transfer savings by ethnic, religious and territorial con- this phenomenon of the 1990s. nn Role of Education in Restructuring De- Background Information Defense Downsizing: fense and Other Industries reminds us At the end of fiscal year 1987, the Some Implications that the U.S. Department of Defense active duty military was at a post-Viet- has been and remains the nation’s larg- nam peak of 2,174,217 positions. The The Cold War ended in dramatic est employer. Furthermore, as an em- changes in the political economy of de- National Defense Authorization Act of ployer it is known for its integration of 1991 projected a strength of 1,613,000 fense in the U.S. Its demise spawned a education and training with the culture positions by the end of 1995. This was new vocabulary as well — terms such of work, and its incorporation of new about a 25% reduction. By 1997, De- as “drawdown,” “downsizing,” “con- technology within skills acquisition. version,” and “transition.”1 The impli- partment of Defense outlays were ex- cations of the end of the Cold War for Given rather unique opportunities pected to decline by approximately workers affected by reduced defense for education, training and retraining 30%. This could represent a loss of as spending has been equally dramatic. within the military, unprecedented many as 800,000 military and civilian The U.S. Department of Education’s drawdown and downsizing in the 1990s Department of Defense (DOD) jobs be- 1993 report, The Conference on the represent an unusual test of the rele- tween 1987 and 1997, and could affect vance of military training for the civil- as many as a million additional private ian economy. Since these changes have sector jobs over the next several years. 1The Defense Conversion Commission with- in the U.S. Department of Defense defines con- involved a major dislocation of work- Although the U.S. has always lost ers within and outside the military, in- version as “the process by which the people, and replaced large numbers of military skills, technology, equipment and facilities of novative strategies have been essential personnel every year, this significant defense are shifted into alternative economic ap- to reducing the negative consequences plications.” It views conversion as “a process of multi-year downsizing effort has trans- transition.” of dislocation.

Evaluation Forum n Issue 11 n Summer 1995 37 Features n Defense Conversion and Dislocation

lated to far greater personnel change personnel, which is important. But we recommended a set of principles and than is traditional, with the potential for still have issues to resolve regarding in- criteria for evaluating these programs, a high level of involuntary separations. voluntary separation. The GAO report with strong emphasis on specifying ob- Clearly there has been a long-term re- recommended that the Congress con- jectives and using quantitative mea- structuring of the nation’s defense pri- sider extending the use of financial sep- sures to judge how well these objec- orities. As a result, the Congress autho- aration incentives beyond 1995, and tives were being achieved. Greater rized a range of assistance efforts in the amending the legislation to include in coordination across old and new pro- 1990s to facilitate downsizing, while the eligible pool for incentives all per- grams was given high priority. Overall, minimizing its impact on the individu- sonnel who have attained six years of the Commission felt that the conversion als, businesses and communities affect- service within the timeframe for which process was, in itself, an opportunity to ed. these incentives are authorized. make improvements.

The first priority of the Defense De- The Role of the Defense This upbeat attitude was reinforced partment has been to achieve voluntary Conversion Commission by the Commission’s computer simula- reductions. It has successfully drawn tions regarding the longer-term conse- down military forces across various The creation of the Commission quences of the drawdown. These were skill and seniority sectors using strate- within DOD was an important step in generally optimistic. Analysts predicted gies that will have the least negative dealing with the issue of dislocation. It that the drawdown’s main short-run ef- impact while maintaining the level of provided a framework for monitoring fect would be a slight downward pres- experience needed. However, as the and judging the process and conse- sure on output and a temporary in- U.S. General Accounting Office (GAO) quences of the defense drawdown. It is crease in unemployment. But if defense pointed out in their September 1993 re- a key source of recommendations for spending were to be replaced mainly by port, Military Downsizing, significant resolving problems, and for embracing investments in plants and equipment, additional reductions will result in a new possibilities associated with con- human capital, and research and devel- larger proportion of involuntary reduc- version. opment, the prediction was that eco- tions. A recent Bottoms-Up Review of The Commission’s 1993 report, Ad- nomic growth might actually speed up DOD needs and programs for 1995 justing to the Drawdown, explains its over the longer-term. Since general through 1999 supports this conclusion. assessment goals and methods, which economic growth has been such an im- We should keep in mind, however, that now provide general guidance to feder- portant tool in reducing the impact of the impact of this current drawdown is al efforts to make the conversion pro- drawdowns, the Commission felt the considered by experts to be smaller cess work. The Commission’s four Administration and the Congress than that of the drawdowns after the goals are: should give special attention to invest- Korean War and Vietnam, and less se- ment tax credits, banking reforms, defi- vere than other recent economic dislo- 1. To facilitate the transition by encour- cit reduction, education reform, the re- cations. aging economic growth over the moval of trade barriers, tax reform, and long run. increased spending on infrastructure. Strategies for Limiting Involuntary Separation 2. To preserve defense capabilities. Assisting the Dislocated Early release of personnel prior to 3. To ease the impact on workers, com- The complex network of assistance the normal end of their enlistments has munities and companies. efforts funded by the Congress in the been one strategy. Another has been fi- 4. To improve government policies and 1990s is very difficult to sort out. Em- nancial separation incentives to induce programs in terms of their efficiency ployment and training administrators, people to leave. The Defense Authori- managers and evaluators clearly need and effectiveness. zation Act of 1992 and 1993 supported to learn more about the many federal the use of financial incentives through The Commission recognized that eco- efforts to assist the defense conversion 1995 for those who had completed six nomic growth was a critical factor in transition. And these efforts need great- years of service at the time the law was facilitating conversion and alleviating er integration. enacted. New early retirement options its negative effects, and took the posi- In the meantime, one important have been used also. On the involun- tion that this growth should come from source of information is the Office of tary reduction side, DOD has tightened redirecting defense resources into in- Economic Conversion Information the quality standards for retention and vestment rather than consumption. (OECI). This office is the result of a has created special boards that can With respect to easing the impact of collaborative effort on the part of the mandate the retirement of selected per- conversion, the Commission felt broad- Economic Development Administration sonnel from among those already eligi- and the Economic and Statistics Ad- ble for retirement. er use needed to be made of integrated community-level planning, encouraged ministration within the U.S. Depart- The careful shaping of a smaller and supported by the federal govern- ment of Commerce, and the Office of military force has favored the retention ment. To improve existing government Economic Adjustment in DOD. The of the more skilled and experienced assistance programs, the Commission OECI manages a major information

38 Evaluation Forum n Issue 11 n Summer 1995 Features n Defense Conversion and Dislocation

clearinghouse on defense conversion tive duty military, and in the DOD ci- large-scale dual-use conversion relevant to workers, communities and vilian sector. This was 5.1% of total projects, believing that the differences industries. This automated system pro- national employment. The authors pro- between defense and civilian cultures vides information on current press re- pose that as many as 2.5 million de- and practices are too great. Producing leases and legislation, publications and fense-related jobs could disappear by components and subsystems rather than resource materials, worker adjustment 2001. They point out, however, that not final products, the report states, is more programs, and related issues. all of this shrinkage means actual job likely to launch dual-use production. Another source is the Logistics loss, since approximately three-fourths Government technical, marketing and of this downsizing will be through at- financial assistance is viewed as impor- Management Institute (LMI), a private trition. tant in helping firms move into more nonprofit research and development or- ganization established in 1961 and The larger problem, they suggest, is commercial activities. funded by DOD. Its new Center for De- the extent of employment losses in any A second OTA report in 1993, De- fense Conversion and Economic Re- one period, large declines in any given fense Conversion: Redirecting R&D, newal produces policy, economic, and period making readjustment more diffi- also offers useful insights and recom- fiscal management analyses and offers cult. Eight of the fifty states are above mendations about converting defense access to a number of databases on eco- average in their dependence on the de- industries and their research and devel- nomic adjustment challenges. The data- fense industry. And even with the opment efforts. The latter is a signifi- bases focus on two sets of issues: de- present level of economic recovery, it cant issue in that the defense complex fense activity, and federal, state and is hard to replace the well-paid jobs de- has supported a disproportionate share local adjustment programs. fense manufacturing has offered. There of U.S. research and development, is also the issue of minorities. Of all some of which has important civilian Congressional funds for assistance programs have flowed mainly to the employed black men between eighteen applications. It has established major and twenty-nine, 10.6% are in the mili- civilian industries, and has afforded DOD and from there to other relevant tary compared with 5.4% of white high-quality, well-paid jobs. Some of federal agencies, such as the Depart- ments of Labor and Commerce. One of males. The shrinkage of the defense es- this R&D provided a large market for tablishment may represent a major loss technologically advanced goods and the Logistics Management Institute’s of employment opportunities for minor- services. Relevant to these spinoffs, the major assignments in providing input to the Defense Conversion Commission ities. report admonishes the government not to waste the valuable human, institu- has been to prepare an exhaustive Com- The Larger Picture tional and technological resources for- pendium to Assist the Transition. This compendium provides a description of The OTA report also indicates that merly assigned to defense purposes. each program, its sponsor, the organi- communities at serious risk due to de- This second OTA report also sug- zation responsible for its implementa- fense cutbacks are not receiving as gests ways to re-use these defense-re- tion, and the amount of its funding. much assistance as they often need. lated resources, such as the develop- This information helps considerably in Federal economic development pro- ment of new environmental protection sorting out the spectrum of programs grams have lacked funding perennially. occupations, the creation of renewable for different kinds of recipients. Defense-dependent communities re- energy sources, the development of ceived an additional 50 million for high energy-efficient and non-polluting Transition assistance in the form of 1991 through 1993, but this was still retraining and reemployment for af- transportation systems, and the produc- much less than the amounts available in tion of a new generation of high-per- fected workers has been considered the past. And states have had difficulty formance computers and networks. As critical. However, a broad-range strate- sustaining innovative economic devel- gy appears to be needed that simulta- with most phenomena of the proportion opment projects. In this context, the re- represented by current defense conver- neously invests in sound, appropriate port contained some useful recommen- sion activities, multiple, coordinated worker training, assistance to business- dations: es to increase their performance, re- strategies are required. We no longer fear global nuclear war. With the end search and development assistance to n Deliver services more promptly. of the Cold War we have the freedom advance technology, and help with eco- n Target communities in greatest need. nomic development efforts. A recent to work toward national goals other than military security. Many experts in- report by the Office of Technology As- n Supplement state programs. sist we must use this opportunity to sessment (OTA) of the U.S. Congress, n After the Cold War: Living With Lower Offer a range of technical assistance move forward as an economic competi- tor in an international marketplace — Defense Spending, has recommended to help companies grow rather than and at the same time ameliorate the this kind of approach. attempting to motivate them through tax subsidies. hardships experienced by defense The authors of the OTA report re- workers, defense-dependent communi- mind us that in 1991 national defense Another complication the report ties, and defense companies during this employed approximately six million gives attention to is the reluctance of transition period. people in the private sector, in the ac- major defense contractors to initiate

Evaluation Forum n Issue 11 n Summer 1995 39 Features n Defense Conversion and Dislocation nn Matt Kane, propose that in the larger that were highly dependent on the de- Studying Defense economic environment military cut- fense budget at that time employed Conversion backs will have a beneficial effect on nearly 1.8 million workers. In industri- the national economy, releasing funds al sectors such as guided missiles, ship- in Affected for deficit reduction. However, the im- building, ammunition development, Communities pact on particular regions and indus- and tank construction, 75% of workers tries clearly varies. The variation is in- owed their jobs to defense spending. In this article we look at an interest- fluenced, they say, by the following This defense workforce differs in ing 1993 study of seven sites in the factors: important ways, the authors propose, U.S. that have been impacted signifi- n cantly by defense cutbacks. The study The size of the cuts. from the average labor pool. It consists largely of professionals, skilled and was sponsored and funded jointly by n The speed with which they are semi-skilled craft workers, clerical the Economic Development Adminis- made. tration (EDA) of the U.S. Department workers, and fairly skilled support n staff. The Office of Technology As- of Commerce and the U.S. Department The mix of cuts vis-a-vis procure- sessment of the U.S. Congress conclud- of Defense’s Office of Economic Ad- ment, operations and/or personnel. justment, and conducted by the North- ed in their 1992 study of the defense n The overall strength of the U.S. workforce that it was more highly east-Midwest Institute. The study re- economy at the time of the cuts. trained and skilled than workers in oth- port, Defense Adjustment and Conversion: Lessons from Seven Sites, The authors give us an important er sectors of the economy. Its workers were making high wages. These char- is the basis for the review. glimpse of defense expenditures, to acteristics clarify what a substantial im- provide a background for understand- The Nature of the Study ing the implications of defense down- pact sudden job loss can have in terms of the reemployment challenge. The sites selected for the process sizing. About 75% of defense purchas- study were San Diego, Minneapolis, In- es are made from manufacturing The change in the fortunes of the de- dianapolis, Albuquerque, Bath (Maine), industries; 21% from the service sector. fense workforce must be understood in and Sunnyvale (California). The re- The rest is from construction, agricul- the context of the trend in manufactur- searchers used focus groups in each site ture and mining. More than 15% of the ing jobs since the late 1970s. The to explore how defense cutbacks were total output of 31 industrial sectors in- occupations most at risk have been in affecting workers, businesses and com- volves defense-related activities. At manufacturing areas — engineers; tech- munities; what plans local leaders had least 29 of these industries manufacture nicians in engineering and science; developed for coping with the conse- durable goods. Some estimates indicate computer specialists; craft and similar- quences; and how these communities that as much as 57% of military-related ly-skilled production workers; and less could promote effective economic ad- employment is in the manufacturing skilled operators and laborers. Overall, justment and diversification. sector. manufacturing jobs have not risen since the late 1970s and have fallen by 5% Recognizing the effect of reduced The additional purpose of the focus since 1990. groups was to support coordination purchasing in this sector, the U.S. De- among key actors and constituencies in partment of Defense (DOD) has re- Areas with a high concentration of these communities and some level of formed its regulatory procedures to as- advanced technology firms and military agreement about what action should be sist companies in defense-dependent bases have been the hardest-hit. Eight taken. To achieve this latter purpose, industries — in order to reduce their states accounted for about one-fifth of focus group participants were provided costs, and to become more competitive. the defense spending in 1991. States with information obtained from case The DOD has also eliminated “recoup- with a large number of military bases studies of defense adjustment and con- ment fees,” or reimbursements much and facilities had become the most de- version, and from programs designed to like royalties, that private companies pendent. The Congressional Budget Of- resolve problems associated with mili- on federal contracts had to contribute to fice reported in 1992 that a defense re- tary cutbacks. They were encouraged to the federal government when they de- duction of 20% between 1991 and 1995 discuss how the roles and responsibili- veloped products or technologies that could result in a significant output loss ties of the different kinds of organiza- became viable in the civilian market. in the District of Columbia and ten states. Commercial-sector jobs associ- tions and groups participating in the fo- In terms of employment, although cus groups should be defined, in the the portion of the labor force in de- ated with defense production are affect- ed in these areas as well. process of developing strategies for fense-related businesses and industries solving problems in their own commu- is small, it is nevertheless significant. Federal Assistance for nities. In 1989, the authors indicate that about Adjustment Purposes 3.4 million private sector workers were The Larger Context employed directly or indirectly in de- The Defense Conversion, Reinvest- The authors of the study report, fense-generated jobs — about 5% of ment and Transition Assistance Act of Charles Bartsch, Paula Duggan and the total labor force. Eleven industries late 1992 authorized a set of new pro-

40 Evaluation Forum n Issue 11 n Summer 1995 Features n Defense Conversion and Dislocation

grams responsive to the difficulties be- n Small Business Innovative Research The Results of the Seven-Site ing experienced by certain businesses Program: this program increased Study and industries, and by whole communi- DOD’s R&D funds directed to small Two major themes ran through the ties, as a result of defense cuts. Much businesses. recommendations of private sector par- of the funds for these programs flowed ticipants in the focus groups: to beneficiaries in fiscal 1993. Some of Looking at the funneling of DOD the funds were channeled into existing dollars through multiple programs ben- n Remove regulatory and administra- economic development, worker train- efiting the American economy general- tive barriers that impede the adapta- ing and education programs, but a por- ly, one cannot avoid exploring issues of tion of dual-use technologies: tion went to new initiatives. Some of corporate charity and the need for pro- these were expected to enhance manu- gram consolidation. Congress provided w Accounting requirements. facturing competitiveness by emphasiz- $694 million to assistance programs w Proprietary rights limitations. ing “critical technologies” and “dual- targeted to the private sector in the use technologies:” 1992 legislation. n Increase government support for di- versification and adjustment pro- n Commercial-Military Integration Assistance to Workers grams designed to help defense-de- Partnerships: the DOD was to work While the issue of equity was raised pendent companies shift to civilian with high-tech firms to develop and by this large allocation to businesses, markets: commercialize high-payoff applica- $686 million was being directed to w Technical assistance. tions for dual-use technologies. worker adjustment programs. Most of n Regional Technology Alliances As- these programs assisted military per- w Advice on gaining access to the sistance Program: funding was to sonnel separating from the service and technologies necessary for the enable the development of alliances civilian workers employed by DOD. shift. The programs are providing separation among defense firms (and prime counseling, incentives for early retire- w Advice on marketing strategies. contractors) to develop and apply ment, extended benefits such as health w Information about new markets. critical defense technologies for dual care, increased training for civilian oc- use — defense and civilian — with cupations, and the establishment of hir- Public sector participants wanted gov- an emphasis on regional diversifica- ing preferences for these workers in ernments at all levels to adopt a more tion in defense-dependent areas. government jobs. Seventy-five million flexible approach to worker training, of the total funding is to be used for a the financing of economic develop- n National Defense Manufacturing new program for dislocated workers ment, and to other programs. Although Technology Program: the federal similar to DOL’s EDWAA — although the sites were quite diverse, the focus government was expected to engage DOD was given the option of simply groups yielded common proposals: small defense firms in using military adding this amount to JTPA Title III. n Business leaders should identify al- technologies as a base for diversifi- DOD is to notify defense contractors of ternative operating scenarios and be cation. federal cuts affecting them, and will re- prepared to restructure their busi- quire them to post jobs with the Em- n Technology Transfer Initiatives: nesses so as to be competitive in ci- ployment Service. The legislation also these were to encourage private sec- requested educational institutions to vilian markets. tor businesses to transfer military provide training in environmental occu- n Defense-related companies need a technology for civilian use. pations. range of information to increase n Defense Advanced Manufacturing their export potential: Assistance to Communities Technology Partnerships: DOD w Identification of overseas mar- was to share the costs borne by firms The budget earmarked for communi- kets. in developing and applying manu- ty assistance in the 1992 legislation facturing technologies for military was $132 million. Eighty million of w The defining of prospective mar- and civilian production. that was to be transferred from DOD to ket “niches.” EDA in the U.S. Department of Com- n Manufacturing Extension Program: merce. In addition, DOD’s Office of w Information about mechanisms this program was to improve the Economic Adjustment received in- linking U.S. manufacturing capa- competitiveness of manufacturing creased funding to permit the distribu- bilities and technologies with technology. tion of both planning grants and imple- prospective foreign purchasers. mentation grants for assisting n Dual-Use Extension Program: this communities, of the kind already dis- w Information about how to target involved a number of programs to tributed by the EDA. markets with growth potential. assist defense suppliers in making the transition.

Evaluation Forum n Issue 11 n Summer 1995 41 Features n Defense Conversion and Dislocation

n Worker adjustment program rules n There should be a study of 1) key The participants in the focus groups and regulations should be more flex- defense-related industries, 2) work- also felt that each community’s defense ible, to allow defense-related dislo- forces and 3) skill needs by the fed- adjustment effort required a mechanism cated workers to be better served. eral government, and a national in- for initiating, organizing and maintain- dustrial agenda should be developed ing action, such as a person, group or w Increased ability to launch pub- organization that commanded commu- — or at the least defense manufac- licly-funded retraining programs nity respect and support, and could turing policies should be coordinat- prior to the loss of a job — for serve as a focal point for policy devel- ed — to ensure that necessary tech- example, training for new tech- opment, planning and oversight activi- nologies and production capabilities nologies as part of a company ties. can be maintained. conversion or product diversifica- Implications of the Study for tion project. Some Final Study Conclusions Different Levels of Government w Better information on emerging The authors conclude that a success- The focus groups placed an unam- new skills and their mesh with ful adjustment strategy must “bring a biguous emphasis on 1) a national dual- existing work skills. high level of visible participation and use business and industrial policy, ac- n More specialists in commercial man- commitment” to this effort by local po- companied by increased federal litical and business leaders. Simple in- agement, productivity, marketing technical and financial assistance to terest in the problem will not suffice. state governments, defense-oriented and purchasing need to be produced. Communities need to launch formal employers, and local communities af- n A successful business retention needs assessments and planning pro- fected by downsizing, and 2) greater ef- strategy should be developed, target- cesses which speed adjustment efforts. forts on the part of community political ed to defense-oriented companies. Those sites that had established formal leaders, government agencies and the planning structures and processes were private sector to identify the array of n Acquisition of capital should be making the greatest progress. They had needs due to defense cutbacks, and to more accessible and affordable, to used a series of meetings and surveys take well-organized action to reduce carry out defense adjustment poli- to identify the potential magnitude of the negative fall-out. cies and strategies. the problem, the nature and scope of The thrust of the study has important existing workforce skills and manufac- n DOD should adopt a more quality/ turing capability, and other information implications for current federal efforts to consolidate and otherwise coordinate results-oriented approach to military needed to construct a workable adjust- workforce development policies and procurement that places less stress ment strategy. This information base on detailed specifications. was considered essential to the achieve- programs, and integrate their informa- tion systems — and for states’ new ments realized. n There should be greater support for efforts to establish comprehensive the development of “critical technol- The following kinds of information state-wide councils responsible for ac- ogies” by small businesses, to in- were recommended for this information complishing the same goals. The major crease the number of high-quality bank: message would seem to be that defense employers and reliable specialized n Numbers of workers employed and adjustment requires a multi-dimension- al strategy involving assistance to dis- vendors for defense-related parts unemployed. located workers placed within the larg- and components. n Numbers of actual and likely plant er context of assistance to businesses n DOD’s proprietary rights regarding closings, and what industrial sectors and communities. the dissemination of information they represented. The recommendations flowing from should be limited, to increase the in- n Numbers of workers on layoff from this study would suggest strongly that tegration of military and commercial defense-oriented companies, and the Departments of Defense, Com- production. their skills. merce, Labor, and Health and Human Services should increase their coordi- n DOD policies and regulations that n Kinds of emerging industries in the nation and collaboration on defense ad- fit the Cold War but now interfere area, and the types of workers they justment issues, and that state work- with efforts to convert or scale back were likely to hire. force development systems should production should be eliminated, incorporate economic development within the goal of maintaining criti- n Information on the education, train- agencies within their formal structure cal technologies. ing and social service resources and functions. available. n DOD stipulations regarding cost and pricing data, certifications and au- n Job creation information. dits should be overhauled.

42 Evaluation Forum n Issue 11 n Summer 1995 Features n Defense Conversion and Dislocation nn Programs to Ease A number of states have their own former military personnel. programs covering services to dislocat- In 1994, rather than transferring de- ed military and workers in defense- the Adjustment to fense adjustment funds from the DOD related businesses and industries. Im- Defense Cutbacks to DOL, DOL requested a general portant examples are California’s Em- funding increase for dislocated worker ployment and Training Panel, New In the early 1990s, the Defense Con- programs. Congress provided over one York’s Skills Training Program, and version Commission in the U.S. De- billion in FY 1994 for dislocated work- partment of Defense funded the Logis- Washington State’s Job Skills Program er assistance in DOL, of which $125 and vocational education effort. tics Management Institute (LMI) to million was to be earmarked for de- analyze and classify information about Clearinghouse Activity fense workers. The Administration’s the myriad programs providing some 1995 request includes approximately kind of assistance to individuals, busi- The DOD and Department of Com- $1.5 billion for Title III, with an esti- nesses and communities related to de- merce jointly maintain a useful clear- mated $195 million going to workers fense downsizing. LMI subsequently inghouse, the OECI Clearinghouse, dislocated from defense-related em- collected data on these programs and which tracks the assistance provided to ployment. interviewed a large number of federal, individuals, businesses and communi- state and local officials about them. ties in the plethora of programs created Evaluation Mandates Their effort produced the Compendium due to defense restructuring and down- Conspicuously absent from descrip- of Programs to Assist the Transition, sizing. For example, summaries are tions of DOD programs for dislocated published in Feburary 1993. available for the Defense Industry Ad- workers within, separated from, or out- The Compendium includes pro- justment Projects which break down in- side but related to the military is any formation by type of award granted to grams in existence prior to the National mention of program evaluation. While states — state planning grants, ad- Defense Authorization Act for Fiscal Department of Education and DOL 1993, and those initiated through this vanced planning grants, and defense in- programs have substantial evaluation dustry adjustment grants — and de- legislation and some of the legislation requirements, the DOD programs do scribe each project. that followed. Sixty-one of the 144 pro- not appear to have such requirements. grams were sponsored by state and lo- Recent Initiatives Although the Defense Conversion cal communities, even though the latter Commission has taken a strong position were sometimes utilizing federal funds. In March 1995, President Clinton on the need for performance measures, announced a new five-year package of no performance management system The Nature of Assistance programs under the Defense Transition comparable to that required for the Programs Program. These were mainly reinvest- DOL programs seems to be in place. If ment and transition programs devel- The major assistance programs in true, this would be a serious deterrent the Compendium are federally-funded oped by the Congress but not imple- to sound policy development for this mented by the Bush Administration. In and emphasize training and economic portion of the dislocated worker popu- the President’s FY 1995 budget re- development. They tend to serve as the lation. ultimate safety net for individuals af- quest, funding for four categories of programs was requested: workforce, The Department of Education fected by defense cuts and drawdowns. community, dual-use technology, and Report Programs targeted to industries stress development and technology transfer. non-defense technology. Under work- In May 1991, the Office of Educa- force initiatives, the main interest was Almost all state programs involve job tional Research and Improvement in separation pay and retirement incen- training oriented to job retention or re- (OERI) organized a conference to ex- employment. Industry programs also tives, and in education and training. amine the role of education in restruc- Out of the seven education and training concentrate on worker training, as well turing defense industries and ways to programs recently established by Con- as industrial development. Service deliv- ameliorate the negative impact of such ery for federally-funded programs are the gress, the President chose to emphasize restructuring on workers. Much was al- two that trained active duty military for responsibility of states, while local pro- ready known. In 1990, over 200,000 public service-oriented jobs, the grams are directed mainly to local em- active and reserve service personnel ployment and economic development. “troops to teachers” and “troops to and veterans used educational benefits cops” programs. to help pay for their education. A study The Compendium is well organized and detailed, reviewing separately pro- Not included in the funding request, completed in 1976 had suggested that however, were three environmental ed- the earnings of veterans using Mont- grams for individuals vs. programs for ucation programs authorized in the FY gomery GI Bill benefits were 10% states and communities, according to their funding sources. Figure I lists the 1994 Defense Authorization Act, and higher than they would have been oth- funding for the Occupational Training erwise. A subsequent study indicated programs for individuals, summarizing Program that was to partially reimburse that educational benefits used by Viet- selected federal programs for workers displaced from the military and/or de- firms for the cost of employer-provided nam-Era veterans and by servicemen in training if these firms hired and trained fense-related employment.

Evaluation Forum n Issue 11 n Summer 1995 43 Features n Defense Conversion and Dislocation

Figure 1 n Programs for Workers

Programs in Existence in 1993 Program and Funder Target Group Type of Assistance Involuntary Separation Pay Program Active members of military with 6-20 Lump sum separation payment. DOD years of service involuntarily separated from service. Montgomery GI Bill Active members of military and National- Educational stipends. DVA Guard. Post-Vietnam Era Assistance Same as above. Same as above. DVA Vocational Rehabilitation and Disabled veterans. Counseling, education, training, and Counseling support services. DVA Voluntary and Special Separation Selected active members of military — Series of annual payments or lump Incentives officers, enlisted members — who agree sum in return for voluntary separation DOD to voluntarily leave military. and continuation in reserves. Defense Automated Transition All members of military separating from Job placement assistance. (The sys- Systems armed services. tem is in cooperation with DOD’s Tran- DOD sition Assistance Offices, which are of- ten co-located with family and commu- nity support centers.) Armed Services Career Alumni All armed services personnel in process Career guidance, benefits counseling, Program of leaving services, and their families. job search assistance. (Members of DOD armed services leaving military receive certification of their job experience and skills while in military.) Interagency Placement Assistance Members of military leaving service. Job placement assistance, and the Program communication of the availability of DOD these individuals for employment to the federal agencies. Reemployment Priority List Members of military leaving service due Listing of individuals on priority list for DOD to dislocation. federal jobs. Transition Assistance Program Separating service members. Three-day job search assistance semi- DVA/DOD nars through state Employment Ser- vice. Priority Placement Program Dislocated federal civilian employees. Job placement assistance. DOD Displaced Employee Program All federal employees involuntarily sepa- Job placement assistance. All agencies. rated. JTPA Title III Dislocated workers. Rapid response assistance, training, EDWAA basic readjustment assistance, needs related payments. (The 1993 National Defense Authorization Act amended Title III to establish discretionary pro- gram designed to respond to reem- ployment needs of 1) those at risk of dislocation and 2) dislocated military, DOD civilian personnel, and defense workers.) Employment Service All workers. Job search and placement assistance. DOL Unemployment Insurance Eligible unemployed workers. Fiscal benefits. DOL Figure 1 continues

44 Evaluation Forum n Issue 11 n Summer 1995 Features n Defense Conversion and Dislocation

Figure 1 n Continued

Programs in Existence in 1993 Program and Funder Target Group Type of Assistance Job Opportunities and Basic Skills Adult recipients of AFDC. Education, training, employment assis- Program (JOBS) tance, and support services. HHS Federal Work Study Higher education students needing Financial aid. DOE financial assistance. Federal Family Education Loans Same as above. Stafford Student Loans, PLUS loans, DOE and Supplementary Student Loans. Federal Pell Grants Same as above. Educational grants, up to five years. DOE Federal Supplemental Educational Same as above. Undergraduate/graduate nonrepayable Opportunity Grants grants. DOE Federal Perkins Loans Same as above. Loans with delayed repayment. DOE 1990 Carl D. Perkins Vocational and Same as above. Vocational education grants. Applied Technology Grants DOE Service Members Occupational Dislocated members of armed services. Six to eighteen months of occupational Conversion and Training Act of retraining. 1992 DOD

Programs Created by the 1993 Defense Authorization Act and Subsequent Legislation Program and Funder Target Group Type of Assistance Continued Health Coverage for Former service members and their Temporary health benefits. Military and Dependents families. DOD Skill Training Programs for Dislocated DOD civilian employees. Occupational training. Civilian Personnel DOD Separation Pay Dislocated DOD civilian personnel. Lump sum payments. DOD Teacher and Teacher’s Aid Separated members of armed forces, Education to acquire teacher or teach- Placement Programs terminated DOD civilian employees, er aide certification, job placement as- DOD displaced scientists/engineers working sistance, stipend during first two years for defense contractors. of teaching. Environmental Scholarship and DOD employees. Fellowship for training in resource Fellowship Programs management/restoration. (DOD also DOD funds grants to higher education insti- tutions to provide training in environ- mental restoration and hazardous waste management.) Defense Diversification Program Displaced military and employees of Transition adjustment and employment DOD/DOL defense contractors. services. (Expansion of Interstate Job Bank Programs and coordination with Defense Outplacement Referral Sys- tem.)

Acronyms: DOD = Department of Defense; DOL = Department of Labor; DOE = Department of Education; DVA = Department of Veterans’ Affairs; HHS = Department of Health and Human Services.

Evaluation Forum n Issue 11 n Summer 1995 45 Features n Defense Conversion and Dislocation

nn the all-volunteer army raised overall Findings from the n At the level of the individual work- earnings for veterans by an average of er, defense conversion is the process about 6%, if used for college or gradu- Defense Conversion by which defense industry workers ate school. These findings were consis- Adjustment and military personnel prepare tent with other studies in the 1980s and themselves for jobs in the non-mili- 1990s showing significant earnings Demonstration tary commercial sector. This con- gains for college graduates and those version process may involve learn- with graduate degrees. Project ing new technical or occupational Another set of analyses offered simi- skills or how to adjust to corporate lar insights. The conference benefited Mary G. Visher culture in the commercial sector. from ten commissioned policy papers Berkeley Planning Associates relying on relevant research. “Lessons In May 1992 the U.S. Department of from the Past: Mitigating the Effects of Labor (DOL) became directly involved Military Cutbacks on Defense Work- Deborah Kogan with defense conversion when it an- ers,” by Lois Lembo and Judith Phil- Social Policy Research Associates nounced the availability of funds to lips, makes a substantial case for edu- support innovative strategies to assist cation as a vehicle for reducing the workers, businesses and communities 1 dislocation effects of defense cutbacks, Background on Defense affected by downsizing. Twelve and for ensuring that defense conver- Conversion Round 1 DCA grants were awarded in sion contributes to workers and the November 1992, and another seven economy. Burt Barnow and Amy As a result of the end of the Cold Round 2 grants a year later. Grants Chasanov’s paper on “Firm-Based Ed- War, outlays by the U.S. Department of were awarded under several different ucation and Training of Workers” il- Defense are projected to drop by 30% categories: community planning, dislo- lustates in a case study of the Xerox between 1987 and 1997. These reduc- cation aversion, and increased worker Corporation how companies can use tions in defense expenditures have af- mobility, reflecting the three different education and training to maintain their fected defense industry workers and levels of impact from the defense draw- economic health in the midst of unex- military personnel, private firms that down.2 pected changes in federal budget priori- are dependent on defense-related sales, ties. and communities with military installa- Overview of the DCA tions and/or concentrations of defense Demonstration Evaluation If we are to determine the actual im- firms. The term “defense conversion pact of the many DOD transition assis- adjustment” refers to the process by The Department of Labor contracted with Berkeley Planning Associates tance programs, the kind of research which communities, firms, and individ- (BPA) to conduct an evaluation of the expertise illustrated in OERI’s report ual workers adjust to the defense draw- on the conference, Military Cutbacks down. DCA Demonstration. The objectives of and the Expanding Role of Education, this evaluation, performed by BPA and needs to be applied to these programs. n At the community level, defense its subcontractor, Social Policy Re- The Department of Labor and Depart- conversion requires decision-makers search Associates (SPR), are threefold: ment of Education research efforts to plan for and address the immedi- (1) to describe and document the im- would seem to offer useful models. ate impacts of reductions in defense plementation and short-term outcomes spending on affected firms and laid- of the demonstration projects, as they off workers, while building a foun- relate to the specific problems faced in defense-related dislocations; (2) to dation for economic renewal by co- identify exemplary approaches to the ordinating all available community specific problems faced in defense-re- resources. lated dislocations; and (3) to identify n At the firm level, defense conversion the factors that contribute to or impede is the process by which defense-de- the success of various defense conver- pendent firms develop new process- sion approaches. es and products that will enable them to compete in commercial mar- kets. Necessary changes may in- 1Section 325(d) of Title III of the Job Train- clude reorganization of the work- ing Partnership Act provides funding for demon- place and training workers and stration projects as part of the Defense Conver- sion Adjustment Program. managers in new technologies and 2 high performance workplace skills Due to limited space, in this summary we only report findings from our observations of the (HPWO). projects using the two latter approaches.

46 Evaluation Forum n Issue 11 n Summer 1995 Features n Defense Conversion and Dislocation

The methodology for this evaluation contributed to the achievement of these performance workplaces. These skills emphasizes the collection of qualitative goals. include teamwork, communication, data using multiple site visits to each of problem-solving, continuous improve- This study has not attempted to col- the 19 demonstration projects, supple- ment skills, process analysis and statis- lect outcome data from comparison mented by limited quantitative data on groups or sites to suggest what would tical process control. While these skills the characteristics of participating were perceived in all projects as being have happened to participating workers workers and firms, services provided, key to successful firm performance in or firms in the absence of services. In and outcomes achieved by participating assessing project effectiveness, we have commercial markets, projects were workers and firms. This article, based most successful: compared each project’s performance on mid-project findings reported in the 3 to its own stated goals, as well as to the n If workforce training was closely evaluation’s Phase I Final Report, outcomes achieved by other demonstra- summarizes findings on how the linked to furthering specific conver- tion projects testing similar approaches. projects have fared thus far, and lessons sion goals at the firm level. that can be learned from their experi- Averting Layoffs at Defense- n If worker representatives had a voice ences. Dependent Firms in the design and oversight of train- ing. Evaluation Issues Projects n If there were structured opportuni- The evaluation is based on detailed Nine projects are testing dislocation case studies of each project document- ties for workers to practice their new aversion strategies. Figure 1 summa- skills in the workplace during and ing project goals and objectives, the in- rizes the dislocation aversion projects after training. volvement of different project partners and their key features. Projects vary in in developing and implementing the the number of firms they are assisting, n If the firm had identified measurable demonstration design, and progress to the size of participating firms, and the indicators of success at the firm or date in achieving project goals. These stage of participating firms in the con- work-unit level case studies have provided valuable in- version process. Some projects target- formation on projects’ successes in im- ed firms early in the adjustment process n If firms were given the autonomy plementing their intended designs as and supported managers during the they needed to choose a service plan well as barriers to successful imple- process of planning for conversion. that was appropriate, as long as the mentation. Other projects targeted firms that had quality of the providers was moni- Particularly challenging for this already developed strategic plans for tored by the project administrator. evaluation has been the assessment of conversion and were ready for work- It is too early to draw conclusions project effectiveness using data on par- force training. ticipant-level and firm-level outcomes. about whether firms were able to avert layoffs as a result of the assistance they Since most of the projects are still in Findings received. However, several of the par- operation, final participant outcomes A sequence of steps is necessary for are not yet available. For projects as- ticipating firms were able to document successful conversion, including identi- improvements in productivity and cost- sisting defense-dependent firms with fying firm strengths and weaknesses, effectiveness measures and/or avoid an- defense conversion, the intended out- developing a strategic plan for entering comes (e.g., increased non-defense ticipated layoffs during the first phase non-defense markets, and implement- of the demonstration. sales and workforce stability or growth) ing the changes necessary to compete may not be observable within the eval- successfully in these markets. Projects Increasing Worker Mobility uation time-frame, particularly for that targeted firms early in the conver- firms at the beginning of the conver- sion process spent substantial amounts Projects sion process. Even where firm-level of staff time and project resources as- outcomes are available, it is not clear sisting management in conducting an Nine projects are attempting to pro- mote increased worker mobility for in- whether and how demonstration-related objective assessment of their strengths dividuals dislocated as a result of lay- activities, such as workforce training and weaknesses and developing a stra- and strategic planning assistance, have tegic plan for conversion prior to work- offs among defense industry workers, military service personnel, and/or civil- force retraining. For these projects, ian Department of Defense employees 3This report was submitted in draft to DOL workforce training sometimes grew out of the assessment process, when work- at military facilities. Figure 2 summa- in March 1995. The report consists of prelimi- rizes the key features of these projects. nary findings on the three defense conversion ers were asked to participate in assess- approaches (worker mobility, defense conver- ing the strengths and weaknesses of The worker mobility projects fall sion, and community planning) in Volume I. their individual work units. into three rather distinct categories: (1) Volumes II through IV contain detailed project those that seek to reemploy participants profiles for each demonstration project, orga- Projects that focused on improving nized by approach. An executive summary will worker skills often emphasized the im- quickly without intensive retraining; be available as a separately bound volume. (2) those that seek to prepare partici- portance of the skills needed for high

Evaluation Forum n Issue 11 n Summer 1995 47 Features n Defense Conversion and Dislocation

Figure 1 n An Overview of the Dislocation Aversion Projects

Grantee/Project Area Strategy/Goal Target Group Key Features/Activities

International Association n Assist defense-dependent n 3 small to medium-sized n Provide consulting servic- of Machinists and Aero- firms to become more firms in Los Angeles area es to assist in conversion space Workers (IAM) Con- competitive planning version Demonstration n Assist in the reemploy- n Provide HPWO and occu- IAM Lodge 727 ment of dislocated de- pational skills training Los Angeles, California fense workers n Develop and provide training to build electric car components n Develop job opportunities for participants

Long Island Defense n Assist defense-dependent n 9 small to medium-sized n Assess firms’ strengths Diversification Project firms to become more firms in Long Island area and weaknesses New York State Department competitive by promoting n Provide HPWO skills of Economic Development HPWO principles training New York, New York

Management Assistance n Facilitate development n 20 small to medium-sized n Support assessment of and Technology Transfer and implementation of firms in St. Louis area workforce skills and pro- Program commercial conversion duction systems St. Louis County Economic plans for defense-depen- n Support strategic planning Council dent manufacturing firms n Support occupational and St. Louis, Missouri HPWO skills for workers

Massachusetts Strategic n Assist defense-dependent n 12 small to medium-sized n Occupational skills Skills Program firms to become more firms in Massachusetts training Massachusetts Industrial competitive area n Other skills training Services Program (HPWO, team and com- Boston, Massachusetts munication skills, TQM, process analysis)

San Diego County Defense n Assess business needs n Defense-dependent busi- n Survey of local business- Conversion Adjustment n Disseminate information to nesses exploring conver- es to assess and address Demonstration affected businesses sion in San Diego affected business needs San Diego Consortium and n n Assist defense workers Conduct Business Private Industry Council find new jobs Roundtables to assist de- San Diego, California fense dependent firms consider conversion op- tions n Deliver services to dislo- cated workers, including high tech entrepreneurial training and TQM skills training

Figure 1 continues

48 Evaluation Forum n Issue 11 n Summer 1995 Features n Defense Conversion and Dislocation

Figure 1 n Continued

Grantee/Project Area Strategy/Goal Target Group Key Features/Activities

Sargent Controls Project n Assist in the conversion of n Sargent Controls (employ- n Provided support to as- Pima County Community Sargent Controls, produc- ing about 400) sess strengths and weak- Services Department er of nuclear submarine nesses Pima County, Arizona valves n Provided occupational, technical and other skills training to workers

Minnesota Defense n Assist in increasing the n Alliant Techsystems (em- n Occupational skills train- Conversion Adjustment competitiveness of Alliant ploying about 4,000) ing for machinists Demonstration Techsystems, a producer n Workplace basic skills Department of Jobs and of explosive devices training Training n Basic skills enhancement St. Paul, Minnesota

The Hummer Project: A n Assist in increasing the n AM General (employing n Consulting services to Dislocation Aversion conversion, stabilization of 1,500 workers) and sever- support HPWO principles Project AM General, producer of al supplier firms (AM General only) Workforce Development an all-terrain military vehi- n Workforce training on Services cle. communications, team- South Bend, Indiana building and process analysis (AM General and nine suppliers)

Workforce Protection n Assist in increasing the n 20 small to medium-sized n Occupational and techni- Program competitiveness of small defense firms in Rhode Is- cal skills training Rhode Island Port Authority defense-dependent firms land and Economic Development Corporation Providence, Rhode Island

Evaluation Forum n Issue 11 n Summer 1995 49 Features n Defense Conversion and Dislocation

Figure 2 n An Overview of the Worker Mobility Projects

Grantee/Project Area Strategy/Goal Target Group Key Features/Activities

Philadelphia Naval Base n Plan and coordinate a re- n Several thousand at-risk n Identify and review strate- and Shipyard Complex sponse to the drawdown and dislocated base work- gies to assist affected Planning Project of the Philadelphia Naval ers workers Pennsylvania Department of Base, scheduled to be n Create a database con- Labor and Industry complete in early 1996 taining skills and charac- Philadelphia, Pennsylvania teristics of workers and needs of businesses n Develop plans for an on- base transition center

Charleston Naval Complex n Plan and coordinate a re- n A small number of dislo- n Produce an economic Community Planning sponse to the dislocations cated base workers to profile of the Charleston Project caused by the drawdown participate in pilot training metropolitan area Charleston County of the Charleston Naval programs n Develop a coalition be- Employment and Complex, scheduled to be tween economic develop- Training Administration complete in 1996. ment entities, educational Charleston, South Carolina institutions, and the job training and placement community n Develop a plan for a “one- stop shop”

International Association n Assist defense-dependent n Dislocated defense indus- n Provide consulting servic- of Machinists and Aero- firms to become more try production workers with es to assist in conversion space Workers (IAM) Con- competitive experience with composite planning version Demonstration n Assist in the reemploy- materials n Provide HPWO and occu- IAM Lodge 727 ment of dislocated de- pational skills training to Los Angeles, California fense workers workers in participating firms n Develop and provide training to build electric car components n Develop job opportunities for participants

San Diego County Defense n Assess business needs n Dislocated defense n Survey of local business- Conversion Adjustment n Disseminate information to workers es to assess and address Demonstration affected businesses affected business needs San Diego Consortium and n n Assist defense workers Conduct Business Private Industry Council find new jobs Roundtables to assist de- San Diego, California fense dependent firms consider conversion op- tions n Deliver services to dislo- cated workers, including high tech entrepreneurial training and TQM skills training

Figure 2 continues

50 Evaluation Forum n Issue 11 n Summer 1995 Features n Defense Conversion and Dislocation

Figure 2 n Continued

Grantee/Project Area Strategy/Goal Target Group Key Features/Activities

Center for Commercial n Assist defense workers to n Dislocated defense work- n Classroom training in self- Competitiveness start up new businesses ers with professional or directed teamwork and State University of New York technical skills commercial competitive- Binghampton, New York ness n Support for project teams to develop new business- es or contract with local firms

Military Certification n Assist military personnel to n Separating military officers n Assistance in completing Project: A Worker Mobility become teachers in South prerequisites for Clemson Project Carolina University’s teacher certi- University of Clemson fication program Clemson, South Carolina n Assist in placing partici- pants in paid internships

Project Earn: A Worker n Assist laid-off employees n Production workers and n Basic readjustment ser- Mobility Project from McDonnell Douglas managers laid off from Mc- vices, including assess- McDonnell Douglas to find replacement jobs Donnall Douglas’s Titus- ment, career counseling Aerospace East ville facility after cancella- and job search assistance Titusville, Florida tion of Advanced Cruise n Short-term training to up- Missile Program grade skills

Operation StepOut: A n Assist participants to tran- n Women dislocated or at n Assessment, career coun- Worker Mobility Project sition to non-defense em- risk of dislocation from de- seling, job search assis- Arizona Governor’s Office ployers fense sector tance services for Women n Seminar on gender is- Tempe, Arizona sues n Access to career network- ing group

Alternative Fuels Training n Train and place dislocated n Workers dislocated from n Support development of Project: A Worker Mobility defense workers in emerg- Dallas-Fort Worth area de- curriculum Project ing high technology occu- fense contractors n Provide skills training Texas Railroad Commission pations n Assist in certification pro- Dallas-Fort Worth, Texas cess n Assist in placement of participants

Evaluation Forum n Issue 11 n Summer 1995 51 Features n Defense Conversion and Dislocation

pants for new careers in “niche” occu- projects have found it very difficult and Mary G. Visher is Principal Ana- pations (e.g., as teachers, alternative time consuming to recruit appropriate lyst at Berkeley Planning Associates fuel technicians for public vehicle participants to match to the available (BPA) and project director for the fleets, or production workers using services. In addition, projects trying to Evaluation of the Defense Conversion composite materials); and (3) those that place participants in emerging indus- Adjustment Demonstration. Since re- seek to support would-be entrepreneurs tries have found that the new jobs have ceiving her doctorate in labor market in starting new businesses or joint ven- not always been available when they sociology in 1984 from the University tures in high-technology fields using were needed. It is too early to say of Wisconsin-Madison, she has been skills and processes developed in the whether these projects are helping par- active in policy analysis and program defense industry. ticipants find high quality jobs at the evaluation. Her area of specialization conclusion of training. over the past ten years has been in the Findings field of employment and training. She The projects supporting new busi- has directed or been involved with nu- The projects providing basic read- ness start-ups or joint ventures with ex- justment services and short-term skills isting firms are testing how high tech- merous U.S. Department of Labor stud- ies. enhancement training are rather disap- nology start-up firms can be used to pointing. Few of these projects identi- stimulate economic growth by creating Deborah Kogan is Senior Social fied new or innovative models of basic new jobs in the local economy. Both Scientist at Social Policy Research As- readjustment services tailored to the Round 1 projects in this category de- sociates (SPR) and co-principal investi- needs of dislocated defense workers. veloped highly innovative and ambi- gator of the Evaluation of the Defense Because project planners in several tious designs using linkages with exist- Conversion Adjustment Demonstration. projects had misjudged local labor ing businesses in innovative ways. One She has played a key role in a number market conditions, the limited services project appears to be particularly effec- of national evaluations of federal em- offered by these projects were often in- tive in achieving positive outcomes for ployment and training programs spon- sufficient to meet the reemployment participants, perhaps because it selected sored by the U.S. Department of Labor, needs of project participants. Projects participants who already had strong including a recent evaluation of the struggled to adapt or refine services af- business concepts and emphasized the EDWAA program. She is also direct- ter project start-up to correct these “nuts and bolts” skills needed to ing a study for the U.S. Department of shortcomings. In addition, projects in achieve a successful business startup. Agriculture of a five-state demonstra- this category that were not closely co- tion project to test the feasibility of ap- ordinated with local JTPA systems ran Conclusion plying Food Stamp Employment and the risk of duplicating the services Like any major experiment, the Training Program regulations to the available from EDWAA or competing DCA Demonstration has—even at this JOBS program for AFDC recipients. with the EDWAA system for clients or early stage—shown some successes and continuation funds. some failures. When the Department The outcomes achieved by the of Labor requested proposals for these projects emphasizing basic readjust- grants three years ago, it deliberately ment services were mixed. Although invited applicants to “break the mold.” most projects appear to be meeting the Project designers responded by taking placement rates specified in their pro- risks in setting goals, forming partner- posals, the placements sometimes took ships, selecting target groups, design- longer than expected, were not in a ing interventions, administering servic- training-related field, or were not at the es, and monitoring progress. desired wage level. Risk-taking leads to mistakes, and The projects preparing dislocated this demonstration includes its share of defense workers for new careers identi- mistakes. But risk-taking also leads to fied careers that they thought would be new knowledge, new models, new les- attractive to and appropriate for groups sons. The DCA Demonstration is suc- of dislocated defense workers. Using ceeding in providing useful lessons on project partners with close organiza- how to support economic conversion, tional linkages to the targeted occupa- whether or not the individual projects tions or industries, they succeeded in succeed in accomplishing each of their developing high quality training pro- objectives. grams in those fields. However, these

52 Evaluation Forum n Issue 11 n Summer 1995 Features n Editorial Commentary

Editorial Commentary puters companies have been buying in Our share of world imports climbed; our record numbers, but not all of these share of world exports declined. At the Most economists agree with the workers were permanently retained. same time, over the past two decades, Clinton Administration’s assessment Expansion has been less profitable in the living standards of more than 80% that economic growth has resumed in the context of international competition. of American families had dropped. the 1990s but at a slower pace than in On the side of a generally positive An important lesson about the new previous upswings. The International economic vision for the U.S. is the U.S. economic environment has been that Monetary Fund supports this conclu- Census Bureau’s 1994 report indicating industries focusing on intense knowl- sion in terms of global economic condi- that Americans were living longer, edge bases, and industries achieving tions, predicting a gradual expansion smoking less, spending more money on significant “economies of scale,” have from 3.8% in 1994 to 4.2% in 1996. The educational materials, owning fewer had a longer-term impact than others Mexican peso crisis and the falling U.S. guns, divorcing less, and staying mar- on technology performance in other dollar have led to cautions, however. ried longer. Infant mortality had economic sectors. Governments that But new trade agreements such as dropped. Per capita income measured have targeted such high-technology GATT have been considered positives in constant 1987 dollars had increased. sectors for special support and devel- in this picture of the future. Disposable income had risen. The level opment assistance have encouraged In fact moderate growth is viewed by of educational attainment had in- economic progress. Governments that some economists as having real virtue. creased. But there was a dark side as have over-shielded major corporations Inflation is remaining low and forecast- well. Average hourly earnings mea- from competition have created econom- ers propose that a 3% growth rate will sured in constant 1982 dollars had de- ic disincentives. In fast-growing econo- lift Americans’ after-tax income in real clined between 1980 to 1993. Median mies, government and industry have terms and reduce unemployment to an scores on Scholastic Assessment Tests worked as collaborative partners in de- acceptable 6%. However, they ac- had plummeted. The increase in Ameri- veloping critical industries facing for- knowledge that military downsizing and cans’ expenditures on video and audio eign domination. In nations with the continued recessions abroad could equipment and personal computers had best records of industrial development, change these predictions. And it is sig- dwarfed their expenditures on educa- government, labor and management nificant that although corporate Ameri- tion. pursue manufacturing excellence to- ca spent an enormous amount on new Meanwhile, international competition gether, using a variety of policies and equipment in the 1990s in order to be- had grown dramatically, particularly be- actions. They ensure that a high quality come more efficient and profitable, little tween the U.S., Europe and the Pacific workforce is available through a com- job growth resulted. Only 2.3% was Rim. Studies of economic performance petent, accountable education and spent on expanding businesses. A in the early 1990s showed clearly that training system. And they give serious large portion of the increase in employ- American manufacturers in general and attention and energy to the readjust- ment in the 1990s has been related to in a wide range of industries, including ment and reemployment of workers ad- the manufacturing, installing or pro- many high technology industries, were versely affected by economic change. gramming of the machinery and com- losing ground to foreign competitors.

Evaluation Forum n Issue 11 n Summer 1995 53 Commentary

Robert B. Reich Basic Causes (1983 to 1993) the number of jobs in Secretary of Labor The fundamental fault line running sectors that typically require higher lev- els of education has grown by an annu- through today’s workforce is based on al average of 2.8% while job growth in education and skills. Well-educated Even though it has been a long time and skilled workers are prospering. sectors requiring less education has been only 1%. This shift in favor of since the picture has been as bright as it Those whose skills are out of date or skills shows up not just in the U.S. but is now for the average American work- out of sync with industrial change anx- er, the experience of the “average” iously contemplate their prospects. in other advanced countries. worker is becoming less and less rele- Those without education or skills drift In a seeming paradox of today’s vant. Some workers are surging ahead, further and further away from the eco- economic news, financial markets fret others are treading water, and still oth- nomic mainstream. The notion that we that unemployment is too low to con- ers are sinking fast — in the same are creating a bounty of bad jobs is a tain inflation, even while eight million economy, at the same time. The state of myth. Most new jobs are good jobs. willing American workers remain job- the American workforce, in short, is di- The problem is that the jobs that remain less. Part of the answer to this apparent vided. for workers without skills or with the contradiction is that markets for highly In the late 1960s, as my generation wrong skills are becoming grimmer. skilled labor are becoming tight in many parts of the economy, creating was joining the workforce, the vast ma- Skills have always mattered, but the conditions that can kindle inflation jority of Americans were middle class, they have not always been such potent in reality as well as in perception. Their determinants of economic destiny. As worries. But millions of less-skilled workers remain idle or underemployed. status and prospects differed only mod- recently as 1979, a male college gradu- This wasted workforce is walled off by erately from the average. In an aston- ate earned 49% more than a similar ishingly short time, the old middle class man with only a high school diplo- skill barriers from the leading edges of the economy. The best way to expand has splintered. The erosion of a sense ma — a sizeable difference, to be sure, the economy’s capacity and lower the of shared prospects poses what may be but not too large for the two to share our nation’s most critical challenge of the label “middle class.” By 1992, level of unemployment needed to bridle inflation is to dismantle these walls by the post-Cold War era. Reversing this however, the average male college preparing underutilized workers for erosion, giving all Americans a reason graduate was earning 83% more than to believe once more that hard work his high school graduate counterpart, more productive work. will lead to a better life, must become a and the notion of common prospects What lies behind these widening central policy objective. had faded considerably. The picture for disparities within the workforce? Man- women is similar, if slightly less stark. ufacturing jobs were once the gateway Broad trends that have converged At every level of education and training to the middle class, even for workers and accelerated since the middle 1970s have split the old middle class into the pattern holds: the higher the skill who started off without high-level level, the higher the earnings. And the skills. While it is a myth that interna- three groups: an underclass largely gap has been growing. tional trade has robbed America of its trapped in central cities, increasingly isolated from the core economy; an Not only the wages and benefits that manufacturing industries, technological changes have diminished the role of la- overclass of those who are positioned workers can command, but also their bor, especially unskilled labor, in the to profitably ride the waves of change; chances for holding a job at all, are di- and in between these the largest group, vided along lines linked to skills. This modern factory. And global trade and investment surely have hastened these an anxious class, most of whom hold gap, too, is widening over time. In the changes. Even within manufacturing jobs but who are justifiably uneasy 1970s, the average unemployment rate about their own standing and fearful for for people who had not completed high industries, a rising share of value is added before the assembly line begins, their children’s futures. Despite the school was 7%. By the 1980s, this rate and after goods are produced. progress of the past year, this division averaged 11%, and in 1993 it was over still casts a shadow over the state of the 12%. By contrast, the unemployment Labor unions have long helped American workforce. We can describe rate for workers with a college degree shore up wages and benefits even for this picture in greater detail with the or better has held fairly steady at workers without high-level skills. But help of new data, much of it recently around 3%. Over the most recent ten- today only 11% of the private sector compiled by the Bureau of Labor Sta- year period for which data are available workforce is represented by a union. A tistics.

54 Evaluation Forum n Issue 11 n Summer 1995 Commentary

revitalization of the labor movement However, the road to opportunity is en who have graduated from communi- would help reverse the erosion of the often much rockier for people of color. ty colleges earn 33% more than women middle class. Last month the national unemployment with only a high school diploma. rate for black Americans was over The most striking change in the Middle Class Revitalization workplace has been the brash arrival of 11%, more than double the rate for white Americans. Today black men the computer. In 1984, about 25% of Skills clearly matter, and they can be hold only three out of every one hun- American workers used computers on learned. We have to combat the forces the job. Last year almost 47% did so. dred managerial, professional, and dividing America’s workers with an ar- technical jobs. The median income for And contrary to the belief that comput- mory of new approaches to lifelong black males working full-time and full- er-literate youth are running circles learning. An additional 130,000 chil- around their technophobic elders, year in 1992 was $22,400, just 72% of dren each year can now be made ready the median income for comparable workers between ages 40 and 54 com- to learn at school through Head Start. white males. Hispanic Americans seek- prise the group most likely to work School systems throughout America with computers. Even workplace com- ing to join the middle class have con- have millions of dollars in new incen- fronted similar obstacles. Their unem- munication has been transformed, as tives to improve their performance. ployment rate tops 10%, and their one in ten workers now uses electronic During the summer of 1994, some mail to communicate with colleagues earnings have been steadily eroding. 120,000 disadvantaged youth combined And Hispanic workers are significantly around the corner or around the world. jobs with classroom instruction. Over less likely than their white counterparts the next six years, almost half a million But the computer revolution has to receive health insurance on the job. young Americans will be entering deepened the division of the American Closing these gaps requires a long, youth apprenticeships during the last workforce. Two-thirds of college grad- two years of high school, many of them uates use computers at work, but only painful process of national growth and healing to which we must remain com- receiving special skill training beyond one-third of high school graduates, and mitted. Virtually every willing worker high school. fewer than one in ten high school drop- outs. The vast majority of managerial, can expand his or her skills. Informa- Starting in the fall of 1994, 20,000 tion technology itself promises to vast- technical, and professional workers use young people will enter National Ser- ly increase the productivity of learning computers. But people in lower-pay- vice, earning money that they can apply ing, lower-skill occupations use them as it makes top-flight teaching tools to a college education. The three and more widely available. We already see far less frequently. The information one-half million people who take out emerging around us examples of so- highway promises to speed some peo- education loans each year now have the ple to desirable destinations, but it may phisticated yet accessible work that can option of repaying their loans as a per- form the platform for a new middle be leaving others stranded in the high- centage of future income. Fifteen mil- class. Some of the most notable job tech version of inner-city ghettos. A re- lion working families with modest in- cent study of census data found that growth has occurred among technicians comes are receiving tax relief. We have who defy the traditional categories of fully 5% of America’s households lack begun transforming the old unemploy- the old economy. Their ranks are pre- access to a telephone, the basic ticket ment insurance system into a reemploy- of entry to the information web. Other dicted to grow by nearly 40% over the ment system, insuring that in 1994 next decade. Data released recently data show that minority children are alone an additional 150,000 Americans show that the most rapid job growth is less likely to use computers at school who have lost their jobs will get the or at home. As personal computers be- in high-paying occupations, even with- skills or the job search help they need in traditionally low-paying industries. come standard equipment for learning to find new ones. We must also deliver This hints at the proliferation of new at home, children who are disconnected on health security, or our mission of from the system will fall further and kinds of middle-class jobs throughout turning the anxious class into a new, the economy. further behind. more secure, more productive middle The technician jobs that will sustain class cannot be completed. Ethnic Divisions in Rebuilding the core of the new middle class usual- American business has a crucial role the Middle Class ly require some education beyond high to play in building a new middle class. Despite the issues cited, long-term school, but they do not always demand Skills learned on the job, or in a work- data affirm the potential for building a a four-year college degree. Degrees related setting, tend to be especially new middle class. The skill-based divi- from community colleges are paying well-tailored to the requirements of the sions of today’s workforce are in some off, and their enrollments are rising. workplace. New data on work-related ways more readily overcome than the Men with community college degrees training show that the impact of such divisions based on race and gender that earn 26% more, on average, than men training is of the same magnitude as have haunted America throughout our with only a high school diploma. Wom- more traditional schooling. Men and history.

Evaluation Forum n Issue 11 n Summer 1995 55 Commentary

women who graduate only from high all workers along on the route to pros- If unchecked, these divisions can school, but who have received work-re- perity, investing aggressively in their corrode our society. Unlike the citizens lated training, earn more than people skills and making them partners in pro- of most other nations, Americans have who have attended some college but ductivity. This new compact is an im- always been bound together less by a who have not received any additional perative not only of corporate citizen- shared past than by shared dreams of a training, and almost as much as college ship but of firms’ own long-term better future. If we lose that common graduates who lack additional work- interests. future, we lose the glue that holds our based training. nation together. Nothing is more vital Achieving A Better Future to fulfilling our nation’s defining Yet most companies are not yet do- promise than preparing all Americans ing enough training, and what training A constant theme of American histo- they do provide is not being directed to ry has been the challenge of forging for meaningful, productive working lives. In this century’s waning years, in the workers who most need it. Among unity out of diversity. We are at an im- the hopeful, fearful confusion of the young college graduates, about 35% portant point once again in meeting this now receive work-related training. That challenge. This time the deepest divi- Cold War’s end, our most important mission is to restore hope to the anx- is nearly double the rate for high school sions are not based as much on race, ious class and lift despair from the un- graduates and more than four times the national origin, or geography. They are rate for high school dropouts. The based on the ability of individuals to derclass, and to affirm the conviction that the American dream of broadly workers most likely to receive training make their way in an increasingly tur- shared middle class prosperity still en- are white, male, well-educated, and bulent economy. The overclass is doing working in high-paying occupations. fine, but questioning its connection to dures. This imbalance serves only to harden the rest of America. The underclass is isolated in marginal enclaves walled off the divisions within the workforce. Editor’s Note from hope. The anxious middle class is The payoff to work-based training, being pulled and stretched by the need The commentary above is excerpted and the inadequacy and uneven distri- to work two or more jobs to keep a bution of such training, highlights the from Secretary Reich’s 1994 Labor Day family solvent, by uneasiness about need for a new social compact among speech delivered at the Center for Na- health care, by the specter that today’s tional Policy in Washington, D.C. American workers, business, and gov- job will disappear tomorrow, and by ernment. Businesses need to do their fears that their children will be denied part in this new compact by bringing the opportunity for a better life.

56 Evaluation Forum n Issue 11 n Summer 1995 Interviews Interviews

Editorial Introduction mocracy and a market economy. This is Faced with the necessity of unexpected a uniquely informative macro-perspec- layoffs in the 1990s, the company and In this issue of Evaluation Forum we tive on economic restructuring and its its unions, working with federal and are privileged to feature our first inter- implications. state agencies, have developed a mod- view with a respected European expert. In addition we are pleased to include el reemployment program to assist dis- This extensive interview addresses our first interview with representatives located Boeing workers in a period of some of the fundamental economic of management and labor in a major adjustment to economic change and challenges facing all industrialized na- U.S. multinational corporation — the the internationalization of markets. tions, and highlights the situation for aerospace giant, the Boeing Company. countries in transition to political de-

An Interview with Salvatore Zecchini Editor’s Introduction ropean nations, and the independent Treasury, the Minister for Agriculture, republics of the former Soviet Union, in and the Minister for the Coordination of Given the emphasis in this issue of their transition to market-based econo- Community Policies. He represented Evaluation Forum on industrial econo- mies. the Banca d’Italia in international bod- mies and their efforts to survive in a ies addressing economic issues in Eu- new competitive global marketplace, The Assistant Secretary-General re- rope, and participated in the technical we are pleased to have an interview ceived a Ph.D. in economics from the meetings of the Deputies of the Group with Salvatore Zecchini, the Assistant Wharton School of Finance in the Unit- of Ten. He was also a member of Italy’s Secretary-General of a strategically im- ed States, and is a former professor of delegation to the two technical negotia- portant international organization, the economics in Italy. He subsequently tions for the establishment of the Euro- Organization for Economic Cooperation served as economist for the Economic pean Monetary System and European and Development (OECD) in Paris, Research Department of Banca d’Italia. budget reform. From 1984 through 1989 he was Exec- France. Established in 1960, the In addition, Mr. Zecchini has pub- OECD’s mission has been to promote utive Director of the International Mone- tary Fund. In 1989 he was named Spe- lished numerous studies on corporate economic growth, sustainable employ- finance, European financial policies, ment, and a rising standard of living in cial Counsellor for Economic Affairs at OECD, and began his tenure as Assis- the consequences of Italy’s member- member nations. Its membership in- ship in the European Union, interna- cludes all the major advanced industrial tant Secretary-General and Director of the Centre in 1990. tional economics, monetary and finance nations, currently twenty-four countries. issues, and the transition from central Mr. Zecchini is also Director of Mr. Zecchini has had an impressive planning to market economies. OECD’s Centre for Cooperation with career, serving as economic advisor to Economies in Transition, which offers the Prime Minister of Italy, the Minister assistance to Central and Eastern Eu- of Foreign Affairs, the Minister of the

Question: For Central and Eastern age a wide-ranging program of dia- related to the transformation of the eco- Europe, the transition to a market econ- logue and cooperation with the transi- nomic system, and the integration of omy clearly represents a difficult his- tion countries. The goal of the OECD’s these reforming countries in the world torical passage. This is not unexpected cooperation has been to contribute to- economy, and 3) training activities. given the magnitude of the economic, wards the development of pluralist de- social and cultural changes occurring in mocracy and market-oriented econo- With regard to the subject of this is- sue of Evaluation Forum, it is impor- these societies. How has the Organiza- mies by mobilizing the experience and tant to note that from its inception the tion for Economic Cooperation and expertise of the OECD secretariat and Development (OECD) addressed the Member Countries. The cooperation CCET has given high priority to the human resource issues related to struc- challenges involved in this transition? takes a variety of forms, including tural change in the transition countries. 1) provision of advice on the design, Mr. Zecchini: In 1990, the OECD implementation and monitoring of ap- In particular, I am referring to educa- Council decided to create the Centre tion, social and labor market issues. In propriate policies, 2) policy dialogue for Co-operation with Economies in the framework of the CCET’s work Transition (CCET) to launch and man- with the Organization and its Member Countries on the main economic issues program for these areas, the OECD’s

Evaluation Forum n Issue 11 n Summer 1995 57 Interviews

Directorate for Education, Employ- Conference on Labour and Social Im- some cases we are still struggling to ment, Labour and Social Affairs holds plications of Structural Change in Cen- solve them. Fortunately, there are many responsibility for provision of technical tral and Eastern Europe. The proceed- encouraging developments in terms of assistance. The work program is fo- ings were subsequently published. Did institutional and economic reform. The cused primarily on 1) improved moni- this contribute a foundation for future leading economies in transition have toring of educational, labor market and OECD technical cooperation? turned the corner and are once more ex- social developments, and 2) design and periencing economic growth. Mr. Zecchini: The High-Level Con- implementation of an appropriate poli- ference was a concrete and early mani- Question: High unemployment has cy strategy that aims at helping to es- festation of our concern for labor mar- become a troubling feature of the labor tablish well-functioning labor markets, ket and social issues in the transition market in most transition countries. social safety nets and educational sys- countries. The main conclusions of this Could you comment on unemployment tems. conference remain relevant to the situa- trends and the underlying dynamics? Specific reviews of education, labor tion today. Broadly speaking, three Mr. Zecchini: The first phases of market and social policy are the main points are of particular relevance: transition to a market economy in the instruments used to pursue these goals. n First, during the transition an appro- Central and Eastern European countries In cooperation with the corresponding (CEECs) and NIS have brought about host country Ministries, these reviews priate set of human resource policies major changes in the economic and so- survey and analyze the full range of can work to help address major social and labor market problems re- cial fabric of each country, including available information on the recent de- widespread employment adjustments velopments in a sector. Drawing on this lated to structural change — these and, in most, a rapid increase in open information, the current policy frame- policies can facilitate a smooth func- unemployment. Although these adjust- work and institutions are examined tioning of the labor market while ments are an inevitable consequence of with a view to offering a critique of the providing the foundation for the so- the process of economic transforma- situation, highlighting positive develop- cial safety net, and the promotion of tion, they are producing growing hard- ments while offering suggestions where the social consensus for reform. ship for the population as a whole, and performance falls short. To date, these for certain vulnerable social groups in reviews have focused on the four transi- n Secondly, these policies can promote longer-term economic development particular. Furthermore, unemployment tion countries participating in the and poverty have created pressures on OECD’s Partners in Transition Pro- through enhancement of human cap- state budgets due to increased outlays gram: the Czech Republic, Hungary, ital, and promotion of cooperative for unemployment benefits and social Poland, and the Slovak Republic. This industrial systems. assistance. They are thus creating new year we will add to the list, as the first constraints for the disinflationary poli- education review of the Russian Feder- n Thirdly, they can strengthen plural- istic democracy by improving the cy strategy as well as for the reform of ation is undertaken. public expenditures. representation of the interests of the Besides the review process, the on- social partners. In most of the CEECs, unemploy- going series of activities targets a range ment materialized in a wave, following of technical and policy issues using a The conference proceedings were pub- the institution of market-oriented re- variety of approaches, including coun- lished by the OECD in a volume enti- forms. Already at the end of 1991, un- try-specific case studies, expert mis- tled Structural Change in Central and employment rates were close to 10% sions and seminars, and international Eastern Europe: Labour Market and (or even above) in Bulgaria, Hungary, conferences and workshops. The imple- Social Policy Implications. The first Poland and Slovakia. In these coun- mentation of the market-oriented chapter of this book addresses key is- tries, unemployment rates continued to recommendations drawn from these ini- sues in the transition that remain cen- grow at a fast pace during 1992. The tiatives is supported through training tral even as the transition has advanced. Czech Republic, which was the subject activities designed to reinforce the These include 1) sluggish growth in of a recent OECD labor market review, working capacity of host country insti- these economies, 2) privatization and is a significant exception to this pat- tutions. The OECD is also contributing enterprise restructuring, 3) a rise in un- tern. After initially experiencing a rise important statistical tools through the employment and labor marginalization, in unemployment (although at a lower development of education and labor 4) increased poverty, 5) widening eco- pace than most CEECs), there was a statistical databases that provide inter- nomic disparities between social significant decline in the number of job nationally comparable data series for groups and between regions, and 6) in- seekers in 1992, and a stabilization in several transition countries. ternational migration. the unemployment rate thereafter Question: In 1991, the OECD orga- Although much has happened since (around 3%). nized jointly with the International La- the time of the conference, the issues it The Russian Federation too has bor Organization (ILO) a High-Level raised proved to be right on target. In shown distinctive labor market pat-

58 Evaluation Forum n Issue 11 n Summer 1995 Interviews

terns. In general, employment in Russia inflows into unemployment have been ily overcome by geographical labor has been declining much more slowly relatively contained, even when com- mobility. Rather they require, at least in than output. In the three years since pared to those prevailing in low-unem- the short term, specific efforts in the 1991, employment is estimated to have ployment OECD countries. Also, there backward areas, and a redistribution of dropped by roughly 4 million to under have been significant flows that com- resources to them from wealthy areas. 70 million. Compared to most of the pletely by-pass unemployment. Direct Unemployment has risen and contin- CEECs, unemployment remains fairly job-to-job shifts and flows out of the ues to grow in some areas. However, low, with estimates of the unemploy- labor market have played an important this must be contrasted with ongoing ment rate ranging from roughly 2% role in the reallocation of labor. reallocation of labor and some eco- (based on administrative data on the Put another way, unemployment has nomic restructuring. Many new jobs registered unemployed) to about 6.5% been a rather stagnant pool, especially have been created although they are (based on estimated Labour Force Sur- in the early stages of the transition. In somewhat precarious, particularly those vey updates) in the fourth quarter of most of the transition countries, once a in small firms and in self-employment. 1994. As of October 1994, the popula- worker has lost his or her job it is very Some of the newly created businesses tion officially registered as being “out difficult to obtain another position. Un- are yielding good wages and benefits, of employment” was 1.75 million. Re- der these circumstances, unemployment but many do not and will not. cently unemployment has begun to rise is highly persistent, and in many cases and is likely to continue to do so, espe- Question: What have been the main the lack of job opportunities results in cially if the transition process gains elements of CEEC and NIS human re- the job seeker leaving the labor market source policy in the face of economic momentum. altogether instead of being reintegrated restructuring? Unemployment rates by themselves, into a job. Mr. Zecchini: The transition countries however, offer only a partial picture of As is the case in most high-unem- the labor market slack during the tran- have been quick to initiate develop- ployment OECD countries, the burden sition process. Sizeable flows from em- ment of human resource policies and of unemployment is unevenly spread programs corresponding to the require- ployment to out-of-the-labor force have across the different social groups in occurred. This is especially true where ments of a market economy. There transition countries. It has fallen dis- extensive use of soft dis-employment have been significant successes in this proportunately on the young, unskilled regard, although further adjustments in measures — such as early retirement and older workers, and some ethnic mi- and “easier” access to various kinds of the new policy framework or policy norities. Furthermore, even in a context disability benefits — was made in the implementation processes will be re- of stable or declining numbers of job quired. Moreover, the task of restruc- early stages of the transition, and where seekers, the average duration of unem- most working pensioners were pushed turing elements carried over from the ployment has increased dramatically. out of the labor market. Flows to inac- previous system has not been complet- Many long-term unemployed have left ed. tive status (rather than into unemploy- the labor market. These phenomena are ment) have also occurred among indi- signaling the difficulties now encoun- Labor market policy has been at the viduals of working age, especially in tered by certain social groups, which center of the reform efforts. Early in the Czech Republic and Hungary. In should be the focus of more attention the reform process, transition countries the latter, for example, the participation by labor market and social policy insti- enacted new comprehensive employ- rate for the working age population tutions. ment laws which are serving as the ba- dropped by some three percentage sis for labor market policy and pro- The economic and social hardship of points during 1993. In Russia, there has grams. These laws generally affirm the been a substantial growth in underem- reforms also has a geographical dimen- rights and responsibilities of people in sion, which was the subject of a Tech- ployment and wage arrears, as well as a employment matters, including the nical Workshop organized by the continuing decline in labor productivi- choice of whether to work and of occu- ty. OECD and the Institute for Advanced pation. In many cases such laws bor- Studies in November 1994. In most of In September 1993, the OECD orga- rowed from similar statutes of OECD the transition countries, large urban nized a Technical Workshop on “The Member Countries. Building on these centers have unemployment rates well employment laws, transition country la- Persistence of Unemployment in Cen- below 10% and rather favorable ratios tral and Eastern Europe.” Evidence pre- bor market policies and programs have between unemployment and job offers, sented there suggests that unemploy- generally sought to achieve: while rural areas and many mono-cul- ment has grown in many cases more as tural industrial sites are suffering from n Reduction in the level of unemploy- a consequence of low rates of outflow unemployment rates above 20% and ment. into jobs, than of unusually high rates few job openings. Given the rigidity n of inflow into unemployment. Despite and tight supply in the housing market, Decreased labor market segmenta- the massive employment adjustments in these regional disparities cannot be eas- tion and improved labor market most countries of the region, monthly

Evaluation Forum n Issue 11 n Summer 1995 59 Interviews

prospects for vulnerable groups in pension eligibility while increasing the budgetary pressures is constraining the society. average payment size. The implementa- modernization of education. tion of such strategies is fraught with n Generally, the system of training in Promotion of occupational and in- political risks and consequently does dustrial mobility, and delivery of the transition countries placed empha- not seem feasible over a short period. sis on enterprise-based programs. The basic income support for the unem- Family allowances vary, and in some transition has led to a substantial reduc- ployed during limited periods of job cases are based on inappropriate per tion in the number of training places search. capita measures that do not take into available in state-owned enterprises, account household economies of scale. Employment laws have provided for the and the private sector has been unable Approaches to the provision of further establishment of public employment so far to provide sufficient opportuni- social assistance have varied, depend- ties for initial and further education services charged with the administra- ing in part on the level of decentraliza- tion of unemployment compensation, training. This comes at a time when tion in the system. In some cases, the placement assistance and additional ac- there is a major need for retraining lack of national norms has caused a workers who have lost their jobs, or for tive labor market programs such as wide variation in the availability and training and public works. In some cas- training young entrants into the labor level of benefits. In the Russian Federa- es, the public employment services market. tion, for example, such assistance is were established as independent agen- primarily the responsibility of the local Question: During the 1991 OECD- cies. The development of these services and regional governments. ILO conference, migration issues were has been the focus of substantial raised as an area of possible future con- The education policies of the transi- amounts of international assistance, in- cern. How has this issue developed, tion countries need substantial and ur- cluding the technical cooperation activ- and how do you see it evolving? ities of the OECD. gent reform as well. As stressed in sev- eral studies conducted by the OECD Mr. Zecchini: Despite the severe limi- Across the transition countries, the during the last three years, there is an tations imposed by the centrally- structure of social policy is still under- urgent need for greater coherence be- planned governments, considerable ex- going substantial restructuring and/or tween labor market and education poli- ternal migration flows have been an development. Witness the fact that sev- cies. Education reviews carried out by aspect of the socio-economic develop- eral countries have recently introduced the OECD in the former Czech and ments of central and eastern European new social laws or policies for which it Slovak Republics, and Hungary and countries during the past decades. The is still too early to assess the effects. Poland, have identified the following ongoing transition process has radically Given the wide range of issues, I will common problems: changed the nature, if not the magni- focus on three aspects of social policy tude, of migration flows. that are closely tied to poverty issues: n The lack of long-term plans for the pensions, family allowances and social development of higher education, Continuing emigration flows are assistance. and the need to render the system likely to result from the labor market more flexible in responding to the slack, especially in certain backward In the old system, inefficiencies en- regions. Cross-border commuting has needs of a market economy. sured high demand for labor. Employ- also emerged in regions on the border ment was virtually guaranteed for those n Problems relating to the capacity of with the West. Evidence presented at a who could work, thereby ensuring in- educational institutions. recent OECD conference on Regional come during one’s working life, and a Unemployment in Transition Countries pension in the event of disability or re- n The growing social differentiation, suggests that this is directly related to tirement. As a result, social assistance and the problems of identifying so- the labor market conditions of the re- targeted only a small minority of the cial needs. gion of origin, and the existence of for- population, such as the handicapped or n The quality of the teaching staff, and mal agreements between the countries orphans. of origin and destination, as is the case “brain drain” problems. Today, pensions and family allow- between the Czech Republic and Ger- ances serve as a primary means for de- n The autonomy of institutions, the many. livering social protection. In reforming decision process, the legislative Official in-migration data suggest countries, restructuring of the pension foundation, and the division of com- that there are increasing flows toward system is required to ensure long-term petencies within institutions. transition countries, due also to the re- viability, particularly in the face of low In all cases, financial resource con- turn of former residents. However, offi- retirement ages and an erosion of the cial data may under-report inflows of value of benefits due to inflation. straints are proving to be a major con- cern. Competition with other govern- foreigners in transit through some tran- To combat low benefit levels, many ment priorities in the face of heavy sition countries toward Western Euro- strategies for reform aim to restrict

60 Evaluation Forum n Issue 11 n Summer 1995 Interviews

pean countries, as is the case for Po- particularly during the economic transi- introduce flat rate systems on a tempo- land. The cooperation between transi- tion. Economic policy must address a rary basis, as in Poland. Another option tion countries and the OECD in the broad range of structural impediments for improving the effectiveness of un- area of international migration had al- to growth, including those related to employment benefit systems might be ready started in 1991 in the framework human resources. Let me give a few ex- to improve the targeting of support of the Continuous Reporting System of amples. measures by reducing benefits for those Migration. This system is planned to who voluntarily quit their jobs, and for Problems in policy coherence are a further contribute to the improvement common issue in transition countries, new entrants to the job market. of information on the dimension of the as in OECD Member Countries. In Improved systems to monitor per- in-migration phenomenon, on one transition countries, OECD surveys formance, and provide feedback to em- hand, and the definition of suitable pol- have found particular problems in the ployment service offices, may also aid icies to cope with it, on the other. linkages between unemployment bene- in raising the efficiency and effective- Inter-regional migration is also an fits and the social assistance system, ness of labor market programs. Anec- issue of great importance in all transi- particularly concerning the long-term dotal evidence from some transition tion countries. As stressed before, re- unemployed. As a consequence, in countries suggests that well-designed forms have brought about major re- many countries a significant number of management information systems can gional disparities in the economic and the long-term unemployed do not bene- contribute toward needed performance labor market conditions of different re- fit from the placement and counseling enhancements. gions. Indeed, labor migration flows services of the employment services. As regards social policy, pensions seem to respond to differences in the Many lose contact with the labor mar- are of the greatest pressing financial economic conditions of regions. The ket. Other gaps in policy coherence can concern. As mentioned earlier, the major urban centers, and other regions be found in the relationship between long-term viability and effectiveness of with a diversified production base, education and labor market policy, with pension systems in most transition have experienced consistent in-migra- the result that graduates of educational countries will require a tightening of el- tion of labor in the last period. Howev- programs risk having an education that igibility criteria. er, inter-regional migration flows have fits poorly with the skills sought in the not increased in aggregate terms in all labor market. In the area of education, the effec- tiveness of the system should be en- transition countries, due to the major ri- In terms of labor market policy, gidities in the housing market, and the hanced by stressing the linkages with there is some evidence that many tran- lack of information regarding job op- the labor market. One means for sition countries could benefit from im- achieving this is to increase the in- portunities. proved targeting of public programs. volvement of the various stakeholders Question: Given the limited amount of resources Each of the transition in the design and implementation of ed- countries has entered the reform pro- available for these programs, and the ucational programs — that is, govern- cess in the context of unique exper- uneven distribution of unemployment, ment representatives from non-educa- iences and conditions which will re- it is most appropriate to focus on the tion agencies, business representatives, quire varied policy responses. Beyond most vulnerable groups, such as the and parents. More generally, education this, however, are there common long-term unemployed. Targeting would benefit from improvements in themes that you have identified for en- might also take into account the geo- national standards. hancing human resource policy? graphic distribution of labor market hardship. Also, evidence from the These examples do not constitute a Mr. Zecchini: It is important to un- OECD’s review of the Czech labor comprehensive policy framework. derline the importance of a lesson that However, they signal some of the poli- we have learned in the OECD Member market indicates that investing more re- sources in assisting the unemployed in cy components that most transition Countries. Experience from a wide their job searches, and in monitoring countries need to consider within the range of countries has shown that ongoing transition process. sound economic policy is the best base the job search efforts of the unem- ployed, leads to better results than oth- from which to promote employment. As er methods in term of job placements. the recent OECD Jobs Study points out, Editor’s Note a primary component of such a policy Many transition countries might consists of an appropriate macroeco- benefit from the simplification of the The interview with Mr. Zecchini was nomic policy that includes sound pub- unemployment benefits system. Often completed in February, 1995. The fol- lic finances, price stability, and promo- these systems involve complicated pro- lowing OECD publications may be of in- tion of a sustainable balance of cedures and calculations that strain the terest to readers wishing to pursue fur- payments position. However, a sound administration, while at the same time ther the issues covered in the interview. macroeconomic policy by itself cannot delivering insufficient benefits. One Again, CCET stands for the Centre for guarantee sustained economic growth, option for simplification might be to Co-operation with Economies in Transi- tion.

Evaluation Forum n Issue 11 n Summer 1995 61 Interviews

Labor Market and Social Review of Labour Market and Social Education Issues Policy Issues: Policies of Hungary. Paper present- Review of Higher Education in the Employment and Unemployment in ed at the ELSA Committee Meeting Czech and Slovak Federal Repub- Economies in Transition: Conceptual in Budapest, October 1994. lics: Examiners’ Report and Ques- and Measurement Issues. Eurostat. Short-Term Economic Indicators — tions, 1992. CCET. Paris, 1993. Transition Economies. Labour Mar- Review of Education Policy in Hungary: The Labour Market in Poland. CCET. ket Indicators Annex, No. 4, 1994. Examiners’ Report and Questions, Paris, 1993. Unemployment in Transition Countries: 1993. Structural Changes in Central and Transient or Persistent. CCET, Follow-Up Review of Higher Education Eastern Europe: Labour Market and 1994. in the Czech and Slovak Republics. Social Policy Implications. A joint The Regional Dimension of Unemploy- Report to OECD’s Education Com- publication of the OECD and the ment in Transition Countries: A mittee, 1994. ILO. Paris, 1993. Challenge for Labour Market and Seminar I: Quality Assurance and Ac- Current Labour Market Problems in Social Policies. Proceedings of the creditation in Higher Education. Russia. General Distribution Docu- Technical Workshop on “Regional General Distribution Document. ment. CCET, 1989. Unemployment in Central and East- CCET, 1994. ern Europe” jointly sponsored by the Review of the Labour Market in the OECD’s CCET in Paris and the Insti- Seminar II: Mobility in Higher Educa- Czech Republic. Paper presented at tute for Advanced Studies in Vienna, tion. General Distribution Document. the ELSA Committee Meeting in November 1994. CCET, 1994. Prague, December 1994. Seminar III: Research in Pedagogy. General Distribution Document. CCET, 1995.

An Interview with Gary K. Jackson and John E. Dryer Introduction* development of IAM hourly employees petitive market in which aerospace at Boeing through a number of unique companies operate, and the volatility of On April 17, 1995, the United States services. orders under government contracts, Secretary of Labor, Robert Reich, came The Quality Through Training Pro- The Boeing Company has always ex- to Washington State to meet with the gram (QTTP) has played an integral perienced changes in the volume and co-directors of the IAM/Boeing Quality role in planning and obtaining the fund- characteristics of its workforce over Through Training Program which Sec- ing for the Boeing Reemployment pro- time. This has been related to the gen- retary Reich called a model of labor- gram, which has two service centers of- eral health of the economy, to trade re- management cooperation. “We are be- fering a “one-stop shopping” approach lations, and to the state of technology. ginning to use this model around the to retraining and job placement servic- However, by the late 1980s Boeing was country, and it’s working,” Secretary es for displaced Boeing workers. The facing a new competitive environment Reich said. “Labor and Management Boeing Reemployment Program is ad- that was considerably internationalized. can work together, even when the news ministered by the State Employment The company was competing with sub- for workers is bad.” The Secretary also Security Department (ESD), and brings sidized aerospace designers and man- brought a check for $4.5 million in sup- together The Boeing Company, the ufacturers in other countries. There port of the Boeing Reemployment Pro- company’s two major unions — the In- were recessions in the industrialized gram. ternational Association of Machinists nations. And defense contractors, in- Leading the Quality Through Train- and Aerospace Workers (IAM & AW) cluding Boeing, were coping with the ing Program, John Dryer, Boeing Co- and Seattle Professional Engineering military downsizing that accompanied Director, and Gary Jackson, IAM Co-Di- Employees Association (SPEEA), local the end of the Cold War. rector, work together with the support of Private Industry Councils (PICs), a con- The specter of business losses that a team of Administrators from both the sortium of seventeen Community and could result from the reduced numbers company and the union to develop Technical Colleges, and the QTTP in a of orders and the cancellation or post- training opportunities that assist IAM- broad-based drive to help dislocated ponement of previously-committed or- represented Boeing employees impact- workers and mitigate the effects of job ders from airlines, who were them- ed by technology change, job combina- loss. selves experiencing a business tions, and workforce reduction. The The aerospace industry by its very downturn, brought about the need to program is also proactive in developing nature experiences peaks and valleys reassign, relocate, or lay off unprece- programs for the career and personal of employment. Because of the com- dented numbers of employees. In addi-

*We would like to convey our special thanks to Ellen Hewitt, a staff person in QTTP, who contributed substantially to this introduction.

62 Evaluation Forum n Issue 11 n Summer 1995 Interviews

tion, because of the intensifying inter- ees, to assist them in preparing for new center” concept was included in an ap- national competition from state-subsi- career options. Funded under the 1989 plication for federal funding of two ma- dized aerospace companies, Boeing and 1992 collective bargaining agree- jor Boeing Reemployment Centers in has had to drastically pare production ments by an amount equivalent to !0 the Puget Sound area. costs, exacerbating problems for work- cents per hour for each IAM-represent- John Dryer, Gary Jackson, and ers. ed worker employed, with a guaranteed members of the Quality Through Train- Although employment at the compa- $10 million base, the Quality Through ing Program staff, along with represen- ny was at its peak in 1989, the IAM and Training Program did much to help tatives of the company, the IAM and the company were well aware of the IAM-represented employees become SPEEA unions, the Employment Secu- forces that could affect employment reemployed by offering an education rity Department, Private Industry Coun- conditions, if not the exact events that assistance program and opening “Hori- cils, and the State Board of Community were to follow. Consequently, negotiat- zons” learning centers offering advis- and Technical Colleges formed a Labor ed into the collective bargaining agree- ing, counseling, career exploration, and Management Committee and started an ment that year was the new jointly-run skills training. QTTP also saw the need intensive effort to apply for funding un- Quality Through Training Program, es- to provide a placement function for em- der the Job Training Partnership Act. tablished to develop training programs ployees. Charles Wetmore and Della Curry of for IAM-represented employees impact- In 1993, QTTP contracted with Main- the QTTP staff led the team that put to- ed by the above-mentioned conditions stream Access and worked with Boeing gether the funding package. The appli- in the workplace. Outplacement and the Washington cation for funding featured the ESD as There were almost no layoffs during State Employment Security Department administrator for the program. This ap- the period between 1983 and 1990 — a (ESD) in a Job Opportunities Program plication was granted and an unprece- period of expanding business and high to deliver services to IAM workers. The dented $5 million was awarded to the production for The Boeing Company. focus of the Job Opportunities Program project — the highest up to that date for The problem then was building an ade- was to provide the supportive environ- any Washington State program. quate workforce to produce the compa- ment and services needed to help IAM The Boeing Reemployment Pro- ny’s airplanes. employees become reemployed as gram, a comprehensive program deliv- quickly as possible. Job clubs were Reductions in defense-contract work ering a full range of reemployment ser- formed, where employees with similar vices at two one-stop, multi-service brought the first cloud on the horizon, interests and goals met together with a resulting from the easing of the Cold centers — a remarkable private/public job counselor on a regular basis. The partnership between The Boeing Com- War and the federal government’s job clubs also afforded the opportunity downsizing of military programs. This pany, its unions, and government agen- to learn interviewing and resumé writing cies and institutions — was born. affected the Boeing Defense & Space skills. Job fairs, job and skill matching, Group. Then came substantial aircraft and job placement services were in- nnnn production cutbacks directly related to cluded in the mix. The interviewer had the opportunity the decline in the commercial aircraft to talk with the two QTTP co-directors market worldwide. Boeing Commercial Services provided by ESD’s local centers were also included, such as as- in March 1995. The following interview Aircraft Group had to resize as produc- is the result of this meeting. The inter- tion schedules shrank. Since Washing- sistance with the Unemployment Insur- ance Claim process, and job search viewees are John Dryer and Gary Jack- ton State’s economy was to a signifi- son. cant extent dependent on Boeing, the and job placement on-line through the downsizing made necessary by these Employment Service. ESD representa- Gary Jackson is Co-Director of the cutbacks also had important ripple ef- tives were located on-site at the QTTP IAM/Boeing Quality Through Training fects on the hundreds of firms in the re- offices. Since QTTP was certified by Program, which is jointly sponsored by gion that constituted Boeing’s extensive the ESD to grant Commissioner-Ap- the International Association of Machin- supply and service chain. proved Training under Title III of the ists and Aerospace Workers and The Job Training Partnership Act — in fact it Boeing Company to address the train- By the end of 1994, Boeing employ- is the only private organization to enjoy ing and employment needs of active ment had dropped 20,000 from the that status — this program also provid- and laid-off Boeing workers who have 1989 peak of 107,000 employees, as ed the benefit of being able to draw un- been affected by changes in the aero- reported by Secretary Reich during his employment insurance while going to space industry. visit here. This included attrition due to school through QTTP. resignations and retirements, as well as Mr. Jackson, an experimental elec- layoffs. According to Washington Em- The Job Opportunity Program gave tronics technician while at Boeing, held ployment Security Department figures, rise to a new, expanded concept — one numerous Union positions as a Local 8,000 employees were laid off during which brought wider participation from Lodge officer prior to becoming a full- 1993 — the largest during any one year the community and provided a full time staff person with District 751 of the in the early 1990s. range of benefits to all dislocated em- International Association of Machinists ployees of The Boeing Company. IAM- and Aerospace Workers. He has The Quality Through Training Pro- represented employees still took the served in the Union’s Health and Bene- gram built up its Career and Personal major “hit” from downsizing the work- fits and Wage Determination offices, as Development Services for both active force, but other areas of the workforce well as Contract Negotiations. He has and laid-off IAM-represented employ- were also being affected. The “one stop

Evaluation Forum n Issue 11 n Summer 1995 63 Interviews

been involved with the Quality Through this philosophy. In addition, the “one velop their own individualized career Training Program since its inception in stop center” concept is a key feature of education plan, utilize career assess- 1989, first as a Joint Policy Board previous QTTP job placement activi- ment tools, participate in on-site com- member and now as Co-Director. ties, and provided the organizational puter-based training, or be referred to John Dryer is the Chairman of the framework for the Reemployment Pro- other training/educational opportunities Labor-Management Committee in- gram. which match their skills and pace of volved with the Boeing Reemployment learning. In addition, the program of- Program, and Co-Director of the IAM/ Our major issue at this time is the impact of company downsizing on our fers on-site courses in math, reading, Boeing Quality Through Training Pro- writing, creative thinking, problem gram. A design engineer, Mr. Dryer was employees. QTTP offers Career and solving, presentation skills, and study Director of Labor Relations at Boeing in Personal Development programs for Kansas prior to coming to the QTTP. both active and laid-off IAM-represent- skills. Tutors are available on-site. Community and technical colleges pro- He has served as a consultant on ne- ed employees of The Boeing Company. vide the professional advisors and adult gotiation strategies for contract negotia- QTTP’s support and leadership in re- tions in both the public and private sec- gard to the Labor Management Com- basic educational instructors. English tor, and has lectured on labor relations as a Second Language (ESL), and a re- mittee and the Boeing Reemployment at the University of Puget Sound. cent addition, American Sign Language Centers has helped a great many laid- Mr. Dryer has been intensely in- off employees “land on their feet” and, classes are available for IAM employ- volved in the evolution of the Quality ees. ESL classes, offered in partnership as Secretary Reich put it, “prove that Through Training Program, providing with organizations throughout the com- guidance to the program in addressing there can be new work and new life af- ter the pink slip.” pany, are set up for both active and the personal, training and career needs laid-off employees in several locations of workers. His emphasis on joint Career and Personal Development convenient to either their workplace or union/management sponsorship has re- programs include four full-scale advis- home. sulted in a high level of acceptance of ing and learning centers at this time: these programs by both Boeing and QTTP’s other two initiatives — pro- union management. the Advising and Learning Center at our main offices in Tukwila, Washing- viding assistance to IAM employees nnnn impacted by technology change and job ton; the North Site Advising and Learn- combinations — address the training Question: As the initial effort to as- ing Center in Everett, Washington; the sist the major segment of the Boeing Wichita, Kansas, Advising and Learn- needs brought on by change in the workplace. This includes IAM-repre- workforce, both those needing to be as- ing Center; and a new QTTP/Boeing sented employees whose jobs are af- signed to new jobs within the company Advising and Learning Center in Port- and those ultimately dislocated — is land, Oregon. A fifth is due to open fected by technology change (defined in the collective bargaining agreement the model established by the Quality soon. Two of these locations also ad- as technological improvement in Through Training Program the core of minister the QTTP Education Assis- the Boeing Reemployment Program? tance program — which offers an an- “tools, methods, processes, equipment or materials”) such as employees at the And what are the main elements of nual benefit to active or laid-off new Chemical Process Line and the QTTP’s program design? IAM-represented employees of the company. Active IAM employees can Calibration Lab. For the new Chemical Gary Jackson: The Quality Through Process Line, self-paced video and receive up to $2,000 per calendar year Training Program came into being to computer instruction developed by provide IAM-represented employees of for education and training at state-rec- ognized institutions, and laid-off em- QTTP in cooperation with Boeing The Boeing Company with learning op- Training Organization is located right ployees whose training needs do not fit portunities that can lead to career at the workplace and a peer mentor growth, provide job security, and re- the Boeing Reemployment Program guidelines for retraining can receive up training system is in place. Calibration train employees impacted by changes Laboratory employees can access a so- to $2,500 per year for three years after in the workplace due to technology phisticated interactive electronics change or job combinations. This con- layoff, beginning on the date of their layoff from the Company. These bene- course cooperatively developed by cept of cooperation between company QTTP and Boeing Training that incor- fits are available to all IAM-represent- and union in order to provide important porates the elements of an AA degree career and education opportunities is at ed hourly employees, regardless of their location. in electronics. Technology change re- the heart of all QTTP activities. Bring- training has been developed and deliv- ing the company, unions, educational QTTP’s Career and Personal Devel- ered in a number of other locations in institutions, Private Industry Councils, opment initiatives include a broad spec- Washington, in Portland, Oregon, and the Employment Security Department, trum of career development and skills in Wichita, Kansas. and other federal and state agencies to- enhancement programs for employees. gether in a cooperative effort to benefit Through this program employees can Job combinations occur when the Corporate Jobs Committee agrees to employees is a natural outgrowth of meet with professional advisors to de- combine two or more bargaining unit

64 Evaluation Forum n Issue 11 n Summer 1995 Interviews

jobs. QTTP has a responsibility to iden- n The Region X office of the U.S. De- diately set to work on opening two one- tify what additional training needs may partment of Labor. stop, multi-service Boeing Reemploy- be required for the affected employees ment Centers, one in each of the major to do the new combined job. QTTP in- In early 1993, QTTP, leading the ef- counties where Boeing dislocated vites employees as subject matter ex- fort to obtain federal funding, contract- workers live and work, and a satellite perts and line managers in the areas ed with the Human Resources Develop- center in a third county. These centers where jobs were combined to partici- ment Institute (HRDI) in Washington, involve the out-stationing of staff from pate in a training needs analysis. The D.C., an arm of the AFL-CIO, to assist Boeing Outplacement, both unions, the requirements of the new job are scoped with the program design and applica- Washington Employment Security De- out through brainstorming techniques, tion. HRDI worked with the Labor- partment, Private Industry Councils, then validated and verified by addition- Management Committee as a facilitator and the State Board for Community al groups of subject matter experts. to develop and submit a grant proposal and Technical Colleges. Finally, the skills list is given to Boeing to the U.S. Department of Labor for a National Reserve Grant under Job Secretary of Labor Reich pro- Training for identification of the train- nounced the Boeing Reemployment ing courses required for each job level. Training Partnership Act (JTPA) Title Program a model program that he The purpose of using this job task anal- III, for services to Boeing workers meeting the eligibility criteria under the wanted to replicate elsewhere through- ysis is to gather data from the employ- out the United States. On June 20, ees who actually do the work. Economic Dislocation and Worker Ad- justment Assistance Act. 1994, Vice President Al Gore and Sec- The Quality Through Training Pro- retary Reich presented the “Hammer gram has shown that the union and the The initial request was for $10.2 Award” to the Boeing Re-employment company, working together, can help to million for a new Boeing Reemploy- Centers for “Putting Customers First.” ment Program for approximately 3,800 increase opportunities for Boeing The success of the Boeing Reem- workers and provide Boeing with a dislocated Boeing workers. The request ployment project was acknowledged in highly skilled workforce for a changing was urgent, as existing budgets ac- September 1994 when the U.S. Depart- work environment. quired through unprecedented leverag- ing of funds and in-kind contributions ment of Labor awarded an additional Question: How was the Boeing Re- across a number of private and public $5.2 million to assist laid-off Boeing employment Program established and sources were insufficient to meet the workers in Washington State. This en- organized and how is it being funded? needs of the large numbers of workers sured that the centers could continue to provide dislocated worker services to John Dryer: The new Labor-Manage- dislocated in the 1990s. The new pro- ment Committee brought together rep- gram redefined the unemployment sys- all laid-off Boeing workers, including an additional 1,000 employees. Of this resentatives of a wider array of constit- tem to a reemployment system and in- requested amount, the largest propor- uencies, and charged them with troduced a number of new concepts responsibility for a new program spe- that met the current thinking of the De- tion was for longer-term retraining, along with money earmarked for sup- cifically targeted to dislocated Boeing partment of Labor: a public/private portive services such as child care and workers across all divisions of the com- partnership; one-stop shopping for ser- pany. These representatives were to vices; a consolidation of customer-fo- transportation, additional case man- agement for work-based learning and speak for an impressive group of stake- cused services; market-driven training, job development staff, and a full-time holders: and worker empowerment and account- ability. project director for the program. n The Boeing Company. In reviewing the request, the U.S. The additional funding received on n The International Association of Department of Labor awarded an im- April 17, 1995, adds another $4.45 mil- Machinists and Aerospace Workers. mediate grant of $5 million and an op- lion — just in time to meet the needs of tion for requesting additional funding employees who will be impacted by ap- n The IAM/Boeing Quality Through should the award prove inadequate to proximately 6,000 more layoffs an- Training Program. the need. Experienced in operating nounced by Boeing for 1995, and n The Seattle Professional Engineer- JTPA Title III and other employment bringing the total grant to $14.65 mil- ing Employees Association. and training programs, the Washington lion. State Employment Security Department Looking at all the resources support- n The Washington State Employment was to receive the National Reserve ing the Boeing Reemployment Pro- Security Department. Grant on behalf of the Labor-Manage- gram, we see an inventive combination n The State Board for Community and ment Committee, and was assigned the of public/private funds and programs to Technical Colleges. role of administrator and operator of service dislocated workers. In terms of the program. funding, the program is utilizing spe- n Several Private Industry Councils: With the federal grant in hand, the cial federal funds, regular federal pro- Seattle/King County, Pierce County, Labor-Management Committee imme- gram dollars, and funds associated with and Snohomish County.

Evaluation Forum n Issue 11 n Summer 1995 65 Interviews

a new state legislative initiative that has workers. Accommodating the various ing and evaluation requirements. Al- expanded training slots in the state’s cultures, policies, and practices of these though no net impact study of EDWAA community and technical college sys- groups has required a great deal of has been conducted at the national lev- tem. Regarding programs and related trust, commitment, communication, el, the Department of Labor did con- support, the Program is bringing to- and sometimes negotiation. tract with a competent research firm to gether federal, state, and sub-state pro- study EDWAA implementation. The The partnership involved accommo- grams; personnel in community and latter may serve as a useful model for dation and a willingness to cooperate in technical colleges; facilities, equipment areas where organizations might be ac- studying coordination and service de- and staff contributed by The Boeing livery in the Boeing program customed to having full discretion and Company; the IAM/Boeing Quality authority. Many strong organizations The use of state funds for increased Through Training Program, IAM and have been involved. In my opinion, the community and technical college op- SPEEA volunteer representatives, and partnership has been an outstanding portunities for dislocated workers, other Boeing programs. success. which involves a set-aside for Boeing To date, over 12,000 employees workers, will require evaluation activi- The Boeing Reemployment Program have utilized Reemployment Center ties consistent with the evaluation plan represents a significant human resource outplacement services. Approximately and reemployment effort involving a developed by the State’s Workforce Ed- 5,580 Boeing employees have obtained ucation and Training Coordinating large multinational corporation — and employment, and 3,365 employees are Board, which calls for a study of both key labor unions — and three levels enrolled in JTPA retraining programs. of government. The program is also the implementation and the legislative intent behind this funding, and a study Question: We have been hearing a unusual in its commitment to keep of outcomes for participants, employers great deal about European models of communication open with employees, collaboration between government, elected officials, business and labor and the community college system. business, and labor, related to the cur- communities, and others invested in the The Labor-Management Committee rent U.S. efforts to increase cooperation program, concerning the program’s has selected a set of outcome measures among these different constituencies. progress. and has established quantitative goals While this kind of partnership has been for monitoring the Reemployment Pro- Question: Who has oversight respon- more traditional in Western Europe, sibilities regarding the Boeing Reem- gram, such as “entered employment these three groups have often behaved rate,” average wage at entered employ- ployment Program, and how are its re- in an adversarial manner in the United ment,” “cost per participant,” and “cost sults to be judged? States. How difficult has it been to de- per entered employment.” In addition velop this kind of partnership for Boe- John Dryer: Because of the multiple the Committee is interested in request- ing workers? funding streams and programs for this ing private/public funds to conduct a new effort — and the fact that differ- study of the program’s overall efficien- Gary Jackson: Clearly it has been a ing expectations and formal regulations cy and effectiveness. greater challenge to develop a govern- exist among them regarding the evalua- ment/business/labor partnership in the tion of performance — there is no one On a human level, the fact that so Reemployment Program than it was to many former Boeing employees are re- source or method of assessment for the coordinate resources in the IAM/Boe- ceiving retraining and have found em- entire program. Nevertheless, oversight ing Quality Through Training Program. responsibilities are given strong atten- ployment through the program, at a rate The Reemployment Program brings to- of pay equivalent to what they earned tion by the Labor-Management Com- gether many more diverse, but well-es- at Boeing, constitutes the highest mea- mittee and the program’s administrator. tablished organizational entities. Asso- sure of success. This program has dem- ciated clusters of stakeholders had to be The use of Title III funds and a Na- onstrated that a cooperative partnership brought together around reasonably tional Reserve Grant require compli- approach to dealing with dislocated common purposes and goals for our ance with federally-mandated monitor- workers can and does work.

66 Evaluation Forum n Issue 11 n Summer 1995 Evaluation Issues and Activities

At the International Level

Editorial Introduction sues in the European Union and the ternational economy. They are issues former Soviet Union. These are issues about which American employment and This portion of Issue #11 emphasizes facing all industrialized nations, even training professionals need to remain economic reform and restructuring is- though to varying extent, in the new in- informed.

Economic Reform in the European Union

Editorial Introduction that the EU must first consolidate its n To achieve the highest sustainable own internal development so that en- economic growth and employment, In the midst of negotiations to form a largement does not have a destabiliz- and a rising standard of living in 12-member European Union (EU), the ing effect on European unity. member countries, while maintaining definition of “Europe” has grown in- The issue of EU enlargement is only financial stability, and thus to con- creasingly elusive. The eligibility re- one of many difficult issues faced by tribute to the development of the quirements for membership in the Euro- the leaders of this important experiment world economy. pean Community (EC) have reflected in transnational integration. However, that confusion. The Maastricht Treaty of n To contribute to sound economic ex- the increased consolidation of Europe pansion in member as well as non- 1991 proposed that “any European has substantial economic implications State whose systems of Governance member countries in the process of for the rest of the world, since it will economic development. are founded on the principle of democ- vastly expand intra-European trade and racy may apply to become members of expand the EU’s access to global mar- n To contribute to the expansion of the Union.” But the concept democracy kets. And the EU’s growing regional co- world trade on a multilateral, non- was not spelled out precisely. The or- operation agreements with surrounding discriminatory basis in accordance ganization of free elections, the prac- non-EU nations will play a significant with international obligations. tice of the rule of law, and respect for role in creating a larger European eco- civil and human rights were assumed to nomic structure. Already European vot- The original member countries of the underly this concept. But member ers participate in the election of the Eu- OECD were the world’s major industri- countries were also expected to devel- ropean Parliament. There is already a alized nations — eighteen nations in op stable market economies based on European Council of Ministers, a Euro- Western Europe, and the U.S.and Can- private entrepreneurship. These twin in- pean Council, a European Commission, ada. Although less industrialized, Tur- tentions complicated the issue of eligi- and a European Court of Justice. The key was also a member. Later Japan, bility since countries varied consider- Single European Act of 1986 supported Finland, Australia, New Zealand and ably in the extent to which they had the development of a common market Mexico were added. The Commission applied these abstract concepts. for goods, labor, capital and services. of the European Communities now par- Beyond that, the gradual enlarge- The Treaty on European Union in 1991 ticipates in the OECD’s studies and ment of the EU, however inevitable, strengthened political and monetary other projects, and OECD has extend- has remained controversial politically ties. ed its work to encompass economic is- sues in Central and Eastern Europe, and economically. A significant memo- Within this larger framework, the Or- randum submitted to the European and the republics of the former Soviet ganization for Economic Cooperation Union. Council in the early 1990s by the three and Development (OECD) has played a Benelux countries stressed that mem- critical advisory role in providing cross- In this series of articles we look at bership for the smaller countries could national data and policy analyses on a labor market issues in the European change the nature and form of the Eu- range of issues confronted by the most Union, and again focus the microscope ropean Community, might distort the advanced industrial nations, including on the new employment and training basic objectives of the founder nations, the EU. The OECD was established in initiative in Great Britain. In the pro- and would likely erode the position and Paris in 1961 to promote the following cess, we rely on a number of analyses role of the smaller members. France impressive goals: such as those carried out by the OECD. has taken the position, for example,

Evaluation Forum n Issue 11 n Summer 1995 67 Evaluation Issues and Activities n Economic Reform in the European Union nn Only a portion of this unemployment Kingdom. In most other OECD nations, The OECD’s was cyclical. Much of it was attrributed unemployment rates were higher for Employment Study to structural factors. Most of this struc- men. Regional differences occurred tural unemployment was considered by more in Europe than in the U.S. Age In 1992 the ministers of the coun- the OECD analysts to be the result of was another factor. tries with membership in the Organiza- dissonance between pressures on eco- There was a striking difference in tion for Economic Cooperation and De- nomic systems to adapt to global eco- velopment requested a literature search long-term unemployment rates. In the nomic change, and their willingness European Community, 40% of the un- and major study addressing key issues and ability to do so. in several broad areas: macroeconomic employed were unemployed long-term policy, trade, technology, wage forma- These analysts made the assumption compared with 15% in EFTA and 11% tion, labor adjustment, education and that the OECD economies were not in North America (U.S. and Canada). training, and benefit systems and taxa- flexible enough in coping with new In several EC countries more than two tion. OECD’s response to this mandate technologies and trade challenges, and out of every three unemployed older is reported in two volumes: 1) The therefore were relying too heavily on workers were unemployed long-term. OECD Jobs Study: Facts, Analysis, methods that were protectionist, that In general, the OECD statistics suggest and Strategies, and a background vol- sought to restrict competition. And they that EC countries have a greater rehir- ume 2) The OECD Jobs Study: Evi- believed that these nations had placed ing/reemployment problem than other dence and Explanations. In this article too little emphasis on developing new member nations, given the correlation we review the first of these companion products and new processes, new kinds between the low rates of flow back into reports. of workforces and workplaces, and new the labor force and the high incidence ways of creating jobs. of long-term unemployment. In the The emphasis in this international U.S. and Canada, workers have a great- study is on unemployment, the most The Nature of Unemployment er risk of becoming unemployed but a troublesome aspect of capitalist econo- The number of unemployed in better chance of reemployment. mies. This focus is consistent with the OECD countries tripled within a single OECD’s mission to contribute expert Changes in the Size and decade, between 1972 and 1982. An opinion and the results of research to Composition of the Labor Force easing of the prolonged period of eco- the efforts of industrialized nations to nomic decline of the late ’70s and early The working-age population in all 1) achieve prosperity and 2) maintain their ability to be innovative in coping ’80s was reduced only slightly by sub- OECD countries is growing more slow- sequent economic progress, and then ly than previously, the report points with an increasingly integrated and rose sharply again after 1990, accord- out. North America and Oceania have competitive global economy. ing to the report. However, the upward experienced a fairly rapid decline, but The major recommendation in the trend in unemployment was more se- the EC, EFTA and Japan have experi- OECD report is that nations must em- vere in the European Community (EC) enced a greater decline overall. Howev- brace change, rather than slow its mo- and in Oceania (Australia and New er, the size of the working-age popula- mentum or offer too much protection to Zealand) — 10-11% in 1994, as com- tion participating in the workforce activities that have not adapted success- pared with approximately 7% in the grew significantly to 75% in North fully to the significant changes occur- U.S. at that time. Unemployment stabi- America, Oceania and EFTA, remain- ring in the world. Strategies that pro- lized at 2-4% in the European Free ing largely unchanged over the past mote adjustments to change and Trade Association (EFTA) countries — thirty years in the EC countries (67%). enhance social cohesion while such ad- Austria, Finland, Iceland, Norway, Higher enrollments and retention justments are being made, are viewed Sweden, Switzerland and Leichtenstein rates in education spelled a decline in as the route postindustrial societies — until 1990 when it rose rapidly to al- must take if they are to provide the lev- the labor participation of young people, most 8% in 1994. Japan’s 1994 rate while early-retirement plans and dis- el of life quality they have traditionally was 3%, in the context of 1-3% ability pensions reduced the number of valued for their citizens. throughout the post WWII period. older workers. The report concludes Facts and Assumptions The report indicates that young peo- that these developments had the effect ple and women were most affected by of infusing some labor forces with According to the report, 35 million unemployment. Austria and Germany, more highly qualified labor market en- people were unemployed in OECD with strong apprenticeship systems, trants. countries in 1994 — 8.5% of the OECD were exceptions to the high youth un- labor force. Another 15 million were employment rates elsewhere. In Italy, Job Growth thought to have given up looking for Spain and Finland more than 30% of work, or had needed to accept a part- Growth in employment has been young people were unemployed. Wom- strongest in North America and Ocean- time job. A third of young workers in en had higher unemployment rates than ia, the report says, and weakest in the some of these countries had no job. men in the EC, except for the United EC and EFTA, with Japan in between.

68 Evaluation Forum n Issue 11 n Summer 1995 Evaluation Issues and Activities n Economic Reform in the European Union

The EC’s slow job growth rate is felt to growth in private sector employment and their capacity and willingness to be related to its slower population has increased. adjust. This gap was created, they in- growth and the lack of increase in the sist, by rigid even if well-meaning eco- proportion of its population in the Wages in Employment nomic, labor market and social policies workforce. Job growth has occurred The OECD report reminds us that that had the unintended effect of de- predominantly in the service sector in “the process of wage determination is creasing the ability of economies to all OECD countries. Agricultural em- strongly influenced by labour market adapt to change. ployment declined from 14% in 1970 pressures, social perceptions, legisla- They ask us to review history begin- to 77% in 1994. Industrial employment tion, and industrial relations systems, ning in the mid-1960s. Essentially, ev- decreased from 40% in 1970 to under which all affect the evolution of real erything was coming up roses for all 30% in 1992. The share of OECD em- wages and wage differentials.” Wage OECD countries in terms of stability, ployment in the service sector rose shares in national income, the report growth and trade. But by the middle of from less than 50% in 1970 to approxi- says, decreased to below 1970 levels the 1960s economic performance had mately 65% in 1992. Most new jobs in following the 1979 oil-price crisis. At begun to decline and inflation to rise. the 1980s were in financial services, in- the same time, wage differentials be- This produced an uneasy economic en- surance and business services, and tween low vs. high-skilled workers wid- vironment. Exchange rates became community and personal services. ened in the English-speaking countries, less predictable with the breakdown of The growth in international trade while EC differentials remained rela- the system of fixed rates. In the 1980s and technology are likely responsible, tively unchanged. In fact, low pay was the deregulation movement regarding according to the report, trade having more prevalent in the U.S. than in other financial and product markets enhanced grown faster than the Gross National OECD countries. In the ’90s one-fourth efficiency while accelerating change. Product. Most member countries’ trade of all full-time workers in the U.S. These and related trends made it more has been with other OECD nations. earned less than two-thirds of median difficult for economies to adapt. Their exports to other countries have earnings, compared with one-tenth in Australia and one-fifth in the EC. In The challenge of adjustment was tended to have a relatively high skill made stronger by continuing rapid content in contrast to their imports, 1990 almost one-fifth of all full-time technological change, particularly in which are most frequently labor-inten- U.S. workers had earnings at or below the official poverty level. This is in the information technology, and by the in- sive raw material and manufacturing ternationalization of economic activity. products. context of a general movement of job Just at this juncture, when more flexi- growth from unskilled to more highly The largest OECD countries, with skilled jobs. ble economies were sorely needed, pro- large domestic markets, have spurred tective social policies extending social the greatest amount of high-technology Implications for Employment benefits were introduced. The assump- manufacturing, but high-tech produc- Policy tion is that these protective policies tion has become a viable option for the made these economies more rigid. smaller nations as well. For example, The analysts preparing the report on the Jobs Study view the unemploy- Within this general history, the U.S. Ireland, Australia, Finland and Norway experience was somewhat novel. Pro- have been quite successful in the high- ment scenario with concern. Further- tective labor market and social policies tech area, in addition to Japan. more they feel that the debate over its causes has been seriously flawed. A were less extensive, labor markets more In thirteen OECD nations, non-farm number of myths have been reinforced flexible, and job growth was occurring self-employment has grown faster than — such as that technology increases mainly in the private sector. Many overall job growth, particularly in the unemployment, that imports from low- jobs were high skill and high wage. UK. Part-time employment has in- wage countries depress wages and pro- Many were low skill and low pay. creased as a share of total employment duce unemployment, and that the inten- Workers without skills had no option in all OECD countries, with women ac- sity of global competition is to blame. but to accept the latter because of the counting for most of the increase. Tem- The evidence developed over the past low level of social support. On the oth- porary employment has not appeared four decades would suggest, they say, er hand, these workers might have been to increase except in France and Spain that none of these factors is the primary worse off with a more rigid economy where there has been a deregulation of cause of rising unemployment. The that precluded any job at all. work contracts. Two-thirds of the job problem is that each causal theory Both the widespread unemployment growth in the EC and EFTA since the leads to a different policy conclusion, in other OECD countries, and the di- 1970s has taken place in the public sec- whether the theory is warranted or not. chotomy in the kinds of jobs available tor in contrast to the U.S. and Japan Again, these analysts look for the in the U.S., were thought to stem from whose increase in jobs has occurred the failure to adapt to the trends emerg- mainly in the private sector. However, main cause in the gap between the need ing over previous decades. Managerial public sector employment growth for these countries to adjust to global skills, the organization of workplaces, slowed in Europe in the 1980s, and the changes affecting economic systems,

Evaluation Forum n Issue 11 n Summer 1995 69 Evaluation Issues and Activities n Economic Reform in the European Union

general productivity and workforce Macroeconomic Policies The proof of the effectiveness of preparation had not kept up with a The report proposes that the macro- these reforms for workers, the analysts more technologically advanced econo- say, is in the creation of more new jobs. economic environment influences an my. While a lack of adjustment meant economy’s ability to create viable jobs Where Job Creation Is Most more low paying jobs in the U.S., it in response to change. Certain ingredi- meant greater unemployment in the EC. Likely to Occur ents should be part of macroeconomic The development of a low-paying job policy, the analysts suggest, if that en- The following predictions and rec- sector in EC countries had been pre- vironment is to have the right effect: ommendations are made in the report: vented by practices such as state-im- posed or union-negotiated wage/income n Sufficiently high levels of national n New jobs must be generated by the floors and employment protections. saving and investment: Since the private sector, given government mid-1970s, national savings and in- budget deficits and resistance to tax Preserving Social Objectives vestment as a share of national in- increases in member nations. Although the main policy message in come have declined significantly in n Many new jobs will have increasing- the report is that the solution to high OECD countries, particularly in Eu- ly high knowledge requirements, unemployment is the restoration of rope. particularly those involving tradable economies’ and societies’ capacity to n goods, related to the need to im- adapt to change, the report’s sub-theme Price stability and low rates of in- is that this restoration must not come at flation: Employers need to receive prove productivity in innovative the expense of an abandonment of so- “undistorted price signals from the businesses that have the ability to cial goals. To satisfy such restoration markets” if there are to be effective use technology effectively. while preserving protectionist values structural adjustments. n Many new jobs will likely be low- has been a formidable task. n Budgetary efficiency and high-quali- productivity, low-wage jobs, which The analysts propose that the key is ty public sector spending and taxa- may help absorb the significant to reform social policies such that they tion: Boosting economic growth at numbers of low-skilled unemployed. no longer have negative economic side- the cost of higher inflation should be The report cautions, however, that effects. They do not see such reform avoided, the analysts point out, since tax concessions and/or employment impossible to achieve, if policy experts the subsequent need to tighten could subsidies supporting the creation of will 1) study the spectrum of social pol- icies implemented over the past three result in lower average growth over low-wage jobs should be avoided, decades, 2) assess to what extent they a number of years. while policies that inhibit low-wage job creation should also be resisted may have crippled economies’ adjust- The report recommends an integra- ment capabilities, and 3) analyze what tion of structural and macroeconomic due to the high unemployment rates. strategies may be feasible for removing policies, since structural reforms, it Clearly these predictions and sug- such disincentives without dismantling states, will not work well in an unstable gestions involve a fine balancing act desirable social protections. The re- macroeconomic environment. On the between two important concerns: 1) the search and analysis already completed other hand, macroeconomic policy is general undesirability of deliberately should, they say, provide a useful more likely to be effective if structural creating low-wage jobs, and 2) the de- framework for such reform. This frame- policies prevent inflexibility and allow sirability of creating jobs for unem- work is summarized in the second vol- resources to be used most profitably. ployed workers lacking the skills re- ume of the Jobs Study. Structural change in good economic quired by high-productivity, high-wage times is less painful, but the need for firms. Some Conclusions such change is not as evident and is The OECD report focuses on cycli- therefore more difficult to rationalize. Technological Change But when economic demand is weak, cal as well as structural unemployment The report claims that technology, structural reforms can involve worker in drawing conclusions about how historically, has generated more worker member nations should approach their dislocation with little job creation else- earnings than displaced workers. Ja- where to absorb those workers, and this major task. Therefore their ultimate rec- pan’s shift to high-tech industries, for effect can reduce political support for ommendations involve the macroeco- example, led to a 4% increase in manu- nomic environment as well as job cre- change. The report therefore recom- facturing employment in the 1970s and mends the simultaneous implementa- ation and workforce preparation — that 1980s. The EC experienced a 20% de- tion of structural and macroeconomic is, they consider that both macroeco- crease in manufacturing employment, nomic and structural policies are need- reforms. ed.

70 Evaluation Forum n Issue 11 n Summer 1995 Evaluation Issues and Activities n Economic Reform in the European Union

associated with an emphasis on lower- way that favors the hiring of low-wage, ment (example: unemployment insur- wage, lower-technology industries. unskilled labor, such as eliminating ance), but they have not shifted much ceilings on employer contribution rates of their resources to active strategies Critical to the positive relationship and reducing low-skilled workers’ in- (example: public employment servic- between technological change and job creation is effective absorption, appli- come taxes. Widening wage differen- es). The report supports a reconsidera- tials is another option, since there is ev- tion of resource use. cation and diffusion of new technolo- idence this speeds employment growth. gies, such as current strides in informa- However, on the basis of research tion technology. Government subsidies However, the analysts recognize that evidence, the report is critical of train- none of these strategies are simple to to firms tend to discourage this kind of ing programs intended to serve large, implement. innovation. Pooling resources to pro- diverse groups of unemployed — i.e. mote basic research and the application For instance, the down side of wid- programs that are not carefully target- of new technologies, particularly those ening wage differentials is that if these ed. It also questions job creation strate- with the higher risks, is recommended become too great, many with low in- gies that involve subsidized hires. In- as an alternative strategy. comes will fall into poverty. In Europe stead it recommends the involvement and Scandinavia, widening differentials of all stakeholders in developing more Small Business are generally unacceptable in terms of effective methods for reducing unem- Entrepreneurship the commitment to equity. Neverthe- ployment — employers, unions, educa- The report encourages support for less, the report stresses the need to re- tional institutions, and local govern- “dynamic entrepreneurship” in the visit the issue of safety net protections, ments. giving more emphasis to “the market- form of removing regulatory barriers to In terms of education and training, clearing role of wages,” and to consider new and expanding private sector busi- the report again looks to the research nesses. Currently government resourc- new ways to express equity values. evidence: es, such as protective subsidies, are tar- Many European countries have n Preschool and early childhood de- geted to large firms. Small business strong employment protection legisla- development requires support for the tion, which can reduce hiring and lead velopment programs are important following, according to the OECD ana- to the use of temporary work contracts. strategies for providing a foundation lysts: workforce preparation; adequate Long-term work contracts encourage conducive to the pursuit of life-long physical infrastructure; modern tele- investment in training and skills up- learning opportunities. communications; networks of small grading, which then increase hiring. n Curricula and teaching methods firms; and improved access to and use But finding the right balance between need to be individualized to meet the of sources of managerial advice, tech- worker protections and hiring has prov- learning needs of underachievers nological expertise and research. Gov- en to be very difficult. ernments should assist small businesses and potential school dropouts, if The analysts suggest the following in their development, they suggest, they are to acquire workforce skills. as important ideas for consideration in through seed, venture and equity capi- n A better balance is needed between tal, and the financing of debt. increasing hiring: post-secondary education, and tech- n The development of rules to prevent nical and advanced occupational The Role of the Social major firms from devising barriers training. Safety Net to entry. n New forms of apprenticeship and The OECD analysts reassert that n The establishment of rules that pro- some of the policies intended to pre- work-based learning options are hibit cartels, while allowing desir- serve equity have had negative side-ef- needed as jobs demand multi-skilled able forms of cooperation among fects for economies. An example is dis- workers with general competencies. incentives to hiring. The non-wage firms. n On-the-job, employer-sponsored labor costs borne by employers, such as n International cooperation in moni- training should be increased through social security contributions and taxes, toring the behavior of firms in- innovations that will encourage em- represent a large proportion of the cost volved in international competition. of labor, particularly in Europe. Gov- ployer investment in training. ernments are becoming serious now Strategies Benefiting Workers n More effective strategies need to be about reducing social spending as a and Their Families developed to prevent families from way to reduce labor costs and encour- age hiring. The report claims that the typical falling into poverty due to the ten- OECD country spends 2-3% of its GNP dency for high unemployment bene- Shifting the tax base from payroll to on labor market strategies such as pas- fits to decrease workers’ incentives other taxes is one alternative. The re- sive methods for reducing unemploy- for increasing their hours of work or port suggests cutting social contribu- their investment in training. tions by employers and employees in a

Evaluation Forum n Issue 11 n Summer 1995 71 Evaluation Issues and Activities n Economic Reform in the European Union

nn The OECD analysts propose that an ciety that have never before faced a integration of tax and benefit systems high risk of unemployment, such as The Persistence of may work best, achieved through a white collar workers, are losing jobs, Unemployment in negative income tax or an income tax with all the personal and societal credit strategy. However they acknowl- costs that implies in terms of lost po- the Postindustrial edge that while this approach may re- tentialand lost investment. Era duce “the poverty trap,” it may not The broad goals of the recommenda- The Economics Department of the raise worker incentives. tions flowing from the Jobs Study are Organization for Economic Coopera- these: to enhance the ability of econo- Overall Policy Recommendations tion and Development (OECD) in Paris mies to adjust and adapt, and to in- publishes a semi-annual journal, OECD The report explains the need and ba- crease their capacity to innovate. Their Economic Studies. In this article we re- sis for the recommendations advanced. major policy suggestions, excerpted view two articles in the Winter 1993 is- from the OECD report are presented in sue. The first is by Jorgen Elmeskov More than ever since World War II, Figure 1. today’s unemployment is causing and Maitland MacFarlan, “Unemploy- damage in ways that cannot be mea- ment Persistence.” The second is “In- sured by the sheer numbers. High un- Editor’s Note terpreting Unemployment: The Role of employment creates insecurity and Labour-Force Participation” by Elm- resistance to organisational and tech- For readers interested in keeping in- eskov and Karl Pichelman. nical change. Long-term unemploy- formed about employment and related ment lowers self-esteem, is demoti- issues in the OECD countries, the ex- Elmeskov and MacFarlan vating and self-reinforcing, and is tensive OECD Catalogue of Publica- The most worriesome characteristic tions can be obtained at no cost from associated with health problems. The the OECD Publications and Information of unemployment in most OECD coun- rise in youth unemployment means Center at 2001 L St. N.W., Suite 700, tries in the 1990s, the authors propose, that many young people are losing Washington, D.C. is its tendency to continue close to the skills or employability. Groups in so- level experienced during the worst eco- nomic times even in good economic times. This has been the outstanding at- Figure 1 n Major Policy Suggestions in the OECD Report tribute of the four unemployment cy- cles since 1966. The European Union n Set macroeconomic policy such that it will both encourage growth . . . and (EU) rate, for example, rose from 6% make it sustainable. to 10% during the economic declines of n Enhance the creation and diffusion of technological know-how by improving the early 1980s, and did not go below frameworks for its development. 8% in 1990. The authors seek explana- tions for this trend. n Increase flexibility of working-time . . . voluntarily sought by workers and em- ployers. Studying the research evidence, two n Nurture an entrepreneurial climate by eliminating impediments to . . . the cre- alternative explanations emerge. Persis- ation and expansion of enterprises. tent unemployment appears to be relat- ed to 1) changes in the deeper structur- n Make wage and labor costs more flexible by removing restrictions that prevent al influences that determine the demand wages from reflecting local conditions and individual skill levels. . . . for and the supply of labor — changes n Reform employment security provisions that inhibit the expansion of employ- that have increased cyclical unemploy- ment in the private sector. ment rates, and 2) the tendency for high n Strengthen the emphasis on active labor market policies and reinforce their cyclical unemployment to create struc- effectiveness. tural unemployment. n Improve labor force skills and competences through wide-ranging changes in The authors’ thesis is that the tradi- education and training systems. tional distinction between cyclical and n Reform unemployment and related benefit systems . . . such that societies’ structural unemployment is dimming in fundamental equity goals are achieved in ways that impinge far less on the ef- the context of two trends: the tendency ficient functioning of labor markets. for unemployment related to cyclical unemployment forces to develop into longer-term structural unemployment, and the tendency for the economy’s structural characteristics to influence the cyclical aspects of economic activi- ty. Indicators used to describe unem-

72 Evaluation Forum n Issue 11 n Summer 1995 Evaluation Issues and Activities n Economic Reform in the European Union

ployment rate trends suggest that the Figure 1 n Theories and Policy Alternatives unemployment rate required to stabilize wage inflation has risen, that vacant Theories Explaining Policy Alternatives jobs are filled with greater difficulty Persistent Unemploy- than before, and that the use of physical ment Microeconomic Options Macroeconomic Options production capacity is reaching a threshold leading to a mismatch be- n Rising natural rate of n Reform directed to n Policies seeking to ex- tween jobs and available sources of la- unemployment reversing or offset- pand the economy. ting the factors caus- n bor. Overall, OECD structural unem- Slow adjustment to ing the rise in and n Activist policies. economic change ployment rose steadily throughout the persistence of unem- 1970s and early 1980s, declining some- ployment. what in the early 1990s. The authors claim, however, that this general pat- n Reduction of employ- ment protections. tern veils important differences across OECD countries and regions. n Reform directed to improving the region- To study basic causes and the rea- al and occupational sons for differences across OECD mobility of the unem- countries, the authors utilize a particu- ployed. lar analytical framework to explore the phenomenon of rising unemployment. The framework gives attention to three On the other hand, reducing employ- employment and training professionals. key relationships: the demand for la- ment protections may be an efficient It focuses on the relationship between bor; the nature of wage-setting; and the option for speeding adjustment to labor force participation and reported labor supply. change but may do little to reduce cy- unemployment. The point the authors These relationships are considered clical unemployment. Expansionary wish to impress on readers is that the macroeconomic policies may have no measurement of unemployment lacks a to affect the important interrelationship influence on persistent unemployment clear-cut distinction between unem- between cyclical and structural unem- ployment. Applying a statistical model, due to cyclical factors. Slow adjust- ployment and nonparticipation in the ment may have only a minimal effect labor force. This, they say, continually the authors empirically test some of the on the natural rate of unemployment. leads to policy confusion. major theories explaining persistent un- employment in the OECD countries. At Activist policies that attempt to fine- The relationship in question is likely tune the economy may be counterpro- either end of the continuum of potential to change, the authors suggest, on the ductive if they lead to a neglect of causes are 1) a rise in natural (cyclical) basis of a number of influences: the de- unemployment, and 2) slow adjustment structural forces. Automatic stabilizers mographic composition of the work- would seem to be important in prevent- to economic change (a structural force, the extent of coverage and the ing sharp economic ups and downs, but cause). The analysts come to a rather generousness of income support sys- nebulous conclusion that neither theory these may also cause a rise in cyclical tems, and other incentives affecting the unemployment which then destabilizes is fully correct! This conclusion actual- decision to join or leave the labor force. the economy. ly has important implications for eco- Low unemployment could mean that an nomic policy, such as those suggested The message is that there are consid- excess supply of labor is being moved in Figure 1. erable policy risks in misdiagnosing the toward nonparticipation, which masks The authors make it clear that the causes of persistent high unemploy- the true extent of unemployment. ment, and in relying only on one strate- problem of persistent unemployment is gy. The authors suspect that placing Trends in Labor Force quite complex, since there are likely Participation multiple causes. Long periods of cycli- major policy emphasis on structural re- forms may be the most effective deci- cal unemployment can dissove into Participation rates have continually sion — particularly reforms that im- structural unemployment. At the same risen since 1970 in all OECD countries time, a common set of policies may be prove the skills of the unemployed and due almost entirely to the increase in offer them incentives to re-enter the la- effective, they say, in reducing both cy- female participation. The authors credit bor force. clical and structural problems. For ex- the gradual shift in cultural and social norms and attitudes with this change, ample, structural alternatives may be Emeskov and Pichelman effective in reducing both the natural but the change has also been supported rate of unemployment and slow adjust- This article on the role of labor force by the increasing availability of public- ment to change — such as policies sup- participation in persistent unemploy- ly-financed day care institutions in porting worker mobility. ment is more understandable by non- many OECD countries. The authors economists, and likely more useful to feel the reform of tax systems, particu-

Evaluation Forum n Issue 11 n Summer 1995 73 Evaluation Issues and Activities n Economic Reform in the European Union

larly reforms that made the individual ness cycles, the authors find that unem- siveness, or the extent to which the rather than the family the basic tax unit, ployment fluctuations — i.e. variability effect of fluctuations in cyclical em- may have been another factor. Howev- in unemployment — are greater in ployment on unemployment are er, long-term trends in participation North America, the UK, Spain and Ire- softened by what the authors define rates have varied across geographical land than in Japan, Austria, Switzerland as “procyclical variations” in the la- areas. For example, male participation and some of the Scandinavian coun- bor force. rates have declined over the past twenty tries. The authors attribute differences years in almost all European countries. in unemployment variability to interac- Conclusions from the analysis: Elas- In North America and Japan they have tion between 1) employment and 2) un- ticity varies widely across countries. been stable. employment and nonparticipation. France has almost no short-run labor An analysis of the OECD data used Acknowledging data inadequacies, force response to cyclical swings in in studying the relationship between they take a look at cyclical fluctuations employment, whereas in Japan these participation and unemployment sug- in output (“volatility”), which they pro- ups and downs have a significant ef- gests that the “hidden component” of pose may lead in different ways to dif- fect. unemployment may increase as mea- ferent degrees of variation in employ- 2. The “elasticity” of labor force partic- sured (“open”) unemployment increas- ment across countries, and may es. There is little evidence in the data ultimately generate differing unem- ipation rates. that countries with low levels or in- ployment conditions. They find that Conclusions from the analysis: In all creases in measured unemployment ex- countries respond differentially to “de- OECD countries, other than Germa- hibit these conditions at the expense of mand shocks, the degree of exposure to ny and France, total participation lower levels of participation, and dif- autonomous supply shocks, the sectoral rates are significantly affected by ferences in measured and hidden unem- composition of output, and the pres- employment rates. And the elasticity ployment across OECD countries may, ence of automatic stabilizers.” of participation rates with respect to the authors surmise, be larger than the A traditional explanation for the rel- employment opportunities is higher data reveal. ative stability of employment, the au- for females than for males, and The authors view data on “discour- thors conclude, is “labour hoarding,” or higher for youth and older workers. aged workers” as an important resource the tendency in economic downturns This is consistent with the fact that in understanding the dynamics. This for firms to reduce hours of work be- the larger proportion of workforces group represents a subset of a larger fore laying off workers. Another vari- in OECD countries are women, and group of people who have stopped able is employment protection legisla- searching actively for employment be- tion, and incentives embedded in they are overrepresented in the dis- cause they have concluded that no via- unemployment benefit systems. These located worker population. ble jobs exist. Although “discouraged influence economic adjustment costs worker” definitions vary across mem- and as a result affect employment flexi- Unemployment Utilized as a ber countries, making an analysis of the bility. Variations in self-employment Socio-Economic Measure data difficult, over time these data sup- are also an influence. The authors point The forces affecting and explaining port the authors’ conclusion from their out that as the volatility of self-employ- cyclical fluctuations in unemployment own analyses, namely that official un- ment drops below that of wage and sal- offer important insights to those inter- employment and discouraged worker ary employment, a generally high pro- ested in better social and economic in- rates are positively correlated — with portion of self-employment (within dicators. The authors define three ma- the exception of Japan. The long-run overall employment) tends to reduce jor roles for unemployment in this relationship between unemployment the latter’s cyclical variability. This is respect: and labor force participation seems to an important finding since self-employ- n Unemployment as a measure of un- be negative. Again, however, there are ment is growing faster in OECD coun- different tradeoffs between these phe- tries than overall non-agricultural em- der-utilization of labor. nomena for different countries. Coun- ployment. n Unemployment as an indicator of tries with low unemployment, such as social hardship. Japan, tend to have high participation Labor Force Sensitivity n Unemployment as a determinant of rates. Countries with high unemploy- The analysts also look at the sensi- ment, such as Spain, tend to have low tivity of the labor force to cyclical la- wage pressure. participation rates. bor market conditions. In doing so they A number of implications identified study two phenomena: Fluctuations in Unemployment by the authors are noteworthy. If ex- 1. The “elasticity” of labor force “trend panding output produces a virile em- Using changes in the average annual ployment response at the same time the deviations” in employment — that unemployment rate as a measure of labor force stays the same, unemploy- is, a measure of labor force respon- fluctations in unemployment over busi- ment tends to fall strongly with rising

74 Evaluation Forum n Issue 11 n Summer 1995 Evaluation Issues and Activities n Economic Reform in the European Union

input — but potential output increases Overall Conclusions Editor’s Note as a result of increasing actual output, The authors comment on the evi- if rising productivity leads to rising em- Emeskov is in the Economic Pros- dence produced in analyzing data and ployment which in turn increases the pects Division of the OECD Economics on the unsolved mysteries that remain labor force. This is why countries with Department. Pichelman is in the Insti- (see Figure 2). Two explanations for tute for Advanced Studies in Vienna, a responsive labor force and productiv- the mysteries are possible, according to Austria. For other publications of inter- ity can expand longer when coming out the OECD analysts: est, please see: of an economic down cycle prior to in- flationary pressures building sufficient- 1. The relationship described above, Barro, R. “The Persistence of Unem- ployment,” in American Economic ly to require corrective measures. In between labor force responsiveness Review. Papers and proceedings, this sense, “cyclical fluctuations in pro- and unemployment may occur if a 1988. ductivity and cyclical labour-force vari- country has experienced predomi- Bertola, G. “Job Security, Employment ations — and not measured unemploy- nantly negative shocks over recent ment — are relevant in gauging the full and Wages,” in European Economic decades. Review, 34, 1990. extent of under-utilisation of labour and the associated output loss during a 2. The persistence of unemployment Clark, K. and L. Summers. “Labor Mar- slowdown.” may be greater following an upward ket Dynamics and Unemployment: A shock to the unemployment rate Reconsideration,” in Brookings Pa- Using unemployment as a measure pers on Economic Activity, 1, 1979. of social hardship may fall victim to the than following a downward shock. Elmeskov, J. “High and Persistent Un- same malady. The authors claim that However, the degree of labor force employment: Assessment of the changes in measured unemployment responsiveness does not appear related Problem and Its Causes,” in OECD underrepresent changes in gainful em- to trends in or levels of labor force par- Department of Economics Working ployment and earned income. Further- ticipation. Participation rates tend to Papers, No. 132, 1993. more, these changes do not reveal the return to their underlying trends while Lindbeck, A. “Macroeconomic Theory amount of economic hardship caused unemployment rates do not. It is no and the Labour Market,” in European by cyclical downturns. Unemployment surprise that the analysts conclude that Economic Review, 36, 1992. benefits clearly reduce such hardship, the complex interactions between un- Pederson, P.J. and N. Westergard-Neil- therefore the hardship is greater for la- employment trends and economic cy- bor force non-participants since they son. “Unemployment: A Review of cles will require further research! the Evidence from Panel Data,” in are not officially employed. In coun- OECD Economics Studies, No. 20, tries where unemployment benefits are 1993. reduced substantially as a spell of un- employment elongates, the labor force tends to respond more to changes in Figure 2 n Evidence and Remaining Mysteries employment. Evidence Remaining Mysteries As an indicator of wage pressure, it is the authors’ interpretation of their Cross-country differences are typical Countries with volatile unemployment data analysis that unemployment rates for trends and cycles in unemploy- rates tend to have larger increases in ment and labor force participation unemployment over long periods are more useful in explaining wage in- across OECD countries, but these dif- than countries with unemployment creases than labor force participation ferences cannot be attributed to dif- that is more cyclically stable. Since rates. This may be due to the fact that ferent participation rates. Unemploy- labor force responsiveness is a major different groups of workers react dif- ment rates are only a partial reflection influence on unemployment volatility, ferently to changes in employment. of cross-country differences in trends there is a relationship between re- Workers in their prime ages are as- and levels of employment. sponsiveness and the increase in sumed to influence wage determination and level of unemployment. This more than others. Also, workers with does not seem consistent with the longer spells of unemployment influ- fact that unemployment tends to re- ences wages less than others. turn quickly to a stable natural rate of unemployment following an econom- ic shock.

Evaluation Forum n Issue 11 n Summer 1995 75 Evaluation Issues and Activities n Economic Reform in the European Union

nn Labor Issues tors in national labor markets,” or much progress in integrating markets will competition among these na- for services, capital or labor. In the in European tional systems simply increase and 1970s, the nations of Europe were pre- become divisive? occupied with domestic problems and Consolidation integration efforts slowed. High inter- 2. Changes in the location and com- This short article briefly summarizes nal unemployment and low economic position of production, and their a 1993 book published by The Brook- productivity consumed their intellectu- ings Institution, Labor and An Integrat- effect on employment. al and political energies. But the reality ed Europe. Edited by three Brookings Questions: Will the integration pro- of rising unemployment and continuing economists, Lloyd Ulman, Barry cess alter the traditional segmenta- lackluster productivity renewed Euro- Eichengreen, and William T. Dickens, tion of European internal labor mar- pean interest in the reorganization of the book provides an important frame- kets by reorganizing labor market resources and labor forces, under the assumption that excessive intra-nation work for understanding the evolution sectors and regions across the EU, regulation was largely to blame. of thinking about a single market for the purpose being to permit firms the European Union, the changes pro- anywhere within the EU that spe- The editors report that the 1987 Sin- posed to reach such a goal, and the cialize in particular products and gle European Act was expected to es- probable effects of this consolidation services to locate together to utilize tablish a common market for products, for Europe and other countries. We capital and labor by 1992. Physical and concentrations of resources and pur- confine our attention to the first chap- technical restrictions on international ter, by the editors, on some of the im- sue economies of scale and scope — economic activities were finally to end. plications of consolidation for the la- or will there be resistance to the re- The Act required European Community bor force and labor market. location required in developing a members to 1) remove trade barriers, comprehensive overhaul of econom- and 2) eliminate domestic policies that Some Useful History ic activities? Will European work- seemed to inhibit intra-EC competition The Maastricht Treaty of 1991 was ers, traditionally resistant to move- — that is, to develop a truly integrated to remake the European economy. Eu- ment even to new regions within market for capital and labor as well as ropean currencies were to be replaced their own countries, be mobile for products and services. However, with the “ecu,” a common currency for enough to supply these new points there were serious questions about the all European Union (EU) countries. A of resource consolidation and eco- potential for some countries within the EC to erode others’ competitive edge, European central bank was to take pre- nomic activity with the labor needed or to seek out cheap labor in another cedence over the existing banks of for such reorganization? member nations. Free trade and free EC nation leading to more unemploy- movement of workers across national 3. Potential intensification of eco- ment. Consequently in 1989 a Social boundaries within the EU were to her- nomic competition between the Charter was established which guaran- ald a new era. Of the changes to be EU and other countries. teed minimum standards of compensa- achieved, by far the greatest, according Question: Given the prediction by tion and welfare throughout the EC, to the editors of the Brookings book, experts that European integration through mechanisms such as EC-wide labor and management institutions. was the transformation of the European will increase the productivity of labor market. Workers were to be free member nations and their overall Unions to seek and accept employment in any competitiveness, will the EU pene- part of the EU. trate outside markets previously Analyses indicate, the editors say, that market integration in the EC The ambitious intentions in the dominated by others and limit the spawns both centralizing tendencies Treaty represented enormous challeng- access of other countries to the EC’s es for governments, employers and and decentralization, as has been the market, with negative consequences case in the U.S. In the U.S. the creation workers. The editors point to these for the U.S. and other nations, par- three as some of the most difficult to of an integrated market across states in- ticularly Eastern and Central Europe meet: creased centralized collective bargain- and the former Soviet Union? ing as unions advocated for more uni- 1. Competition among differing na- form wages and labor costs. Later there tional systems of workforce prepa- The Evolution of European has been a decentralization movement ration and social welfare. Integration in which unions have lost membership. The EU is not likely to experience as Question: Will the integration pro- The 1957 Treaty of Rome moved a cess result in uniform systems of in- large number of European nations in strong a decentralization tendency, the editors suggest, because of the tradi- dustrial and business relationships the direction of freeing up import du- tional strength of European “employ- and social welfare, through negotia- ties and quota restrictions on interna- tions among “major institutional ac- tional trade. But this did not translate to ers’ associations, mandatory works

76 Evaluation Forum n Issue 11 n Summer 1995 Evaluation Issues and Activities n Economic Reform in the European Union

nn council elections, effective deterrents to in the Brookings volume is whether strike breaking, the representation of pension and health benefits reduce Political Economy of unions in public administration and on needed worker mobility. In the U.S., “Borrowing” labor tribunals, and the legal restriction workers with health problems find it of certain benefits to union members.” difficult to change jobs because of the Issue #9 of Evaluation Forum fea- potential loss of health coverage. In tured a series of articles on the new em- Worker Participation in Europe, health coverage mandated by ployment and training system adopted Management national governments remove this bar- by the United Kingdom (UK) under rier to the movement of labor. The re- Prime Minister Thatcher — the Train- The Social Charter proposed that ing and Employment Council (TEC) workers be infused into management search seems to confirm that national system. This new initiative privatized decisions. This action was thought to social security mandates appear to in- crease mobility. So if this is the case, the government’s effort to provide edu- strengthen employer associations and cational, occupational training and em- unions if it were initiated by employ- the lower mobility in the EU must, the ployment services to workers consid- ers. However, it was acknowledged that experts suppose, be due to other fac- tors, such as historical and cultural dif- ered in need of a range of assistance. increased worker participation some- The series of articles included an ex- times eroded union power if initiated in ferences. planation of the elaborate evaluation the context of weak unions. Both em- According to the book’s editors, re- system designed to study the imple- ployers and unions have problems with moving barriers to labor mobility will mentation and performance of the TEC this component of the Charter. German, require a common policy regarding system. Belgian, Dutch and Danish workers, for workers migrating into the EU from instance, have supported employee in- other countries. There will be strong This article analyzes some of the un- formation, consultation and participa- pressure, they say, to negotiate an EU- derlying features of the U.S. and U.K. tion because there is a common em- wide compromise between “rotation approach to investment in employment ployer-employee commitment to the policies [Germany, as an example] and and training. It is based on a Brookings application of high standards to low- permanent immigration restrictions [as Institution book published in 1993 and standard EU countries. On the other for example, in Sweden].” They con- edited by David Finegold, Laurel Mc- hand, French labor has not supported clude that the rotation approach is most Farland and William Richardson, this part of the Charter. feasible, inasmuch as an immigration Something Borrowed, Something policy that is too restrictive could in- Learned: The Transatlantic Market in Some experts writing in the Brook- Education and Training Reform. Fine- ings book recommend industrial rela- crease mismatches between skills and gold is a political scientist and educa- tions systems which do not restrict or job, leading to more unemployment. tion specialist who was involved in the unduly limit worker participation and The Effects of European development of the TEC system and its labor-management cooperation. There Integration on Other Countries evaluation strategy as a faculty member is some agreement that centralizing of Warwick University in the UK, and tendencies will ultimately dominate Although the book’s experts do not is currently at the Rand Corporation in over “national differences in industrial see a significant negative economic ef- California. McFarland is an economist relations traditions and institutions, in fect for the U.S. in the integration of who was at the Brookings Institution union organizational strength, and in European economic activity, they are when the book was written. Richardson union attitudes concerning the desir- concerned that emerging Eastern Euro- is a senior research fellow at the Centre ability of uniform EU-wide labor stan- pean nations could suffer considerably for Education and Industry at the Uni- dards. from restrictive EU immigration poli- versity of Warwick. cies and a lack of access to Western Other analysts in the book make the European export markets. If economic important point that a significant condi- The Theme of the Book tion for successful economic competi- restructuring and dislocation concen- trates in regional pockets of high unem- The editors reveal their interesting tion is “long-term employment rela- ployment, this could lead the EU to anthropological theme in the introduc- tionships that encourage investments in tion to the book. They begin with the training skilled and versatile workers.” adopt protective measures that would be damaging to Eastern Europe and the premise that education and training They look to the German industrial re- former Soviet Union. The latter, the an- policy in the US and Great Britain con- lations system as a model for a unified verged in the late 1980s, both countries Europe that is economically viable. alysts say, would be counterproductive for Western Europe in the long run. becoming more dependent on each oth- er’s ideas. They claim that extensive Worker Mobility and Protection borrowing, reshaping and application A major question about EU integra- of each other’s employment and train- tion in the minds of the analysts writing ing ideas took place in the 1980s, high- lighting the influence of JTPA on the British system. Their analysis of the reasons for this borrowing and its im- plications for policy is tightly packed, a

Evaluation Forum n Issue 11 n Summer 1995 77 Evaluation Issues and Activities n Economic Reform in the European Union

well-organized and clearly-written sort- quantity of outputs from their employ- the American underclass creates unique ing out of major influences and effects ment and training systems, was a strong problems. The editors view the stronger which ties their conclusions closely to energizer for change. At the same time, role permitted the central government expert opinion and empirical evidence. pressures for educational change were in the UK in reforming education, The economic context of the analy- endemic throughout Europe — even in training and employment systems as the Japan and South Korea — reinforcing most significant difference between the sis is two-fold. Their major emphasis is these change efforts. two countries. on general economic change in the postindustrial period. Such change, Common Characteristics, It is such differences that complicate they say, has been the main influence But Differences as Well the transfer of policy between the two on contemporary employment and countries, the editors say, and may ex- training reform in all advanced indus- More generally, the editors see com- plain some of the unintended conse- trial nations. They point to the integra- mon economic forces driving employ- quences of borrowing. The greater the tion of world markets, to technological ment and training reform in all indus- differences, they contend, the more change, and to increasing anxiety about trialized nations over the 1980s. Rapid general and abstract the policy transfers sustained economic progress as charac- technological change and the interna- should be, if they are to be useful and teristics which differentiate this period tionalization of capital generated more productive. from the continuing economic growth intense political scrutiny of employ- and relative stability that defined the ment and training systems — the ade- Transplantation quacy of the skills they produced, and period between World War II and the The editors’ main fascination is with their overall efficiency. Under such oil crisis of 1973, and constituted a the effect of differing social, economic stimulus for reform. But they do not ig- scrutiny, the US and the UK displayed and political contexts on transplanted common characteristics: declining com- nore important US borrowing from policies — that is, whether particular petitiveness in world markets and low Western Europe. Their conclusion is policies and programs can work in iso- that such transnational borrowing will skill demands by employers; a lack of lation from the system that created cooperative strategies and institutions likely increase in the future, as the them, much like an organ transplant, for sharing the knowledge and cost re- world’s academic and political commu- and how transplantation may alter the nities become increasingly internation- quired to develop and implement high- original idea. A case in point is the UK skills programs; and high drop-out alized due to growing economic com- borrowing of the JTPA Private Industry rates in secondary education and low petition and greater interdependence Council (PIC) concept, making greater among industrialized economies. participation in postsecondary educa- demands on the British employer com- tion and training. The second significant focus of the munity than JTPA makes on American book is on differences between US and The two countries also share a com- employers, even though the research British reforms, and those of these mon view of government’s role in solv- evidence has suggested that the effec- countries’ main international competi- ing such problems, the editors suggest, tiveness of PICs was questionable. The tors. In this respect, the editors are in- such as the emphasis on voluntarism Training and Enterprise Councils terested in the “shared context of rela- (allowing the market to stimulate the (TECs) are now encountering problems tive economic decline” — that is, the provision of training) and weak corpo- similar to those faced by PICs, such as structural features of the two econo- rate-government arrangements. Added the maintenance of employer leadership mies that have discouraged the adop- to all these similarities are historical at the top levels of management, and tion of high-skill, high-performance linquistic, cultural and governmental the development of performance indi- strategies, and the comparatively low commonalities that were strengthened cators. It is not yet known whether the levels of education and training provid- in the 1980s by the political ideologies distribution of central government ed by both. But they see important dif- of Reagan and Thatcher, which stressed training dollars directly to employer-con- ferences between the US and Britain as the need to free market forces from un- trolled TECs will work, given differences well, that tilt the US further than the necessary government control, cut tax- between Great Britain’s and the US’s UK toward high-skill enterprises. es, reduce union influence, and increase cost reduction goals, national income dis- consumer choice in education and tributions, tax systems and approaches to In 1976, Britain’s prime minister training. private sector risk assessment. criticized the UK’s employment and training system for failing to respond These shared attributes are not the The direction of British borrowing is adequately to the needs of industry, and full story, however. In the US, the edi- changing with John Major’s tenure, and launched a series of reforms. The 1983 tors propose, higher education is seen the growing emphasis on European report A Nation At Risk served a similar as an investment most families and in- Union status, the editors observe. The purpose in the US, and was subsequent- dividuals should be willing to make; in policy exchange appears to be giving ly supported by Workforce 2000 reports the UK it is viewed as a right of citi- greater emphasis to EU education and in 1987 and 1990, and by Investing in zenship, therefore the government un- training models, although they feel that People in 1989. Faced with declining derwrites tuition for the smaller group broader, more wide-ranging borrowing affluence, the two countries’ emphasis of students involved. Also, the larger is likely to occur on the part of all in- on skills gaps, and the quality and size and the more complex nature of dustrial nations in the future. Mean-

78 Evaluation Forum n Issue 11 n Summer 1995 Evaluation Issues and Activities n Economic Reform in the European Union

while, the UK/US linkage is too strong cantly different from the international Technological Change to preclude continued sharing of policy competition that characterizes the 1990s. It is the rapid evolution of micro- concepts. The editors say that “British Within the new competitive environment, electronics and information technolo- conservatives remain covetors of the Finegold views the increasing interde- gies that Finegold cites as the major U.S. labor market’s capacity to create pendence of industrial economies as the change, in that this evolution has al- jobs and convince individuals to invest most dramatic change. Less striking but tered what is sold in the new market in their own education and training.” nevertheless influential is the growing and how goods and services are pro- At the same time, the US has become importance of international trade, par- duced. The new technologies, he says, intrigued with Britain’s approach to na- ticularly among the developed coun- allow companies to respond within a tional assessment, school-to-work tran- tries and regarding services. short timeframe to shifts in customer sition programs and experiments with Joint cross-country economic ven- demand. Forcing rapid restructuring, educational vouchers. tures have grown enormously. In the these breakthroughs have reoriented Finegold’s Perspective on the three-year period between 1985 and manufacturing approaches and reorga- Economics of “Borrowing” 1988, 400 alliances between American nized manufacturing processes. They and European corporations have been have also demanded new kinds and lev- There is so much of interest in the created, 150 between European and els of worker skills. As companies have various chapters of this fine book, a Japanese firms, more than 200 between moved in this direction, fewer manual thorough review must of necessity ex- Japan and US companies. Also, the in- workers have been needed, Finegold tend beyond this issue of Evaluation ternationalization of capital has stimu- points out, and higher skills have been Forum. In this issue we focus more nar- lated cross-national high-technology in demand for the workers retained. rowly on David Finegold’s intriguing development. The establishment of the However, he makes it clear that techno- chapter on “The Changing International EU and the signing of NAFTA have logical change does not always lead in Economy and Its Impact on Education supported the integration of world mar- this direction. Managers are free to de- and Training.” kets further. cide whether to economize and upskill, or Finegold’s interest is in the econom- These developments, Finegold says, to increase managerial control and reduce the skill level of their workforces. ic context of the heavy borrowing that have moved the industrial giants away he claims occurred between the UK and from “export-driven, nationally-cen- Economic Restructuring the US in the 1980s. Within that con- tered companies” to a new phase direct- text he addresses three issues: ed by global competition and global The oil crises in 1973 and 1980 pro- duced deep recessions, and increased 1. Economic change as the major influ- corporations. In this new economic en- vironment there are formidable new unemployment and inflation. In addi- ence in stimulating employment and competitors for the world’s capital and tion, firms found themselves awash in a training reform in all advanced in- goods: in particular Japan, but also more uncertain economic environment dustrial nations since the mid-1970s Hong Kong, Singapore, Taiwan and filled with many changes. All of the in- — particularly the integration of South Korea. These new entrants into dustrialized economies felt “compelled world markets, technological change the world market scene, Finegold be- to restructure,” Finegold says. A major and growing anxiety about econom- lieves, are “moving rapidly into higher restructuring objective was the creation ic health. quality, higher value-added market seg- of greater flexibility — for firms, for ments once dominated by the US and workers. This new flexibility was to en- 2. The two countries’ common envi- Western Europe.” To explain the impli- courage more rapid adjustment to eco- ronment of “relative economic de- cations of these changes, Finegold sum- nomic change. It was played out in cline” — with an emphasis on the marizes Robert Reich’s analysis thus: more interest in improving the school- structural characteristics of their to-work transition, in more high-tech economies that have resisted the de- As the major factors of production training, and in greater attention to up- velopment of high-skill initiatives, (capital, technology and raw materi- skilling and career redirection for such as their low level of education als) move more freely and quickly workers dislocated by restructuring and training. across national boundaries, labor, the economies. least mobile factor, becomes a more Early production systems, with their 3. The differences between the two central component of firms’ search large semi-skilled workforces, perform- countries that provide the US with a for competitive advantage. Since the ing a narrow range of work assign- greater capability to create and sus- advanced industrial countries cannot ments on machines confined to those compete with developing countries tain high-skill economic activity. purposes,were being abandoned for on the basis of labor cost, the surviv- more specialized “niche systems” that al of their manufacturing base — and Economic Change are able to rush the development and the vast array of services which sup- production of new goods and services The period between the end of ports it — depends upon superior la- to meet rapidly emerging consumer de- World War II and the 1973 rise in oil bor quality and hence the skills of mands and opportunities. The Japanese prices was one of sustained economic current and future employees. growth and relative stability, signifi-

Evaluation Forum n Issue 11 n Summer 1995 79 Evaluation Issues and Activities n Economic Reform in the European Union

employment and training than do other Many American ET experts have been surprised and confused industrial countries, which tends to dis- to learn that Margaret Thatcher looked to their country for policy courage the development of high-skill ideas. They are surprised because they tend to view the British strategies.This conclusion is not meant education and training system as relatively successful; they are to ignore common factors across na- confused because, in the United States, the ET system is widely tions that affect skill requirements. Among such factors, the author high- considered to be in such a state of crisis that no nation would lights firm size, the nature of the pro- want to emulate it. For observers of the British scene, these duction process, and the type of mar- American problems will seem familiar: significantly lower levels of kets involved. Even those countries attainment in reading, math, and science compared to its main encouraging high-skill strategies evi- international competitors; high drop-out rates in secondary dence aberrations. In Japan, multina- schools, particularly in the inner cities; and growing skills tionals subcontract with firms provid- shortages in specific industrial sectors. Ironically, it is the ing considerably lower wages and less common institutional setting of these problems, rather than any job security. In Germany, a core of record of success, that may explain the affinity between the two highly trained, strongly unionized countries’ policymakers. workers co-exists with a migrant labor force. Nevertheless, Finegold’s interest David Finegold, Laurel McFarland, and William Richardson is in what disincentives exist institu- in Something Borrowed, Something Learned tionally and otherwise in the US and UK that have discouraged the march toward high-skill, high-wage manageri- al practices. These are the major influ- were first, Finegold says, to create such panies hunting high value-added mar- ences Finegold outlines in analyzing “just-in-time production systems” kets, high-skill flexibility strategies comparative data. which used coordinated supply chains provide significant advantages, such as and close monitoring of consumer atti- better product quality, a reduction in n In Britain and the US, the majority tudes and preferences. These yielded downtimes, and the encouragement of of individuals leave compulsory ed- substantial savings on inventories while continuous innovation. ucation with a relatively low level of achieving a high level of market re- Different firms make varying judg- knowledge and skill attainment, par- sponsiveness. However, Finegold qual- ments about the cost/benefit tradeoffs ticularly in math and science — in ifies the movement toward greater flex- of these two general strategies. These fact, they rank at the bottom on the ibility, pointing out that companies in judgments, the author says, are influ- latter. the advanced industrial nations have enced by the interdependence between pursued flexibility in different ways. u In England, there is also a high “corporate/product strategies, compe- dropout rate prior to the comple- Some firms, he says, have focused tence requirements, and the employ- on cutting fixed costs, reducing their ment and training [ET] system.” For tion of compulsory school. Only full-time workforce through the use of example, managers may be less apt to one-third of the age group 16-18 part-time or temporary workers, and ty- invest in their own ET if the message remains in school or college and ing wages more directly to worker per- from the labor market is that companies less than 18% go on to higher ed- formance through profit sharing, bo- are organized in ways that preclude re- ucation. nuses, Employee Assistance Plans, and wards for higher skills. Managers may u other strategies. Other companies have well make choices based on the exist- In the US, the vast majority of emphasized decentralizing control over ing level of ET and on the institutional students remain in school until the work process to frontline workers environment they are in. In Germany, age 18, and approximately one- who are multi-skilled. Relevant to this says Finegold, the high quality ET sys- half of this age cohort continues strategy, Finegold distinguishes be- tem generates innovation because man- into some form of higher educa- tween measures involving multi-skill- agers understand they must organize tion. ing vs. multi-tasking. The former in- the work process such that workers u The US high school diploma in- volves increased worker responsibility continuously have opportunities to and enhanced skills for those absorbing learn and use new skills — the sanction volves no assessment of the new technologies or production process- being that they will be absorbed by the knowledge or skills attained, and es. The latter involves the simultaneous competition. the route into vocational training performance of a number of tasks that are for the non-college-bound is ob- at a comparable skill level. Institutional Impediments to scure. Investment in Employment Low-skill flexibility strategies, he and Training n Students leaving the educational says, reduce costs through dependence system in the US and UK may not on cheaper labor, and through reigning Finegold’s message is that the US close the skills gap in these modern and UK evidence less investment in in the size of the workforce. For com- economies.

80 Evaluation Forum n Issue 11 n Summer 1995 Evaluation Issues and Activities n Economic Reform in the European Union

u One-fourth of British manufac- support a short-term approach. 2. Inadequate Educational Approaches turing companies reported in n The two countries’ top management 1988 that they lacked skilled work- echelons look more to the stock n Both countries provide few incen- ers. Teaching and nursing were ex- market than to the banking system tives to students to invest their time periencing the same deficits. as the primary source of financing, and effort in ET. n US and UK managers do not de- because of security laws. Financial n The U.S. labor market does not re- mand as high a level of skills on the market analysts focus on easily ac- ward high school effort. The UK’s part of the average worker as in oth- cessible company performance elitist examination structure “brands er industrial nations. measures, such as quarterly earnings the vast majority of students as fail- and revenue, which support a short- u Studies find that both managers ures.” term view. American executives and workers are poorly trained. give highest priority to the increase n Employers attach little importance u West German hotels, for exam- in share prices. Japanese managers to education-based vocational quali- ple, had 65% higher labor pro- emphasize market share. fications, and apprenticeships are ductivity than Great Britain, re- limited to a minority of students. In n The sheer number and size of hostile lated to the greater quality and Japan and Germany, all students corporate takeovers in the US and quantity of training of German have incentives available to boost UK in the 1980s further reinforced a frontline staff and supervisors. their work effort in compulsory edu- focus on short-term results. Govern- cation because their performance is n US and UK firms introduce new ment regulations and ownership directly related to their future occu- technologies as ways to reduce costs structures in Japan and Germany pational opportunities. and improve efficiency, rather than preclude such takeovers. to enhance frontline workers’ skills, 3. Relative Ease of Hiring and Firing n The organization of firms into “cost make the workplace more flexible, n and profit centers,” and the reliance US and UK models and laws dis- or position workers to utilize re- on traditional accounting methods, courage a linking of job security to search breakthroughs to enhance discourage measurement of the less the acquisition of skills. The labor products or services. tangible benefits of skills upgrading, market is the main impetus for the Using a particular analytical frame- giving training a low priority among individual’s choice to obtain more work, Finegold draws insights from the managers. training. German labor law and the UK and US experience in ET invest- Japanese “lifetime employment” ment. The interdependence of three ma- n Occupational trainers themselves are model accomplish that linkage. jor players in making investment deci- accorded low status and are often n The US provides strong incentives sions forms the core of this framework poorly trained and remunerated, mak- — the three being company managers, ing training vulnerable to cost cutting. for individuals to invest in further government policymakers, and individ- Japanese and German companies have education following high school. ual workers. Another factor, the institu- made skills development an integral More high school students continue tional setting, tips the scales in the de- part of corporate structures, using eco- into higher education than in any cision to invest or not invest in ET, nomic downturns as an opportunity to other advanced industrial nation in according to the author. Emphasized increase training, and entering upturns part because of the value employers are two characteristics of high-level with the advantage of better-trained place on formal degrees and open skills: the length of the period prior to workers. access to higher education. The Brit- realizing a benefit from an investment ish exam system essentially rations in ET, and the various correlates of the n The electoral cycles in the US and higher education opportunities. Wage decision to invest. Within these at- UK mediate against a long-term per- differentials between the skilled and tributes, these are the factors Finegold spective. concentrates on: unskilled are not great enough to in- n In the US, the general decentraliza- terest those not in college in occupa- 1. The Short-Term Outlook of the tional training. US and UK tion of educational decisions to states has moved policymakers away n Both countries have failed to design Finegold maintains that a long-run from strategic planning involving “a clearly defined, high-status struc- strategic perspective is more conducive training. Central governments in the to investment in ET because of the high ture of vocational qualifications that UK and the US have focused on dis- cost and deferred nature of the returns on would help motivate individuals to this investment. However, three structural advantagement rather than on high- obtain training, as in Germany.” elements of the UK and US economies er-level skills acquisition.

Evaluation Forum n Issue 11 n Summer 1995 81 Evaluation Issues and Activities n Economic Reform in the European Union

4. Absence of Institutional Struc- 5. The Lack of Sustained Pressure the Organization for Economic Cooper- tures and Processes to Foster to Innovate ation and Development. Cooperation For more information about the Brit- n The large size of the US domestic ish TEC system, the UK Employment n The lack of formal structures and market and historically low depen- Department produces an interesting, in- processes in both countries erodes dency on experts has supported slow formative series of reports titled Re- interest in the mutual benefits of co- adaptation to changes in world com- search Series, many of which analyze operation, such as those gained petition. The UK is more integrated TEC activity. The Research Manage- through information sharing, the co- into the world economy but has trad- ment Branch of the Department also ordination of activities, joint financ- produces the series Individual Commit- ed mostly with developing coun- ment to Learning and Investors in Peo- ing of training, the clustering of spe- tries. And the UK has insulated cer- ple. cialized businesses and industries, tain economic sectors from Another useful reference is a recent and the development of employer competition, contributing to eco- book by Bennett, Wicks and McCos- organizations. nomic decline. han, Local Empowerment and Business Services: Britain’s Experiment with n Cooperation between managers and The Author’s Conclusions Training and Enterprise Councils, pub- policymakers is weak, leading to an lished by UCL Press, Ltd. in 1994. Finegold’s position is that the UK over-emphasis on job-specific train- The following is a sample of relevant ing at the expense of general skills, and US cannot any longer avoid inter- reports in the Employment Depart- and on the inability to keep pace national competition and its implica- ment’s Research Papers and Training with technological change. tions. A decade of market-oriented pol- Research and Development reports: icies, such as privatization and The Anglo-American stress on Boreham, N. and T. Arthur, Information deregulation, in the context of the in- Requirements in Occupational Deci- “voluntarism” leaves training deci- creasing integration of markets sion Making. University of Manches- sions to managers and their employ- throughout the world demand that these ter, 1993. ees. This reduces an exchange of in- countries match or outdo the high-skill competition. So both countries are rec- Crowder, M. and K. Pupynin. The Moti- formation between government and vation to Work. Minds at Work, key producers which is critical in ognizing the need to improve their ET 1993. systems. But Finegold is not all that op- encouraging ET investment in timistic. Even if the attainment level of Ernest and Young. TEC Participation in transferable skills. Connections be- students is raised significantly, too National Development Activity,1993. tween government, unions and em- many other features mediate against Marchington, M.et al. New Develop- ployer organizations in the US and major ET reform, he concludes. He be- ments in Employee Involvement. UK ranked last in a study of eight of lieves that these other influences may Manchester School of Management, the largest capitalist economies in discourage key actors in both countries 1992. their ability to formulate and imple- from investing in higher-level skills de- Pearson, I. The Role of Evaluation in ment consensual labor market and velopment. TEC Planning. WMEB Consultants, industrial policies. 1994. Pupynin, K. and M. Crowder. Regional n The industrial relations arrange- Editor’s Note Advice Unites: An Examination of ments in the US and UK have dis- Models for Delivering Advice and couraged managers from pursuing David Finegold is the author of Edu- Guidance to TECs and Department the acquisition of high-level skills cation, Training and Economic Compet- of Employment Regional Offices. itiveness, published by the UK’s Oxford Minds at Work, 1994. on the part of workers. Britain’s Press in 1993. He has also written in craft unionism has interfered with the Oxford Review of Economic Policy Rix, A. et al. Investors in People: A the reorganization of work and in- and contributed to International Com- Qualitative Study of Employers. vestment in new technologies, which parisons of Vocational Education and CRG People at Work, 1994. are needed to support a high-skill Training for International Skills edited Taylor, S. and L. Spencer. Individual by P. Ryan in 1991. William Richard- Commitment to Lifetime Learning: flexible strategy. In the US, the son’s most recent book, on the reform weakness of organized labor and the Individuals’ Attitudes. Social and of post-16 education and training in Community Planning Research, absence of protective legislation en- England and Wales, is co-edited with 1994. suring employee’s rights has al- David Finegold. Laurel McFarland has Vaughan, P. TECs and Employers: lowed managers to design corporate written numerous articles on the econo- mics of postsecondary education and Developing Effective Links. Employ- restructuring with little concern for has been involved in several compara- ment Department, 1993. skill levels. tive education projects sponsored by the National Academy of Sciences and

82 Evaluation Forum n Issue 11 n Summer 1995 Evaluation Issues and Activities n Economic Reform in the Former Soviet Union

Economic Reform in the Former Soviet Union

Editorial Introduction Why should American educational, from manufacturing to services and in- training and employment professionals formation in postindustrial countries, In the Interviews Section of this is- and evaluators want to know about and with the internationalization of eco- sue of Evaluation Forum, we feature an economic reform ideas in these post- nomic competition, we clearly need a interview with Salvatore Zecchini of the socialist countries? It is understand- broader canvas on which to paint Amer- OECD concerning current economic able that our first priority must be a bet- ican policy options. Therefore, we are and employment policies and their con- ter understanding of our own economic making this beginning investment in the sequences in Central and Eastern Eu- system and workforce. But in the past history of economic reforms in the ropean countries and the now-indepen- this logical emphasis has tended to iso- former Soviet Union. dent republics of the former Soviet late us from the experiments of other This cameo of reform efforts ex- Union. In this part of Issue #11 we fo- industrialized nations. This singularity presses the basic tension in all market- cus on the former USSR through a se- of focus is changing rapidly. The Ameri- oriented economies between economic ries of articles on pre-Stalin, pre-Yeltsin can literature on employment and train- efficiency and socio-economic equity, economic reform efforts. These reforms ing is now filled with insights from and therefore contributes in its own dis- constructed an important foundation for Western Europe, Japan, and the re- tinct way to our current thinking about the economic reform debates continu- structured nations of Central and East- the conduct of the U.S. economy in a so- ing in the separate republics, and in ern Europe and the former Soviet cial democracy. particular in Russia. Union. In the context of a major shift nn ment of that system . . . would not nonproductive individuals and busi- A Selective Overview prove an onerous burden, given the nesses, according to Hewett, and unfair of Soviet Economic educational level of the labor force in distributing rewards for successful and the smooth operation of systems competition. These correlates of mar- Reform Policies inherited from capitalism. Socialism, ket-driven systems are not supportive In this article we selectively review for Lenin, was the marriage of the ef- of egalitarianism. In fact the introduc- an informative Brookings Institution ficiency of capitalism and the equity tion of extensive social and economic book by Edward A. Hewett published so fervently sought by generations of entitlements, Hewett contends, can seri- in 1988 — Reforming the Soviet Econ- socialists. . . . Now, seven decades ously reduce a market economy’s effi- omy: Equality versus Efficiency. A se- after the Russian Revolution, Soviet ciency and profitability. Therefore the nior fellow in the Foreign Policy Stud- leaders are still searching for a for- form Soviet economic reform has tend- ies program at Brookings, Hewett’s mula that will make that “marriage” ed to take has essentially required a re- analysis of the pre-Gorbachev and Gor- work. They have a system that can definition of socialism. boast many successes, most notably bachev reforms provides useful insights Hewett views the continuing debate in achieving some of the equity goals about the Soviet experience in revital- over the nature and direction of eco- of fundamental importance to Lenin izing and restructuring its economy. nomic reform in the USSR as a debate and his Bolshevik party. The state, Hewett’s main message is that while over efficiency vs. equality principles. virtually the sole employer, has a tre- the Stalinist system was successful in He says the essence of this debate is mendous appetite for labor. Thus the achieving industrialization and secur- whether a viable method can be found USSR has almost no involuntary ing social and economic equity, it was to “dilute the egalitarian bias” of the unemployment, still one of the main not suited to the ongoing management system in achieving more economic ef- sources of waste and human misery of a mature and increasingly complex ficiency — that is, whether a strategy in much of the capitalist world. . . . industrialized economy or to the can be implemented to accomplish re- This is the ultimate “welfare state.” achievement of economic efficiency. form goals without destroying socialist . . . It is the efficiency side of the Relevant to this viewpoint, the follow- legitimacy. ing excerpt from his book is enlighten- marriage that has not gone at all well. ing: Pressures for economic reform in the The Meaning of Economic direction of a market economy have Reform in the USSR Lenin’s vision of the first, socialist, been the legacy of this dilemma, but We tend to forget that Nikita Krush- phase of communism was powerful such reform remains a quandry. Well- in its simplicity. He foresaw a system chev proposed economic reforms in functioning market-oriented economies 1957, Aleksei Kosygin in 1965, and in which workers . . . would enthusi- are dependent on economic efficiency astically pitch in to produce for each Leonid Brezhnev in 1973 and 1979. and competition for customers. These However, Hewett does not view these other, distributing the fruits of their economies can be ruthless in punishing labor fairly and equally. The manage- as “comprehensive” reforms. They

Evaluation Forum n Issue 11 n Summer 1995 83 Evaluation Issues and Activities n Economic Reform in the Former Soviet Union

were more in the nature of partial re- outcome of reform efforts. Studies of earn income. This dependence on wages forms that contributed to later reform these outcomes for the Krushchev and and salaries, rather than on assets, then efforts by alerting leaders to what Brezhnev eras suggest that the particu- had a leveling effect on income distri- seemed to work well and what did not lar mix of characteristics these reform bution. The reduction of wages related work. Neither does he see Perestroika strategies involved ultimately proved to low unemployment, and the security under Gorbachev as a comprehensive fatal, leading to economic retrenchment protections afforded workers irrespec- reform in the economy, if we define rather than to improved performance. tive of the economic profitability of economic reform as a change in the their workplaces, also equalized the dis- way resource allocation decisions are Strengths and Weaknesses in tribution of income. Added to these in- made — i.e. in the “institutional ar- Soviet Economic Performance come-leveling effects was the high par- rangements” that determine how re- Hewett’s analysis pinpoints Soviet ticipation of women in the labor force. sources are produced and distributed. central planning as the most critical At the same time, economic losses were absorbed not by single enterprises but The author points out that economic source of performance problems. Sovi- reform in the Soviet Union and Eastern et leaders attempted to preserve what spread out across the whole system. Even the shortage of goods was an Europe has been associated most often they judged were the strengths in the equalizing influence. with “economic decentralization.” This existing system, including central plan- focus is understandable given these ning, while seeking to reduce or elimi- The main strengths in the Soviet sys- countries’ economic performance prob- nate the weaknesses, but the two sets of tem — egalitarianism, economic securi- lems. However, Hewett acknowledges attributes were so intertwined as to re- ty, and the growth in living standards that “economic recentralization” can duce the effect of this compromise. — were generally regarded by Soviet also be seen as a reform, whether or not The three main strengths Soviet leaders as socialist achievements that it is appropriate in resolving economic leaders saw in the existing economic must be preserved. Consequently these problems. system were 1) the high level of growth leaders tended to minimize their poten- tial negative effects on economic per- The important issues to study in ana- in the economy and in living standards, lyzing economic reform efforts, accord- 2) the high degree of economic security formance in proposing economic re- forms. Since many of the weaknesses in ing to the author, are 1) whether the re- in the context of this growth, and 3) the performance were closely tied to a com- form decentralizes or recentralizes de- system’s commitment to egalitarianism. cisionmaking regarding resource pro- Hewett claims that the USSR came as mitment to economic security and equi- ty, reformers had to redefine these con- duction and allocation, 2) whether the close to full employment as any indus- cepts in the process of designing new reform is comprehensive, and 3) trialized economy can hope to accom- whether the reform affects all major plish — “the degree of personal eco- strategies. Hewett feels this helps ex- plain the severe difficulties leaders en- economic institutions. Some changes nomic security in the workplace [was] countered in Soviet reform movements. are not reforms, he says, such as eco- virtually unparalleled in Western coun- nomic policy changes that utilize the tries.” Inefficiencies in resource allocation existing system to improve economic Egalitarianism was, of course, a key were not the primary concern in devel- performance. But such changes are im- element in Marxism. Marx’s major crit- oping new approaches. Technical ineffi- portant in making significant shifts in icism of capitalism was that it produced ciency garnered more attention. That is, the way the economy is guided without and/or reinforced inequities in the dis- Soviet reformers felt that the misuse the risk of organized resistance to tribution of wealth, income and power. and waste of resources in the production change. He viewed the existence of unemploy- process were more critical issues to deal with than the issue of what sectors of However, Hewett feels that while ment as a symptom of this inevitability. economic policy changes should be This ideological perspective explained the society benefited most or least from economic progress. This priority led So- part of reform packages, more systemic socialists’ emphasis on the means of viet planners to force enterprises to economic reforms must be the major production as well as on the allocation drivers of change, particularly given of resources. The Russian revolution economize on inputs — that is, on labor and material — since these were being the performance problems that faced was the first effective strategy, Hewett used at much higher rates than was nec- the Soviet Union during the Gorbachev says, for acting on this perspective in a period. He would likely take the same nation-state, even though the socializa- essary within Soviet society, and more than was typical in other industrialized position today. tion of the means of production was completed only after an abusive collec- nations. And Hewett cautions that reform tivization of the peasantry. The decline in economic growth movements are not all of one piece. rates and labor productivity was Gor- Each is distinguishable from others In this kind of system, where the through its own array of personalities, state is the only owner of financial as- bachev’s key worry regarding economic performance. In a search for causes, So- forces and conditions, the particular sets and production, Hewett comments viet experts identified several important combination of which influence the that one must be gainfully employed to antecedents:

84 Evaluation Forum n Issue 11 n Summer 1995 Evaluation Issues and Activities n Economic Reform in the Former Soviet Union

n The low level of mechanization of mined the allocation of bonuses to en- protect their empires and enterprises Soviet industrial enterprises. terprises, and in price control. Among from sharing decisionmaking with oth- other things, these reforms proposed a er businesses. If there were efficiency n The tendency to produce low-quality recentralization of planning, moving it gains, Hewett’s view is that they were goods that did not respond to con- back from regional authorities to cen- swallowed up by burdensome adminis- sumers’ needs or world standards. tral ministries in Moscow, and a tration. strengthening of the discretion of exist- n An excess demand for consumer or The author draws three key lessons investment goods. ing state committees. from the failure of these reforms: The most rapidly implemented strat- n A tendency for industrial develop- 1. Do not ask the very entities you egy was administrative reorganization. ment to move ahead of infrastruc- Implementing the incentive system and hope to reform to be responsible for ture development, creating supply/ price reform lagged considerably be- the reduction of their own powers. demand imbalances. hind. This delay allowed the existing 2. Do not neglect the demand-side con- With the slowing of economic system to organize its resistance to the sequences of reform, while focusing growth, Soviet policymakers had less changes ahead. Many of the reforms on supply-side solutions — the discretion in setting priorities among were never implemented, and the im- mergers actually increased the high pact of those put in place was not en- the competing interests and pressures concentration of Soviet industry and during. from consumers, enterprises and gov- the power of suppliers, which can- ernment agencies seeking sources of in- Hewett places the most blame on the celled out the reform’s potential in- vestment, particularly in the area of de- design of the reforms. It involved too centive effects. fense spending. Consequently they many changes at one time, some unre- were faced with the need to preserve alistic. This created too much pressure 3. Do not allow a central government economic equity and security while on the system and led to a dilution of body to control merger activity, boosting economic growth. This chal- strategies before they had been given a since this activity then may not be lenge underlaid all reform efforts. chance to work. driven by a search for increased Since it was difficult to determine the profits. potential causal linkages between phe- The author concludes that the ad- ministrative recentralization strategy nomena such as falling productivity, In 1979, there was another effort to had the most lasting effect. It reinstated low quality output, imbalances in sup- redirect the economy. The most inno- ply/demand, excess demand for invest- a Stalinist decisionmaking hierarchy to vative component, Hewett believes, manage the economy, expanding the ment goods, the sustaining of full em- was to develop performance indicators. number of central ministries and state ployment, and the maintenance of price At the risk of oversimplification, under stability, economic reforms always rep- committees. Succeeding leaders ex- this system an enterprise using more la- panded this apparatus further in an ef- resented agonizing compromises, often bor than a certain determined amount fort to control an increasingly complex based on inadequate data and research. would not be rewarded, whereas more economic system. productive and efficient ones would be. Earlier Reform Efforts There were problems with the use of The 1970s Reforms aggregate measures of enterprise activi- Hewett is adamant that Western The reforms proposed in 1973 at- ty, however, and no serious effort was readers place the Gorbachev reform package in historical perspective if they tempted, Hewett claims, to increase the made to implement this reform. efficiency of this managerial system are to understand its importance. His without decentralizing it. This was to Reform Accomplishments emphasis in analyzing pre-Gorbachev reforms is to cast Gorbachev as the first be accomplished by merging various Taken as a whole cloth, there were enterprises under one decisionmaking soviet leader since Kruschev to con- two common elements in this series of authority, permitting this authority to ceive, strongly support and actually im- generally unsuccessful post-Stalin re- plement economic reforms. shift production tasks and input allot- forms: ments from the least to most efficient n The basic tool of reform was the The 1960s Reforms enterprises. Brezhnev wanted the cen- tral authority to move away from the central economic plan. This plan It is the comprehensive 1965 re- operational management of enterprise was developed by central govern- forms that provide the best context. activity into long-range strategic plan- ment ministries and involved indica- The strategies proposed were similar in ning for entire economic sectors. tors to determine enterprises’ com- many ways to Gorbachev’s, and were pliance with the plan, as well as the intended to affect fundamental aspects However, the 1975 reforms, which allocation of bonuses to enterprises of the Soviet economy. In particular were to merge enterprises into more they involved changes in who super- controllable units, did not result in the that met or exceeded the plan’s ex- vised enterprise activity, in what deter- changes planned. Ministers managed to pectations.

Evaluation Forum n Issue 11 n Summer 1995 85 Evaluation Issues and Activities n Economic Reform in the Former Soviet Union

n The reforms did not focus on the Hewett emphasizes that reform in- ciency of central planning, but in gen- needs and demands of customers in volves a complicated process requiring eral supported the ministerial system the open market, but rather on cen- the involvement and investment of all even though opposing its authoritarian tral decisions about what constituted relevant stakeholders. This participa- control over enterprise activity. The high quality goods. There was a fear tion cannot occur without neutralizing idea of mergers appealed to most of the effect of the proposed changes on these economists, where planning could of loss of control if world markets those losing something, and without be consolidated for a small number of were allowed to direct Soviet enter- enlisting the support of those having major economic sectors. prises. something to gain. But the Soviet re- In the midst of these considerations, forms continued to permit a more au- As a result of Soviet resistance to Mikhail Gorbachev opened Pandora’s giving “the market” the primary role in thoritarian process. Box, taking the position in his speech increasing economic efficiency, the Relevant to the design of reforms, at the 1985 Party Congress that the eco- pre-Gorbachev reforms experimented he advocated a first step dedicated to nomic situation was such that it was no mainly with plan indicators and incen- the development of very clear priorities longer possible to make partial im- tive systems, leaving the organizational and emphases, followed by strategies provements — what was needed, he mechanisms operating the existing capable of anticipating and reducing said, was “radical reform.” The subse- economy largely intact. the inevitable conflict created by gains quent development of radical reform Hewett makes these comments at the and losses in the major groups affected positions took two forms: 1) compre- end of his analysis of the pre-Gor- by change. hensive reforms similar to the Hungari- bachev reforms: an model, and 2) reforms that sought a The Economic Reform Debate better economic compromise between In their totality, economic reforms in the 1980s economic performance goals and the imply a theory . . . concerning the na- Hewett classifies the Soviet players vast safety net protections for individu- ture of the problems, the systemic in the ongoing 1980s debates about als and families. The latter position changes needed to fix the problems, economic reform into three general cat- brought to the surface the underlying and the best strategy for bringing egories: and ongoing tension between economic about those changes. In the history of efficiency and socio-economic equity. n The neoconservatives, who support- Soviet reform efforts, most of the Although Hungary was viewed with ed a return to the basic principles of leadership’s attention has been fo- interest, insights from the reforms initi- cused on the first two components of Stalinist central planning, but with- ated in other socialist countries had re- the theory: what’s wrong, and what’s out its correlate of political repres- markably little influence on the nature the fix? The strategic issue has not sion. of Soviet reform ideas. This was the been addressed openly, and . . . the case even though Gorbachev encour- approach has been rather simple- n The moderates, who accepted the existing system but had significant aged a formal study of reforms in East- minded: the bureaucracy will, in ern Europe, particularly Poland, Hun- ideas for improving it. good faith, design and implement de- gary and East Germany. The latter crees conforming to the wishes of the n The radical reformers, who advocat- models could have been very useful, party. However, the modest results of ed system-wide reforms, particularly but serious consideration of them flew previous reform efforts have gradual- in the face of historical Soviet beliefs ly stirred up interest in the strategic a decentralization of the manage- ment of the economic system. that the problems of the USSR were issue, with the result that a rather unique and could be understood best in simple, widely held theory has The core of the neoconservative po- isolation from the broader tapestry of emerged that previous reforms failed sition was that the economic system 20th century socialist reforms. primarily because of bureaucratic re- must become better disciplined — but sistance centered in the ministries . . . proponents differed considerably about The Gorbachev Reforms this view is not wrong, but it is in- the means for accomplishing this. The complete and superficial...it fails to main question was whether markets The five-year plan emerging under Gorbachev for 1986-1990 had very am- take into account the flaws in reform should be used as the tool for imposing bitious goals. It proposed a reversal of design that create incentives for min- such discipline. istries to preserve controls over en- the downward trend in economic terprises. Furthermore, it assumes Most Soviet economists fell into the growth rates in all major output catego- that other elements of the system are moderate group, which represented a ries. Hewett reports that national in- anxious for the reforms to work . . . wide range of views about how to re- come was to grow at an average of this ignores the large and important structure the economy, stretching be- 4.1%, and labor productivity at 4.6%, political issues that affect in a funda- tween a commitment to strong central comparable to the early 1970s. Real per mental way the choices made in the planning at one end of the continuum, capita income growth rates were to av- design of reforms and the fate of the to radical comprehensive reform at the erage 2.7%, significantly above recent resulting decrees. other. Most wanted to see substantial rates, even if considerably below the improvement in the quality and effi- early 1970s. In the 1990s these rates

86 Evaluation Forum n Issue 11 n Summer 1995 Evaluation Issues and Activities n Economic Reform in the Former Soviet Union

were to be increased even more dramat- warding or sanctioning enterprises for economically if it could not compete in ically. their performance. Interestingly, East- world markets. But he was equally con- ern European and Chinese economic cerned, according to Hewett, about na- The means for meeting these targets, reforms emerged in some of these prin- tional security and the level of domes- over a two-phase process, was to be a massive improvement in the quality of ciples, even though uncredited. tic support for the Communist Party. Hewett’s fear was that if the Gorbachev goods and services produced. Gor- Direct, detailed central management reforms lost momentum, the USSR bachev presented the five-year plan as a of enterprises through an annual plan route to setting world quality standards, was to be jettisoned, leaving enterprises might rely primarily on military power in the context of residual strength in not simply a way to meet the plan’s responsible for the economical use of that sector but weakness economically goals. This was unrealistic, more a po- their assets, and for either the rewards litical ploy than a rational economic or costs related to the general perfor- and politically. position. Hewett’s opinion is that the mance goals set for their own economic Concerned about the outcomes of real test of Gorbachev’s reform plan sector by the central government minis- the Gorbachev reforms, Hewett propos- lay in whether he would support suffi- tries. These enterprises were also to be es four possible scenarios for the fu- cient economic discipline to achieve the freed, Hewett says, from local manage- ture: high-quality output proposed. ment interference. In essence they were 1. The success scenario: everything to be left vulnerable to competition in The first stage of reform was to fo- works as planned. The reforms oc- the marketplace. Within this frame- cus on partial changes. The second was cur rapidly, output quality improves to involve the more comprehensive work, state enterprises were to be placed in competition with new private substantially, and so does economic changes. In 1987 the major outlines of sector self-employment schemes and efficiency. the second stage were fleshed out by the Aganbegian Scientific Council small cooperatives. For the Soviet labor 2. The high growth scenario: the na- force, the Gorbachev reforms, accord- working with a system of committees tional income growth rates are close ing to Hewett, amounted to “threaten- assigned to design different compo- to what was planned, with only nents of the reform. This activity pro- ing the workers of poorly managed en- terprises with lower income, and even moderate improvement in the quali- duced much open debate and better the loss of their jobs, but at the same ty of goods and services. consensus-building, and a set of guid- ing principles were established that time . . . giving them a role in choosing 3. The high quality scenario: econom- and controlling enterprise manage- would direct economic reform based on ic growth performance is moderate ment.” the Council’s work. but quality indicators express signif- The deadline for implementing this Hewett describes these principles as icant and sustained improvement, second and most all-encompassing a set of interrelated changes in all of even though economic efficiency is the key mechanisms operating in the phase of economic reform was January not substantial. 1991, in retrospect a deadline beyond economic system — a comprehensive Gorbachev’s tenure as Soviet leader. 4. The failure scenario: nothing and radical reform package: works. n Annual central plans were to be The Implementation of the Hewett sees some scenarios as more eliminated. Gorbachev Reforms probable than others. The extremes n Private sector economic activity and Hewett thought implementation were less likely to occur. However, the development of economic coop- would be very difficult, with “powerful “success” would, he thinks, require that eratives were to be encouraged in and frequent temptations to retreat from almost everything ‘go right’ — the de- the principles of the reform.” The most certain sectors. sign of the reform perfect, the imple- important political question, he sug- mentation strategy flawless, and radical n Enterprises were to be allowed to go gests, was how to reduce the negative improvement in quality resulting. Re- bankrupt. effects of the transition to a new eco- ferring to the history of previous re- nomic system and therefore create and form efforts, he says neither the work- n Workers were to be allowed to lose maintain the new system’s legitimacy force nor industrial management was their jobs. with important stakeholders and the likely to respond at a level that would n Prices were to be far more flexible. Soviet people. Serious economic ques- allow this to happen. tions attached to the pace and chronolo- n The monetary system was to re- gy of reform, and the fairness with Implications of Success or spond to a truer medium of ex- which the transition was carried out. Failure for the West change. Hewett is convinced that Gorbachev It is intriguing, given the coup at- This agenda was to make substantial was more interested than any other So- tempt against Gorbachev, the subse- changes in the decisionmaking hierar- viet leader in the USSR’s position in quent end of his leadership, and the dis- chy, the information system for policy- the world economy. He believed the solution of the Soviet Union, that making, and the incentive system re- Soviet Union would not be successful Hewett predicts that the “failure” sce-

Evaluation Forum n Issue 11 n Summer 1995 87 Evaluation Issues and Activities n Economic Reform in the Former Soviet Union

nn nario would be very unlikely, since it environmental groups, and press rich would require some “dramatic turn- Gorbachev’s countries and corporations to assume around” such as the removal of Gor- Perestroika their share of the global clean-up task.” bachev and a shift of power to econom- As president of the former Soviet Although Gorbachev hardly seems ic hardliners. Without benefit of a to qualify as an environmental czar, crystal ball, Hewett suspects that the Union, Mikhail Sergeyevich Gor- given some of the history of his tenure two middle-range scenarios would do bachev became deeply involved in eco- nomic issues, particularly reforms that as president of the Soviet Union, the most to enhance Soviet economic rela- concept of an International Green Cross tions with the West. However, he is could lead to greater economic perfor- was, in fact, Mr. Gorbachev’s idea — less certain about the benefits for Sovi- mance. The Twelfth Five-Year Plan, for 1986 through 1990, gave substan- although the organization is supported et foreign and defense policy. His gen- by an international group of prominent eral assessment is that successful eco- tial expression to his goals for reform: political, scientific and spiritual lead- nomic reform would give Soviet higher national income growth rates, the acceleration of economic develop- ers. And his affinity for ecological pro- leaders the capability either to follow a tection is not as inconsistent as it might less aggressive foreign policy or to fi- ment, the improvement of production seem to the West. nance a greater military build-up. The and its products, and changes in the choice, he thinks, would rest with the public’s view of the economy and soci- He joined the Communist Party in level of success in reforming the Soviet ety. The reform concept and plan for 1952, quickly becoming a leader of the political system. change was described in his book Pere- Stavropol Komsomol. In 1955 he grad- stroika: New Thinking for Our Country uated from the Law Department of Hewett expects that successful re- and the World published in 1987, Moscow State University as part of an forms could lead to a country better which significantly influenced econom- elite group of Russian lawyers. Be- able to pursue traditional foreign policy ic thinking not only in the former tween 1985 and 1990 he followed a and defense goals, become a new cus- USSR but in Central and Eastern Eu- straight-line political trajectory leading tomer for Western goods, and a new rope. to the status of First Secretary of the competitor in world markets. It could Stavropol Territory Committee of the also mean a gradual democratization of Mr. Gorbachev no longer exerts di- Communist Party’s Central Committee, the political system, consistent with the rect political, economic or ideological influence in the new republics formed then Chairman of the USSR Council of long-term interests of Western nations. Defense and a member of the Presidi- Success could lead, he believes, to by the break-up of the USSR, but he is um of the USSR Supreme Soviet, and greater participation by the Soviet anything but a has-been. Following his fall from power in 1991, he formed the ultimately to the president of the Soviet Union in international economic insti- Union. tutions and markets, and to new rela- International Foundation for Socio- tionships with the West based more on Economic and Political Studies in Born on a farm in Southern Russia economic than defense issues. A reform Moscow, and subsequently an arm of and steeped in rural agricultural life scenario that stimulated technological this foundation in the United States, growing up, he had long been con- innovation and improved the quality of The Gorbachev Foundation USA in cerned about deteriorating fisheries and Soviet goods and services would, he San Francisco. The foundation serves forests, and about the management of concludes, enhance Soviet economic as an economic and political ‘think farming, and had served as the USSR’s ties with the West. Unsuccessful reform tank’ intended to promote social de- top agricultural official prior to becom- scenarios could reverse these trends. mocracy in the former Soviet republics, ing president. Therefore the interview and sponsors international seminars Mr. Gorbachev provided to Colin Greer Hewett does not think Western in- and conferences on global economic, for Parade Magazine in January of fluence over the outcome of the Gor- political, environmental and peace is- 1994 is informative regarding his view bachev reforms would be more than sues, including defense conversion. of his new role vis-a-vis international minimal. The West would be capable environmental activism: only of exerting pressure for minor cor- Only sixteen months after serving as president of the Soviet Union and gen- rections in the reform process, perhaps We need to spawn a different intel- eral secretary of its Communist Party, making that process more likely to oc- lectual and values climate and a dif- he was named president of the Interna- cur. This forecast appeared to be accu- ferent set of economic goals. No tional Green Cross and Green Cres- rate for the early Yeltsin period, but is country actually believes the free cent, which aspire to represent for envi- now debatable as neoconservative market can be socially irresponsible. ronmental crises what the Red Cross views have gained momentum with the I want more socially oriented market and the Red Crescent already represent growing divisions in political leadership. economies in the world. . . . At the for disaster relief. According to Micha- same time, the business marketplace el Wines of The New York Times, Mr. that is based on economic freedom, Gorbachev’s idealistic plan aims to that can function with independent “standardize antipollution laws world- producers and owners, forms the ma- wide, integrate the diverse efforts of terial basis for democracies. I see no

88 Evaluation Forum n Issue 11 n Summer 1995 Evaluation Issues and Activities n Economic Reform in the Former Soviet Union

In this sense, Perestroika involved a The course of intensification is dictated by objective conditions, by renewal of every aspect of Soviet life. the entire course of development of the country. There are no alternatives. Only an intensive economy, developing on the basis Implementation Challenges of a state-of-the-art scientific-technical base, can serve as a Gorbachev’s thinking, the work of reliable material base for increasing the welfare of workers, the designers of his reforms, his ac- guaranteeing the strengthening of the position of the country on claimed book, and the institutionaliza- the international arena, ensuring that it will deservingly enter the tion of new ideas in significant policy new century as a great and prospering power. pronouncements, launched enormous changes in the USSR which have con- Mikhail S. Gorbachev, in Pravda, December 1984. tinued to influence developments in Russia and the other republics. This is not to deny that there were serious challenges to economic reform during contradiction between the market and active, constructive collaboration to this early reform period as well as cur- democracy. If we are going to protect improve the economic situation. Pere- rently. the planet’s ecology, we are going to stroika was not only about reforming need to find alternatives to the con- the economy but about opening up and One challenge has been the need to sumerist dream that is attracting the democratizing Soviet society, and en- modernize the economy and shift in- world. Otherwise, how will we con- gaging people more directly in its im- vestment policies. Another has been the serve our resources, and how will provement. necessity of freeing central authorities we avoid setting people against each from an operational role, to a more In June, 1987, the CPSU Central other when resources are depleted? general focus on improving the eco- Committee adopted principles for im- nomic environment and directing the Given this revised view of the politi- plementing this approach to reform in basic steps in economic development. cal architect of Perestroika, what were “Fundamentals of Radical Restructur- The new economic system was expect- the major reforms proposed within this ing of Economic Management.” This ed to emphasize more decisionmaking second Russian revolution in the was likely the most radical program for authority and much greater responsi- 1900s? economic reform in the Soviet Union bility and accountability at the regional since Lenin introduced his New Eco- The Concept of Perestroika and local level. The expanded role for nomic Policy in 1921. It proposed a private and cooperative economic ac- transition from an excessively central- The most immediate priority at that tivity also posed challenges. time, according to Gorbachev’s book, ized management system to one based Certainly the shift from obligatory was to put the economy in order, tight- on democratic centralism and self- State economic targets to norms linked en discipline, and raise the level of or- management. So Perestroika had sev- ganization and responsibility. A pro- eral dimensions: to the performance of enterprises has been a genuine challenge. The reorga- found structural reorganization of the n The overcoming of economic stag- nization of different levels of the gov- economy was needed, he felt — a re- nation and an acceleration of social construction of its material base, the ernment hierarchy, and the emphasis on and economic progress and dyna- the coordination of activity have con- development of new technologies, mism. stituted new hurdles. Establishing a changes in investment policy, higher new incentive system affecting both en- management standards. n The comprehensive development of terprises and workers, as well as the democracy and an encouragement of The government, he proposed, need- branches of government monitoring individual initiative, creativity and ed to shift economic activity from new them, has also tested the reform move- construction to the technical retooling discipline. ment. An additional and very important of enterprises, to saving resources, to n A shift to scientific methods, and a challenge has been deciding whether sharply raising the quality of output. At better linkage between the achieve- some reforms should be implemented the same time it needed to change the ments of the scientific and techno- before others, and sorting out how to moral and psychological situation in deal with the effects of the transition. society, and the organization, style and logical revolution and a planned methods of work. A policy of openness economy. Comment and social justice was needed. n Better living and working conditions For Central and Eastern Europe, and It was imperative, he contended, that for Soviet citizens. for the former Soviet Union, the transi- government take into consideration the n Greater social justice. tion to a market economy clearly repre- diverse interests of people, of work col- sents a difficult historical passage. This lectives, of public bodies, and of vari- n An elevation of honest, highly-qual- is certainly to be expected given the ous social groups, drawing them into ified labor. magnitude of the economic, social and cultural changes occurring in these so- cieties in the context of their traditions.

Evaluation Forum n Issue 11 n Summer 1995 89 Evaluation Issues and Activities n Economic Reform in the Former Soviet Union

nn Editor’s Note The USSR in the n To hire business managers and ex- perts to increase efficiency. For readers interested in more infor- Transition Period mation about Mr. Gorbachev’s ideas, n To provide fiscal incentives to work- please see: In this article we again take a look at ers to increase productivity, such as Peace Has No Alternatives. A collection the pre-Yeltsin period in the Soviet removing the limits on personal in- of speeches, articles and interviews. Union. The perspective this time is come and increasing the amount of in- South Asia Books, 1986. based on Alexander Samorodov’s come exempt from taxation. Perestroika: New Thinking for Our chapter in a book published in 1993 by Country and the World. New York, the Organization for Economic Cooper- A Framework for Reform NY: Harper Collins, 1987. ation and Development, Structural A wide range of new laws were A Road to the Future. The text of Mr. Change in Central and Eastern Europe edited by George Fischer and Guy passed in 1990 and 1991 to implement Gorbachev’s address to the United reform goals. However, Samorodov’s Nations. Ocean Tree Books, 1990. Standing. The author gives us a more detailed appreciation of economic and conclusion is that despite this legisla- The August Coup: the Truth and the labor force issues in the Soviet Union tive activity no adequate legal frame- Lessons. New York, NY: Harper Col- work for a market economy had been lins, 1992. prior to the creation of the independent republics. put in place by late 1991. The reasons were many. Samorodov suggests the Launching the Reforms following: Samorodov introduced his chapter n Population growth rates were declin- on “The Labour Market, Social Policy ing except in the Central Asian re- and Industrial Relations in the USSR publics. Under Transition” with some bleak sta- n Urbanization was continuing — tistics. The USSR’s GNP was estimated to have declined 17.9% in 1991 and two-thirds of the population were production by 15%. Trade with other urban residents in 1990 compared countries had fallen over 39% in 1991. to 48% in 1957. The official inflation figure was 10% n Total employment was declining but for 1990, but may actually have been as new employment forms were emerg- high as 25%. The rouble had been de- ing, such as cooperatives, self-em- valued six times since December 1990, ployment, and joint stock companies. a decline of over 80% in its value. This was a very difficult way to be- n Although mass layoffs had not oc- gin a transition to a market economy, curred, part-time, less than year- Samorodov muses. Nevertheless, in round, and seasonal employment April 1991 the Soviet Cabinet of Min- was growing. isters indeed proposed to launch such a n Labor supply and demand were seri- reform. A number of goals were to be accomplished within the reform, such ously mismatched — in 1991 nearly as these: 97% of laid-off workers were gov- ernment employees, but 85% of the n To resuscitate economic ties be- job openings were for blue collar tween the USSR and COMECON, workers. the trading network involving Cen- tral and Eastern Europe. n The lack of geographical mobility, housing and employment services, n To increase internal economic link- and complicating administrative reg- ages between enterprises. ulations, were hampering the reem- n To provide “most favored nation” ployment of the unemployed. This status in international trade to those was in the context of a society inex- republics willing to sign the union perienced in dealing with unemploy- treaty. ment, where the tendency was to view the unemployed as personal n To begin privatization, particularly failures. regarding small businesses. n Two-thirds of young people (teenag- n To ban strikes temporarily to stabi- ers and those in their 20’s) lacked lize the labor market. vocational training.

90 Evaluation Forum n Issue 11 n Summer 1995 Evaluation Issues and Activities n Economic Reform in the Former Soviet Union

Consequences for Workers — it became the foundation for subse- six weeks, and there were special ex- Samodorov reports that a high and quent market-oriented laws. The 1991 tended benefits available for the long- law and related legislation reinforced term unemployed, unemployed military continuing level of displacement and the traditional right to work, and ended personnel, and other categories of dislocation were occurring during this early period of reform. He explains this what Samorodov labeled “the system of workers such as new entrants to the la- administrative coercion to work.” The bor force. The law also required that phenomenon in terms of increased eco- central government was ensuring that strengthened retraining programs were nomic accountability pressures, the mi- gration of labor from the public to the individuals would have the freedom to to be an integral part of labor market choose their form of employment with- policy. Disadvantaged groups — youth, private sector, and the greater discre- in the limits set by the market. This women with young children, the aged, tion on the part of businesses in hiring and firing workers. He says some ex- “right to work” guarantee included the the disabled, and the unskilled — were right to pursue unpaid work, such as given special priority, as were geo- perts predicted a loss of 40 million jobs acquiring additional education, to con- graphical regions with the more promi- from the public sector for 1995, a num- ber likely to be increased by the num- tribute work for the public good, and to nent labor surpluses or deficits. raise children — all of these were to be ber of workers let go from enterprises All of these laws were intended, considered productive and creative suffering economic losses. Also, the Samorodov says, to speed the transition elimination of villages considered non- work. to a market economy, provide a legal productive was seriously affecting rural The law also established formal def- framework for reform, and reduce the employment. initions for “employed,” “unem- negative consequences of change. After ployed,” and “suitable jobs.” In addi- the break-up of the Soviet Union, how- Ethnic unrest and a substantial tion it placed emphasis on employment ever, the individual republics began to movement of workers to Central and Eastern Europe and elsewhere could, services, which had been inadequate adopt their own legislation. Each re- and little used in the past. These ex- public charted its own course — but Samodorov proposes, significantly in- panded services were to include screen- with elements in common with others: crease transition problems, such as the reduction of the supply of highly quali- ing for the unemployed, paying unem- privatization, price liberalization, and ployment benefits, organizing public the lifting of limits on personal income. fied young workers. The most difficult works projects that could absorb the situation concerned youth employment Family Incomes in Central Asia, where in the 1980s half unemployed, and referring workers to sources of training and retraining. The of all unemployed workers were young Service occupations were viewed as qualification and training of employ- people. The main issue was dissonance the major sources of employment for between their skills and available jobs, ment service staffs were to be greatly the unemployed and disadvantaged de- improved and the number of offices in- and their vulnerability to layoffs in the spite the fact that wages and salaries in creased. midst of restructuring. Long-term youth this sector had fallen. This position was unemployment was becoming a major The 1991 law also supported the de- maintained even though approximately problem. velopment and implementation of spe- a quarter of the population of the re- publics already had incomes close to More generally, Samodorov points cial programs at the central, regional and local levels to create jobs, prevent the minimum wage level. out that a key consequence of restruc- unemployment, assist workers in be- turing in the direction of a market Social support for the poor and other economy was the absorption of the coming reemployed, protect workers vulnerable worker populations became from the negative consequences of eco- public sector and industrial unem- imperative, given the need to reduce nomic change, and increase workers’ ployed by the service sector, reducing unemployment without restricting labor blue collar workers’ wages and sala- mobility through retraining and reloca- flexibility. This new mandate was exac- tion. Workers laid off from business or ries. Service income fell from 80% of erbated by inflation which had grown industry were to receive severance pay non-service sector income to 70-75% 500% prior to price liberalization in between 1975 and 1990. up to three months, equal to their aver- 1992. Even with the freeing of prices, age wages for that period. Those work- Samodorov says, the government found Remedial Strategies ers laid off due to the reduction of it could only protect the very poor. staffs or to business failure or reorgani- Family and child allowances were in- In the context of these develop- zation, and who were to be retrained ments, the author emphasized the im- creased based on family income, but a for a new job, were to receive grants debate persisted over what the guaran- portance of the 1991 USSR Law on equal to a portion of their previous av- teed minimum income should be. Employment of Population in address- erage wage while in retraining. ing major labor force problems. Al- However, the deregulation of wages, though the USSR separated into repub- Unemployed workers meeting estab- part of the move toward a market econ- lics eleven months later, this law lished eligiblity requirements were to omy, had widespread effects of another affected subsequent employment poli- be paid unemployment benefits until kind. The average monthly wages in cies in Russia and the other republics they found another job or up to twenty- the USSR were allowed to rise continu- ally during the Perestroika years, lead-

Evaluation Forum n Issue 11 n Summer 1995 91 Evaluation Issues and Activities n Economic Reform in the Former Soviet Union

ment than to the skill level of the work- The Market Meets Its Match (Harvard University Press), by force, particularly to structural unem- Alice Amsden of M.I.T., Jacek Kochanowicz of the University of ployment. However, with the changed employment situation of the 1990s, re- Warsaw, and Lance Taylor of the New School for Social placing labor shortages through labor Research, offers a critique of “shock therapy” as practiced in surpluses and increasing retraining and Eastern Europe, envisioning instead a gradual transition to free related support programs became criti- markets based on East Asia’s experience. . . . But Jeffrey cal. Many workers had a narrow range Sachs, an economist at Harvard and an intellectual architect of of skills which frequently failed to fit shock therapy, brushes aside the authors’ alternative. . . . Mr. the emerging occupational structure. Sachs contends that the successes writ so large on Asia’s The author feels that “the scale of nearly blank economic slate can hardly be models for retraining” at this time was inadequate. economies with legacies of bureaucratic meddling and a fierce The average Soviet worker was less sense of populist entitlement. . . . Mr. Sachs argues that the well trained than his counterpart in Eu- comparisons [with Japan, South Korea and China] are rope. Most workers were used to stay- spurious. The Asian miracle economies started the process of ing with a given enterprise their entire development as rural societies with little capital, no government working lives. With the move to a mar- safety net and very low expectations. Eastern Europe, by ket system, businesses questioned the contrast, is largely urban and heavily — indeed, excessively — wisdom of retraining, given the uncer- industrialized. . . . Eastern Europe is stuck with the crumbling tainty of long-term worker tenure. So in 1991, the USSR initiated a reorgani- remnants of a Stalinist industrial infrastructure, wary multina- zation of their retraining system, estab- tionals that insist on controlling their own technology, and lishing formal training/retraining cen- workers who are linked to enterprises the way medieval ters. peasants were tied to manors. . . . As for gradualism, Mr. Sachs points out that China dropped the commune system overnight. Privatization . . . And while China has never disciplined its state-owned Samodorov views privatization as a industrial sector by opening markets or eliminating subsidies, it major component of USSR reform ef- has built a modern economy around the relatively small forts under Gorbachev. The main strat- decaying core — something Eastern Europe cannot afford to egy implemented during that period do because the decaying core isn’t small. . . . Shock therapy was the establishment of joint stock still seems superior to the alternatives.” companies in place of public enterpris- es. These often involved shareholding Peter Passell, in the January 19, 1995 New York Times by employees. By the end of 1990, over two thousand industrial enterprises, ap- proximately two hundred large con- ing to larger social expenditures, a and trade unions. The market, however, struction firms, thirty-three thousand greater budget deficit, and rising infla- was to determine wage and salary levels small businesses, and eight hundred tion. Therefore wages were growing in through “collective contracts between municipal and public transportation en- spite of the decline in production and management and labour.” These were terprises were placed under a lease sys- labor productivity. And higher wages bargaining agreements, quite different in tem. Based on a survey of enterprises, were, to large extent, demolished by concept from contracts under the previ- those defined as unprofitable were rap- higher inflation. Pressures from work- ous centrally-planned economy. idly privatized. In the agricultural sec- ers to right this wrong began to surge. tor, private farming began in earnest in Skill Levels and Retraining 1989, replacing a portion of the collec- Meanwhile, growing income in- tive and state farms, and by 1990 there equality was occurring due to the polar- Samodorov reminds the reader that were approximately one hundred thou- ization between those with fixed in- the USSR’s workforce was, in general, sand private farmers. comes (grants, stipends and pensions) quite skilled. For example he reports and those with deregulated wages and that in 1991 there were 35 million spe- A new Union of Lessees and Em- salaries. New trade unions of million- cialists (skilled workers and profession- ployers covered more than ten million aires emerged, while pensioners’ stan- als) and 82 million skilled blue collar people. The Scientific Industrial Union dard of living plummeted. A new poli- workers. Graduates from new vocation- incorporated many of the largest enter- cy for negotiating wages and salaries al training schools had increased annu- prises and associations. This Union evolved to address this new socio-eco- ally. Over 33 million workers and 10.5 represented two-thirds of the USSR’s nomic stratification. It involved “a sys- million managers and specialists were industrial output. Even so, Samorodov tem of tripartite collective labour agree- being trained on an ongoing basis that reports that only about ten percent of ments,” according to Samodorov — year. So the new training challenges the total labor force was employed by the government, employer associations, were more related to rising unemploy- private sector companies.

92 Evaluation Forum n Issue 11 n Summer 1995 Evaluation Issues and Activities n Economic Reform in the Former Soviet Union

Unions continued to grow in power, Becker, A. Sitting on Bayonets: The Sovi- Editorial Commentary even though they were part of new et Defense Burden and the Slowdown three-way partnerships. Their new sta- of Soviet Defense Spending. Santa The articles in this international sec- tus under Perestroika gave them inde- Monica, CA: Rand Corporation, 1985. tion of the journal have focused on a range of economic and employment is- pendence from political and state insti- Bergson, A. and H.S. Levine (eds.) The sues facing all industrialized nations in Soviet Economy: Toward the Year tutions, Samodorov explains, and the the 1990s. Major efforts to reform and 2000. London, England: Allen and opportunity to provide legal and social restructure economic activity in the con- Unwin, 1983. protection to workers in the private sec- text of unprecedented international tor. Laws regarding unions assured Brada, J., E. Hewett, and T. Wolf (eds.) competition are having a significant im- equal rights for all unions, whether in Economic Adjustment and Reform in pact on cyclical and structural employ- the public or private sector, and permit- Eastern Europe and the Soviet ment in these countries. ted them to participate in negotiations Union. Washington, D.C.: The Some of these nations are in a over a range of issues, including their Brookings Institution, 1986. postindustrial phase of development. interest in wage indexation. Brown, A. “Gorbachev: New Man in Others are in transition from collectivist, authoritarian regimes to social demo- But the author comments that few the Kremlin,” in Problems of Com- munism, Volume 34, 1985. cratic market economies. But they managers of enterprises in the newly share a major policy challenge: to in- privatized businesses had training or Campbell, R. “Economic Reform in the crease economic efficiency and com- experience in how to operate private USSR,” in American Economic Re- petitive performance while preserving sector market-oriented companies. As a view, May 1968. socio-economic fairness and equity. result, management training was accel- Gaddy, C. Perestroika and the Soviet Many advanced and developing erated, often utilizing experts outside Military Industry. Washington, D.C.: economies are experiencing substantial the republics. The Brookings Institution, 1992. political change in this decade. How Gorbachev, R. Hope: Reminiscences politics will intersect with economic de- Some Conclusions and Reflections. New York, NY: velopment goals and the pace of tech- Harper Collins, 1992. nological change is yet to be decided. While the independent republics that This web of complex relationships will followed the dissolution of the Soviet Guroff, G. and F.V. Carstensen (eds.) ultimately affect what employment and Union continued to favor the transition Entrepreneurship in Imperial Russia training professionals are able to con- to a market economy, their economic and the Soviet Union. Princeton, NJ: tribute in resolving employment prob- health remained problematic. Unem- Princeton University Press, 1983. lems, and in taking advantage of inno- ployment continued to rise. Retraining Hewett, E.A. “Gorbachev’s Economic vative workforce and workplace centers were slow to function. The Strategy: A Preliminary Assess- possibilities. most significant challenge for govern- ment,” in Soviet Economy, October- ments was the development of efficient, December 1985. effective labor market policies that ______Energy, Economics and could reduce the negative social conse- Foreign Policy in the Soviet Union. quences of the market reforms. Washington, D.C.: The Brookings In- stitution, 1984. Hough, J.F. The Struggle for the Third Editor’s Note World: Soviet Debates and Ameri- can Options. Washington, D.C.: The Alexander Samorodov is with the Brookings Institution, 1986. Employment and Development Depart- ment of the International Labor Organi- Jones, A. and W. Moskoff (eds.) Pere- zation in Geneva, Switzerland. For two stroika and the Economy: New additional references relevant to the is- Thinking in Soviet Economics. sues the author addresses, please see: Washington, D.C.: The Brookings In- stitution, 1989. Samorodov, A. “Labor Market Problems and Developments in the Republics,” _____ (eds.) The Great Market Debate in in G. Standing (ed.), In Search of Soviet Economics. Washington, D.C.: Flexibility. Geneva, Switzerland: Inter- The Brookings Institution, 1991. national Labor Organization, 1991. Matthews, M. (ed.) Party, State, and Standing, G. (ed.). In Search of Flexibil- Citizen in the Soviet Union. Wash- ity: The New Soviet Labour Market. ington, D.C.: The Brookings Institu- Geneva, Switzerland: International tion, 1990. Labor Organization, 1991. Zaleski, E. Planning Reforms in the So- For additional references on the viet Union, 1962-1966. Chapel Hill, former Soviet Union, please consider NC: University of North Carolina the following: Press, 1967.

Evaluation Forum n Issue 11 n Summer 1995 93 Evaluation Issues and Activities n At the National Level

At the National Level

Editor’s Introduction government. This concept has been how we are to judge the performance of welded to two existing characteristics of separate programs, and programs co- In the remaining part of this section our employment and training efforts — ordinated within workforce development on Evaluation Issues and Activities, we i.e. decentralization and privatization. systems. In the next series of articles turn to the issue of the coordination/in- The new emphasis is on system-wide we obtain a snapshot of federal efforts tegration of workforce development policy development, strategic planning, to integrate human services, explore programs and services and their ac- simplified and coordinated service de- some of the problems in the existing countability requirements. We first ad- livery, and a common concept of ac- E&T system now well identified through dress this issue at the national level, countability at the state and local lev- process and outcome evaluations, and then at the state/local level. In discuss- els. In this movement toward system discuss some of the issues involved in ing expert opinion and research focus- development, the role of the federal developing strategies for measuring ing on coordination, we see the con- government is yet to be determined. and judging performance — both in cept of workforce preparation systems Defining rights and responsibilities at all separate programs and in workforce gaining wide acceptance at all levels of levels is now laced with concerns about systems.

nn porters advocating coordination. And forts. Federal authority should have an Some Thinking its funding arrangements mediate oversight component also, according to About E&T against coordination and integration. the report, in terms of defining and measuring expected outcomes across Coordination at the Studies of Services Integration: programs. The analysts propose that an National Level Some Results of Interest additional component of central control Twelve reports detailing studies of should be networking — that is, bring- In this article we trace a bit of the service integration conducted between ing individuals involved in integration history of national-level coordination 1972 and 1987 are reviewed for their efforts together to share information efforts — admittedly a very selective major findings. Figure 1 provides an and build support across federal agen- piece of that history. We look at an in- overview of some of the results these cies and program bureaucracies. At the teresting but little-known review of co- reports share in common. same time, the federal government ordination efforts in 1991 prepared by needs to provide more funding flexibil- the Inspector General’s Office in the Recommendations ity, they suggest, in order to reward U.S. Department of Health and Human in the IG Report separate organizational entities for Services, and at a more recent series of coming together. reports produced by the U.S. General The goal of earlier integration ef- Accounting Office. forts, the report says, was major com- An Impetus for Coordination: prehensive system reform — too high The U.S. General Accounting Lessons from Past Federal an expectation, the analysts claim. The Office Reports Efforts to Integrate Human evaluators suggest a more modest goal Services Management and that reflects satisfaction with shorter- With increasing interest in deficit re- Delivery term, more incremental accomplish- duction, recently the U.S. Congress has ments. requested a series of analyses giving at- A rather extensive literature has ac- tention to the proliferation of federally- cumulated regarding human services The Inspector General’s report also funded programs in various program integration since the 1970s. Services recommends a service provision focus areas. Two general reports in the em- Integration: A Twenty-Year Retrospec- on clearly-defined target groups, con- ployment and training area, titled Mul- tive is an eye-opener in this respect. sistent with more manageable goals. tiple Employment Programs, were pub- The tragic conclusion of this report, Both recommendations underline the lished in June 1992 and July 1993. however, is that over the long haul importance of goals which can be sus- Subsequently, a series of reports were “service integration efforts have had tained politically and organizationally. produced under the same general title little institutional impact on a highly The evaluators acknowledge that but with the following subtitles: fragmented human services system.” some degree of central control (federal n National Employment Training The barriers the report identifies are fa- control) over an otherwise complex and Strategy Needed (June, 1993) miliar: the system is too large, com- fragmented human service system is plex, bureaucratic, professionalized, necessary. This would provide a needed n Conflicting Requirements Hamper and specialized. It lacks powerful sup- external stimulus for integration ef- Delivery of Services (January, 1994)

94 Evaluation Forum n Issue 11 n Summer 1995 Evaluation Issues and Activities n At the National Level

Figure 1 n Overview

Some Constraints to Integration 1 Time constraints: service integra- 3 The lack of cross-training of staff funding levels leading to an unsure tion requires an evolutionary pro- and the development of “general- resource base. cess that allows time to develop ists.” 8 Differing program rules and proce- linkages, gain support and legiti- 4 Competition among providers. dures. macy, plan, organize, and imple- ment. 5 Unclear accountability expecta- 9 Lack of timely, comprehensive and tions. relevant data. 2 Negative attitudes and behavior regarding the concept of service 6 Differing funding rules/regulations. 10 High administrative costs. integration on the part of system 7 Different funding cycles, and fluctu- 11 Organizational territoriality and po- personnel. ations in the type of funding and litical opposition.

Some Lessons

1 Moderate-scale service integra- 3 Linkages are more readily devel- 6 Effective leadership and consen- tion efforts are often more suc- oped in stable rural environments sus-building are necessary. cessful than large-scale compre- than in large urban areas. 7 Long-term planning is critical, in- hensive human service planning. 4 Staff co-location often enhances cluding setting goals and develop- 2 Human services organizational coordination. ing strategies for service integra- networks must be adapted to lo- tion and its funding. 5 R & D funds are essential. cal environments. 8 Political support is required.

Major Benefits of Integration

1 Improved service delivery acces- 3 Increased efficiency. sibility. 4 More effective assessment of client 2 Greater continuity. needs, and strategies for respond- ing to them. n Most Federal Agencies Do Not U.S. Department of Labor. Many of n Assessment and job placement assis- Know If Their Programs Are Work- these provided similar services and oth- tance were often duplicative and ing Effectively (March, 1994). er assistance to overlapping client pop- wasted resources. ulations. A number of problems were n Overlapping Programs Can Add occurring: n Programs were not tracking partici- Unnecessary Administrative Costs pants’ progress through programs or n The federal network of employment (June, 1994). their outcomes effectively. and training (E&T) programs was n Overlap Among Programs Raises sufficiently fragmented to be creat- In the midst of these problems, how- Questions About Efficiency (July, ing difficulties for workers, employ- ever, the GAO was finding that a num- 1994). ers and program administrators. ber of states were reorganizing their E&T efforts to increase the coordina- n Major Overhaul Needed to Create a n A lack of access for potential appli- tion of service delivery at the local lev- More Efficient, Customer-Driven cants to information about these pro- el. The GAO analysts recommended System (February, 1995). grams was creating confusion and this kind of energetic coordination frustration for those needing various movement at the federal agency level, Conclusions in the Reports kinds of services. and the development of a national E&T In the two earliest reports, the GAO policy and strategy. n Needs assessments were often being found that 125 federal programs were made by service providers with a In the January 1994 report, the GAO providing some form of employment summarized results from an intensive vested interest in particular service and training assistance to out-of-school review of nine E&T programs primari- youths or adults not enrolled in ad- “treatments.” ly serving the economically disadvan- vanced degree programs: 49 in the U.S. taged. They found them to have similar Department of Education; 30 in the goals, often to serve clients with com-

Evaluation Forum n Issue 11 n Summer 1995 95 Evaluation Issues and Activities n At the National Level

mon characteristics, and to have paral- n Employers had difficulty obtaining n Utilize evaluation strategies for lel but separate service delivery sys- qualified workers from the E&T sys- learning about the system’s effec- tems. These attributes, they concluded, tem. tiveness that are pragmatic as well as led to unnecessary administrative costs. scientific. The evaluators also discovered con- Elements of Effective E&T flicting eligibility requirements and dif- Systems The GAO’s assessments and recom- ferences in eligibility criteria, compli- mendations generally coincide with re- cating the eligibility determination The GAO did not hesitate to suggest cent Department of Education and De- process across programs. And differ- what an effective E&T system might partment of Labor initiatives in ences in annual operational and budget- look like. It should: education, training and employment, ing cycles hindered administrators in n Be customer-driven rather than pro- with state-level coordination efforts, coordinating their services with those gram-driven. and with local-level E&T agendas. of other programs. These initiatives and efforts are n Provide easy access to services. The February 1995 report character- moving workforce development in the ized the existing E&T system in this n Involve an efficient use of resources. direction of greater integration through state/local, private/public systems that way: n Offer a wide variety of E&T servic- are guided at the federal level by a gen- n It was fragmented organizationally es. eral policy and strategy and by general and in terms of service delivery de- n Be accountable for the results. accountability parameters, but which spite common goals, overlapping give states the major responsibility for client groups, and similar services. n Afford flexibility at the local level pulling programs, services and data to- in determining how best to meet gether and developing methods for es- n It was wasteful of resources. community needs. tablishing system accountability. n Its administrative structures and n Move toward a consolidation of pro- functions overlapped. Comments grams. n Separate funding streams reinforced The HHS and GAO reports give us n Have clear points of entry to the sys- program territoriality. an updated perspective on the problems tem and identifiable pathways and possibilities in workforce prepara- n It was difficult for clients and em- through the system for customers. tion. The concept of systems of pro- ployers to understand ways to enter grams is the outcome. The 1970s’ and n Negotiate a common set of require- 1980s’ notion of services integration and access the system, and learn ments and administrative procedures about its services. was coordination at one level of such a across programs. system — at the local service delivery n There was considerable data incon- n Design its service delivery strategies level. The modern boundaries of “sys- sistency across programs even in partnership with employers and tem” include the policy, planning, man- though measures were being collect- unions. agement and oversight levels as well. ed on similar variables. n Develop a data collection system ca- n There was a lack of objective infor- pable of tracking participants, the Editor’s Note mation about programs for manage- services provided to them, and their ment and oversight purposes. Infor- In a separate report produced in outcomes across programs, in-state mation systems did not allow February 1995, the GAO provided a geographical areas, and states. analysts to learn about relationships useful summary of its numerous reports and Congressional testimonies on the n Reduce administrative staffing for between client characteristics, ser- Department of Labor’s programs be- vices and outcomes. the system to a minimum. tween 1989 and 1995. The summary includes a review of DOL-funded pro- n n Develop clearly defined goals, mea- Only modest E&T gains for partici- cess and net impact evaluations of pants were identified in program surable objectives, and performance these programs. evaluations. standards.

96 Evaluation Forum n Issue 11 n Summer 1995 Evaluation Issues and Activities n At the National Level nn linguistics. As a result, these highly databases. Figure 1 lists these, as ex- Measuring trained individuals are taking greater cerpted directly from the report. The Performance in interest in one another, and each oth- authors propose that a training stu- er’s policies and strategies for estab- dent’s labor market status can be Programs and lishing accountability. Not unrelated to thought of as a joint outcome reflect- Coordinated this new accomodation, there is signifi- ing three sets of influences: the stu- cant movement toward a beginning dent’s qualifications, the skill require- Systems consensus about the meaning of perfor- ments of employers, and employment mance management. opportunities. The multiplicity of influ- This article focuses more specifical- ences underscores the conclusion that ly on some of the variables that are In this context, Stevens and Shi fo- important to measure in performance cus on a major issue in performance we should not expect any single cause- effect relationship to exist between vo- systems coordinating data across pro- management, the relationship between cational education and labor market grams. It also addresses major data vocational educational “treatments” or sources for obtaining such measures. interventions, and employment and outcomes. Two national reports are reviewed: one earnings. Several questions concern To reinforce this position, the au- emanating from the vocational educa- them: thors refer us to previous national as- tion system, one from the employment sessments of vocational education that n What does information about this re- and training field. Although these re- reached a similar conclusion. The Na- lationship tell us? ports were written at the beginning of tional Institute of Education’s 1981 Vo- extensive state-level experimentation n What variables should be given at- cational Education Study, for example, with comprehensive workforce devel- tention? stated that research results concerning opment systems, the general themes are this relationship could not be interpret- highly relevant to the measurement of n How should these variables be mea- ed as an assessment of the effectiveness performance in these systems, as well sured so that the indices are reliable of vocational education programs — as to programs within them. and valid? because of general data limitations, but more importantly because of the diffi- Beyond Performance Standards Some Useful Variables in culty in attributing either economic or by David Stevens and Jinping Shi Studying the Relationship noneconomic outcomes to participation of the University of Baltimore. Between Vocational Education in various kinds of vocational educa- and Employment/Earnings Prepared in 1994 for the National tion training. And the authors report Center for Research in Vocational Edu- The authors propose several catego- that the final monograph by the Nation- cation (NCRVE), which is funded by ries of variables that should be consid- al Assessment of Vocational Education the U.S. Department of Education, the ered, and which are often absent from in 1989 concluded that “institutional subject of this short paper is informa- tion production, expansion and coordi- nation. Traditionally, employment and Figure 1 n Vocational Educational Labor Market Outcomes training professionals and vocational education specialists, and those who evaluate these different sets of pro- Qualification Factors grams, have practiced in relatively sep- abilities arate worlds. Beginning in the late motivation 1980s, and now in the 1990s, Presiden- nonvocational education tial initiatives and federal legislation vocational education have been bringing these worlds to- work experience Observed Labor gether, albeit gradually and sometimes other Market Outcomes reluctantly. At the same time, new employment employment continuity/ workforce preparation councils and Requirement Factors to generate skills mobility boards have been struggling to bring other earnings these professional territories together in change in earnings a new way at the state level, at the Opportunity Factors industry affiliation(s) same decisionmaking table. entry-level hiring These new federal and state coordi- retention promotion nation efforts have informed each worksite learning group of experts about the other’s pro- continuing education fessional backgrounds, education, ex- perience, knowledge base, culture and

Evaluation Forum n Issue 11 n Summer 1995 97 Evaluation Issues and Activities n At the National Level

performance may reflect not only dif- Labor, in agreeing on a definition of outcome information, were drawn from ferences in program quality but also skills standards for industries. Two a four-state consolidated database de- differences in the types of students steps are involved: 1) defining the in- veloped in 1991 by The Jacob France served.” Sheets and Shi clearly seek dustry itself, and 2) deciding whether Center at the University of Baltimore. greater appreciation of these complexi- the standard will be based on one of The goal of this project was to illustrate ties on the part of managers and evalua- three kinds of thresholds — entry-level how wage record data could be com- tors. performance, the typical performance bined with administrative data from of those working in the industry, or state vocational education systems in Qualification Factors. These fac- leading-edge performance. order to conduct more extensive and tors refer to attributes employers tend useful management analyses. The to consider in making a judgment of a In reality, each type of definition is potential worker’s qualifications, in influenced by changing technology, job project was successful in raising man- agers’ awareness of ways they could terms of initial hiring, retention in em- availability and the qualifications of learn more about patterns in student ex- ployment, skills training, subsidized workforce candidates. Although UI education, or promotion. For example, wage record data can offer an opportu- periences following their participation in programs. evidence of having reached a certain nity to understand more clearly how educational threshold, such as posses- worker qualifications and employer re- Earnings Profiles sion of a certificate or degree, is a sig- quirements fit, an intervening variable nificant influence. The national skills in this relationship must also be consid- One example from the Center board, part of Goals 2000, is expected ered, namely opportunities. project was the identification of distin- to reduce dependency on program com- guishable patterns in earnings profiles, Opportunity Factors. Job avail- pletion as a threshold, substituting in- which corresponded to descriptions of ability and the nature of available jobs whether and when students experienced stead skill competency thresholds con- are two major opportunity factors. In a sistent with the establishment of progress on a career path — that is, in fast-paced economy, it is not possible, industry requirements and standards. jobs that involved “sustained affiliation the authors recognize, to align skill ac- with a single employer, and an observ- Another attribute may be an employ- quisition with its use when the time- able increase in annual earnings.” er’s recruiting strategy. The authors ac- frames are short. However, worker Stevens and Shi identified five pat- knowledge that “the value of an at- qualifications and employer require- terns: tribute as a discriminator among ments mean little if there are limited candidates falls as a fraction of those employment opportunities. n The traditional career path pattern: who offer the attribute rises.” However, immediate and sustained earnings they claim that in instances where few- Isolating Cause-Effect growth. er potential hires exhibit a particular Relationships from a valued asset, the qualification factor Complex Mix of n The delayed-start pattern: low ini- may enhance or diminish labor market Influences tial earnings following by sustained earnings growth. outcomes associated with the training The authors’ ultimate goal is to im- institution or program involved. Such prove vocational education managers’ n The short-circuited pattern: initial qualification influences complicate per- understanding of the mixture of influ- earnings growth followed by earn- formance measurement. For instance, ences affecting labor market outcomes, ings reaching a plateau. the use of UI wage record data in the and how these tend to define the rela- absence of additional information can tionship between vocational education- n The one-step improvement pattern: either help or damage a vocational pro- al interventions and employment and low initial earnings with discontinu- gram. The message is that where you earnings. The challenge, they say, is for ity, followed by a higher but flat are dealing with more than one influ- managers to have access to enough ap- earnings path. ence on multiple outcomes, the inter- propriate measures to permit them to n The dead-end pattern: flat or declin- pretation of data on relationships be- tease out the effects vocational pro- tween these variables must be carefully grams may be having on hiring and re- ing earnings over time. qualified. tention, and to merge this information The importance of distinguishing pat- Requirement Factors. The require- with a database on employment out- terns, the authors believe, is in their use ments of employers interact with the comes. The use of wage record data is in monitoring earnings over time. The qualifications of potential workers, analyzed in this context, and the au- emphasis in such monitoring is on un- making it difficult to understand causal thors’ conclusion is that it is a useful derstanding the “horizontal trajectory relationships between training and em- database for this purpose. of earnings over time,” rather than de- ployment outcomes. Sheets and Shi Examples of fruitful coordination of pending on single point-in-time snap- point to problems faced in the Skills databases, which include variables in- shots. The longitudinal approach per- Standards Project, funded jointly by fluencing employment outcomes plus mits managers to look both at levels the U.S. Departments of Education and and directions of earnings.

98 Evaluation Forum n Issue 11 n Summer 1995 Evaluation Issues and Activities n At the National Level

Cautioning that these earnings pro- with some of these issues in the process such as from other states, the federal files should not be treated as gospel, of encouraging the development of a government or educational institutions. the authors nevertheless propose that reliable, comprehensive wage record This missing data is important, the such ideal-type constructs contribute database. OTA report states, since analyses that usefully to the debate over the rele- compare the outcomes of educational vance of training-related placement Purposes Served by Wage and training programs can be misinter- standards, and over the timing and use Record Information preted in the absence of such data. Ob- of snapshot measurements of earnings. The OTA paper suggests four major taining data from other sources can be Neither do the profiles allow clear-cut purposes for UI wage data: complicated, however, requiring specif- cause-effect judgments to be made. ic data-sharing agreements that assure More must be known about a student’s n To conduct research on trends in the confidentiality. education subsequent to the vocational employment and earnings of work- To address the latter issue, the 1992 education treatments being examined, ers, and the employment effects of JTPA Amendments requested the Bu- or about the work-based learning ac- alternative strategies of income sup- reau of Labor Statistics (BLS) to devel- quired. port, education, training, and social op a plan for creating a National Wage Even so, the Sheet and Shi analyses services. Record Database (NWRD), which of data on the labor market experiences n To monitor and evaluate specific ed- would pool all wage records in existing of ex-vocational education students state unemployment insurance systems ucation, training, work/welfare and three and one-half years following their into a national database available to all completion of this education are very related social programs regarding states. Core data elements were to re- innovative. They alert us to important their effects on the economic wel- side in the states, or be sent to a central issues in data reliability and validity, fare of individuals, in order to make data bank operated by the federal gov- and to the advantages of consolidating improvements — that is, for perfor- ernment. The database was to consist of appropriate indicators for a range of mance management. a national sample of individuals, sam- critical variables in better understand- ples within states, or the entire working n To provide the public with informa- ing the effects of training programs. population. tion about the outcomes of pro- Wage Record Information grams, and employment trends in Both the cost of maintaining a na- Systems. Office of Technology different industries, occupational tional database and the confidentiality Assessment, U.S. Congress. sectors, and geographic areas that challenge remain a concern, according support their decisions as customers to the OTA report. The BLS has con- Last year the Office of Technology sidered proposing a statutory strategy of service. Assessment (OTA) reviewed efforts by for resolving the privacy problem. Or the National Occupational Information n To enforce compliance with policies data confidentiality might be assured if Coordinating Committee to encourage concerning eligibility and payments wage record information were classi- the use of UI wage records for conduct- in other social programs. fied as statistical rather than adminis- ing policy studies and monitoring the trative information. Another concern outcomes of vocational education. The These purposes can be served by UI discussed in the report is whether the Committee had previously designed a data universally, the report says, since development of such a database would series of demonstrations to explore the 90% of the U.S. working population is undermine the primary purpose of the technical problems in producing and included in the UI wage record data- UI information system in the eyes of using wage records for this purpose. base. And three key data elements — employers and the public. Beyond these Subsequently it was to develop proce- the SSN of all employees covered by issues, there is some anxiety that the dures for maintaining a nationally-ac- UI, their quarterly earnings, and the in- use of a national database for enforce- cessible database for wage record infor- dustrial code or business name/address ment purposes might erode its function mation. The Committee also contracted of the employer — are collected by all for performance management and eval- with MPR Associates to design a train- states. Furthermore, longitudinal earn- uation. ing guide to assist states in using wage ings histories can be constructed be- record data for studying the outcomes cause state agencies accumulate em- The Level of State Use of vocational education and other pro- ployment and wage data over time. of UI Wage Records grams. Obtaining Additional Under the 1988 amendments to the The purpose of this OTA research Employment and Deficit Reduction Act, states were re- paper is to summarize what has been Wage Data quired to collect quarterly wage reports learned from the demonstrations and from all employers as part of a new In- from other studies, to identify data For individuals not included in state come Verification and Eligibility Sys- quality issues, and to make suggestions UI information systems, information tem. This vastly increased the number about how the Congress could deal must be obtained from other sources, of data-sharing agreements. The Na-

Evaluation Forum n Issue 11 n Summer 1995 99 Evaluation Issues and Activities n At the National Level

tional Commission for Employment start-up costs were considered to be they would otherwise — that is, Policy reported a 50% growth in such higher than long-run average costs, and they may classify themselves in- agreements between 1986 and 1991. there was considerable uncertainty correctly as belonging in indus- about what cost categories were includ- tries or jurisdictions with the low- The program providing the greatest experience in using UI wage records to ed in these estimates. Also, no esti- est tax rates. mates were made of the cost of entire track outcomes has been JTPA, the u Employers may not report non- management information systems of OTA report concludes. The use of wage payments as accurately as JTPA data was a sufficiently important which wage records were a part, to de- they do wages and salaries. termine what proportion of the costs option that the U.S. Department of La- the wage record portion represented. u Employers may report severance bor funded demonstration projects in payments to dislocated workers 1993-1994 to study this use vis-a-vis One study reported by OTA, con- only after these workers are ter- follow-up questionnaires for judging ducted by the National Commission, minated from employment. compliance with JTPA performance compared the costs for UI tracking with standards. the JTPA follow-up system using ques- u UI system policies may change, tionnaires. The estimate for the JTPA leading to changes in data defini- Meanwhile, in Florida’s Education tions and measures that are poor- and Training Placement Information telephone survey was $19/per complet- ed interview. In this respect, the Florida ly communicated. Program, UI earnings data were being system provided a reasonable bench- combined innovatively with data from n The “matching rates” for the indi- the state’s community college and uni- mark regarding the cost of comprehen- viduals tracked in the UI system sive information systems — approxi- versity systems, the public schools, the may be too high or too low. mately sixteen cents per person tracked, state corrections system, and federal agencies. Employers were also sur- based on tracking 1.8 million people. n UI data may need to be obtained Clearly there are large economies of veyed to determine the occupational from multiple sources, with the pos- scale. categories of all students tracked within sibility of inaccuracies. the system. A common set of outcomes The OTA report asks us to view The choices made about the number measures was used: quarterly earnings; costs in the context of the length of and kind of sources used can affect the weeks worked; occupations; further en- time individuals are tracked. Attrition “matching rate”, or “the percentage of rollment in adult education, community in survey samples increases with the all persons being tracked for whom college or higher education; and move- length of follow-up. Such problems are some positive identification of employ- ment into the military. The consolida- virtually eliminated by the requirement ment, unemployment or other activity tion permitted analyses of outcomes for in UI for a quarterly submission of appears in the outcome data.” This rate, all state education, training and em- earnings information. And the difficul- the OTA report stated, is analogous to ployment programs. This comprehen- ty in locating members of the follow-up the response rate in questionnaire- sive data bank has become a basis for sample using the survey raises costs. based surveys. The higher the matching state strategic planning and budgeting. The Commission estimated that the UI rate, the more trustworthy the results of In 1992, the National Governors’ method cost only about one-fifth as data analyses. much as the survey strategy. Association reported that 20 states had The OTA report says there are no explored the use of UI wage data for Quality Problems with UI Data clear-cut rules for deciding what the tracking vocational education students, matching rate needs to be in order to and 12 were using these data on a regu- But the world of information sys- avoid bias. The suggestion is that this tems is never perfect. The OTA report lar basis. Washington State, for exam- determination will require time, the ac- takes the position that administrative ple, had combined 1) wage record data cumulation of state experience with dif- across five states within its region with sources of data are more difficult to use ferent configurations of data for differ- and interpret than data from surveys 2) enrollment data from the state’s uni- ent purposes, and specific studies where samples of respondents can be versity system, and 3) employment in- comparing UI data with data from other formation from the U.S. Department of randomly selected and response catego- sources. Defense. ries are under the control of the analyst. And there are some other problems. Issues to explore are differences in Some states had also estimated These were the major ones to which the 1) the rates of unemployment among costs. In reviewing states’ information, report gives attention: the group tracked, 2) where these indi- the National Commission estimated av- viduals are employed, 3) how they earn n erage development/operation costs for Wage record data are subject to sev- their income, and 4) the likelihood of a wage record information system as eral kinds of systematic errors. their acquiring education, joining the approximately $1.75 per person u Employers’ tax liabilities may armed forces, going to prison, or en- tracked. In vocational education, one cause respondents to report dif- gaging in another activity. People who expert estimate was $3/person. But ferently to the UI system than receive UI payments can be positively

100 Evaluation Forum n Issue 11 n Summer 1995 Evaluation Issues and Activities n At the National Level

identified. Those who do not receive experience — all of which are relat- cantly between wage record and sur- them cannot be. More vocational edu- ed to employment and earnings. vey data. cation graduates continue on to further This information can come from n Earnings estimates may be different education than do JTPA completers, for program Management Information for different client groups. Low-in- example, and therefore the latter’s Systems, but dependence on these matching rates are higher. come, disadvantaged populations information systems would tend to appear to have more earnings from To increase the matching rates for limit wage record studies to popula- unreported sources than dislocated different groups, the number of sources tions of individuals for whom there worker populations, for example. for outcome data must be expanded, the is such data. Without such informa- Systematic differences may then report suggests. With vocational educa- tion, employment and earnings tion students, for instance, outcome persist in the accuracy of earnings trends emerging in analyses may be data can be requested from neighboring estimates for these groups. due more to changes in the composi- states, the home state’s higher educa- n Differing earnings estimates can be tion system, and the federal govern- tion of the labor force than to pro- related to the scientific sophistica- ment. Some experts say a matching rate gram interventions. And the differ- of 90% is needed. It is clear that a per- ences in outcomes may be more tion of follow-up surveys: sampling formance management system based related to differences in the charac- procedures, questionnaire construc- only on wage record data from one teristics of program participants than tion, interview formats, and methods state could be very misleading. to program treatments. In terms of of analysis. The report also points out that local performance management, the OTA n Wage information recordkeeping in changes can strongly affect the results reports says, this lack of adequate the UI system can be a source of er- of analyses “when comparisons are data could lead to “creaming.” ror. No large-scale audits of the made among specific institutions and n Wage records do not distinguish be- system have been conducted, there- programs rather than among statistical- fore the nature of errors will not be ly-selected samples of individuals or tween part-time and full-time em- known until the large BLS audit is institutions covering much larger ar- ployment. There can be large differ- completed. But the OTA analysts eas.” ences in the earnings estimates, the OTA report states, due to very small think it is likely that the quality of n Earnings data are affected by the fluctuations in the proportion of in- recordkeeping varies across states. quarter selected for reporting them. dividuals working part-time rather Some states compute annualized than full-time. If “weeks worked” is Some Conclusions earnings. If the first quarter of the included as a data element, analysts The OTA report concludes, as do year is used as the basis, actual earn- can restrict their studies to full-time most researchers, that “bad data may be ings will be overstated. Earnings are workers. However, only seven states worse than no data.” They recommend more accurately calculated if the were collecting this element in 1994. that studies look at the relationship be- fourth quarter is used. Therefore, in tween data quality and system design, performance management systems n There are differences in the accura- as well as the other issues identified. cy of UI records vis-a-vis survey in- where earnings are estimated on the The report offers recommendations formation. The New Jersey Unem- basis of the first quarter’s worth of relevant to BLS’s plan for the creation earnings, the results of analyses are ployment Insurance Re-employment of a national wage record database. likely to punish those programs with Demonstration Project found that These are useful in framing information the larger proportion of graduates/ earnings estimates for a sample of production issues. Given that the pur- completers. Estimating earnings dislocated workers were higher if pose of a national database would be from four quarters of data, or adjust- based on UI data on the first quarter policy research, program evaluation ing for the quarter used could yield a after unemployment, than if based and consumer information, the report very different result. on surveys. In the second and fourth includes these suggestions: quarters, they were lower. The high- n Congressional review of states’ ex- n Analyses of wage data cannot con- er figures in the former case were periences with the UI wage record trol for program effects due to due to severance payments; the low- system, the improvement of state differences in the background char- er figures were due to a growing studies using UI wage data in terms acteristics of individuals. Wage number of workers occupying out- of their quality and utility, and the records do not include information of-state jobs. The National JTPA use of these data within performance on demographic characteristics, so- Experiment found the same phenom- management systems. cial background, or years of work enon. Estimates of earnings of low- n income individuals differed signifi- Improvements in the federal govern- ment’s assistance to state user agen-

Evaluation Forum n Issue 11 n Summer 1995 101 Evaluation Issues and Activities n At the National Level

cies in utilizing wage record data in Editor’s Note Employment and Earnings Dynamics in comprehensive workforce develop- the 1990s: Policy Issue Needs and ment systems. For more detailed information about Potential Uses of Nationally Ar- some of the references mentioned, as chived UI Wage Records by C. King. n Development of federal/state dem- well as related publications, please see Washington, D.C.: Northeast/Mid- onstration projects to encourage the the following: west Institute, 1989. development of regional consortia The Vocational Education Study: The A Feasibility Study of the Use of Unem- that can promote the use of wage Final Report. National Institute of ployment Insurance Wage-Record Education, Vocational Education record data. Data As An Evaluation Tool for JTPA Study Publication 8, Washington, by J. Baj et al. Washington, D.C.: n Conducting of systematic research at D.C.: U.S. Department of Education, National Commission for Employ- the federal level on the use of wage 1981. ment Policy, 1991. record data in policy studies and National Assessment of Vocational Ed- Schools and Labor Market Outcomes performance management systems, ucation — Final Report, Volume I, by D. Crawford et al. National Cen- including program-specific and Summary of Findings and Recom- ter on the Educational Quality of the mendations by J. Wirt et al. Wash- Workforce. Philadelphia, PA: Univer- workforce system-oriented evalua- ington, D.C.: U. S. Department of sity of Pennsylvania, 1992. tion. Education, 1989. State Systems for Accountability in Vo- cational Education by M. Rahn et al. Some Comments on Using State Unemployment Insurance Wage Records to Trace the Subse- National Center for Research in Vo- the Two Monographs quent Labor Market Experiences of cational Education. Washington, Interest in strategies for establishing Vocational Education Program Leav- D.C.: U. S. Department of Educa- accountability in public programs and ers by D. Stevens. National Assess- tion, 1992. systems has never been more intense. ment of Vocational Education. The School-to-Work Transition of High Despite barriers to coordination and in- Washington, D.C.: U.S. Department School and Community College Vo- of Education, 1989. tegration, the federal government, the cational Program Competers: 1990- states and local communities are mak- Using State Unemployment Insurance 1992 by D. Stevens. National Center ing strong efforts in the 1990s to col- Wage Records to Construct on the Educational Quality of the laborate, coordinate and consolidate Measures of Secondary Vocational Workforce. Philadelphia, PA: Univer- sity of Pennsylvania, 1993. workforce development programs, and Education Performance by D. Stevens. Washington, D.C.: Office of to incorporate within that movement a State Capacity to Use UI Wage Technology Assessment, U.S. Con- Records: The Vocational Education solid notion of how to judge process gress, 1989. and performance. Experience by L. Amico. Washing- Systematic Bias in Earnings Data De- ton, D.C.: National Governors’ Asso- Whatever the general approaches rived from Unemployment Insurance ciation, 1993. taken in establishing accountability, the Wage Records and Implications for adequacy, quality and accuracy of the Evaluating the Impact of Unemploy- information systems on which perfor- ment Insurance Policy on Earnings mance management judgments are to by P. Decker. Princeton, NJ: Mathe- be based are critical issues. These two matica Policy Research, 1989. reports are useful in understanding Policy Evaluation and Archived Wage some of these issues. They also force us Record Data: Limitations of Existing to consider the potential imperfections Data Sets by J. Bishop. Washington, in whatever direction we take in inte- D.C.: Northeast/Midwest Institute, grating information production, and en- 1989. courage us to seek ever-better systems for making essential decisions.

102 Evaluation Forum n Issue 11 n Summer 1995 Evaluation Issues and Activities n At the National Level nn can be made confidently about the ous controlled impact evaluations. This More About quality of policy and program out- conclusion is consistent with other cur- Performance comes. rent definitions of the boundaries of performance management systems — Management Defining Accountability i.e. they tend to isolate net impact eval- Reviewed in this article are two The authors’ definition of perfor- uation from performance management. guides for developing performance mance accountability is at once useful: Characteristics of management systems: Getting Results: “ a means of judging policies and pro- Accountability A Guide for Government Accountabili- grams by measuring their outcomes or Systems ty published in 1991, and A Guide to results against agreed-upon standards.” Strategic Planning for Performance In this sense, accountability efforts pro- The authors distinguish also be- Measurement, Reporting and Improve- vide a framework for measuring out- tween policy accountability and pro- ment Systems produced in 1994. Devel- comes as well as processes, and gram accountability. Policy account- oped by the Council of Governors’ Pol- organizes this information “so that it ability is viewed as giving attention to icy Advisors and the National Institute can be used effectively by political the outcomes for whole groups or for Literacy respectively, these guides leaders, policymakers and program classes of people, not only for those offer assistance to policymakers, plan- managers.” Accountability systems are who have participated in particular pro- ners and oversight staff in developing to provide feedback of information to grams. Program accountability looks at viable, useful accountability systems. these important groups so that policies outcomes only for those served by a Both guides direct their suggestions to and programs can be adjusted when program or group of programs, and at state governments. needed. And they are to disseminate how programs are operating. Brizius useful information to service providers, and Campbell describe usefully the key The Council of Governors’ Guide service customers, and the public. elements of state performance account- Authored by Jack A. Brizius and ability systems, in the ideal: Developing Accountability Michael D. Campbell, Getting Results n Systems A focus on outcomes. was funded by the U.S. Department of Health and Human Services, and in- The authors make the usual observa- n The use of a few selected indicators volved an array of experts on perfor- tion that performance accountability to measure performance. mance management. The authors ex- differs from traditional assessments of n The generation of data over time. plain that the guide answers a number “inputs” by focusing attention on of basic questions: changes in people’s lives. However, n Continuous reporting of outcomes. they do not neglect the importance of n What is a performance accountabili- n The use of performance information studying program implementation as an ty system? to create incentives to service pro- integral part of performance manage- viders to improve outcomes. n What is the political and policy con- ment. Nevertheless, they emphasize the text within which accountability sys- establishment of performance standards n The use of performance information tems must be implemented? which they contend tie desired out- as a basis for developing sound comes to longer-term program goals. questions for evaluation purposes, n What are the key components of a These standards, they say, must be real- and for conducting competent evalu- workable accountability system? istic estimates of expected outcomes ations. which have been jointly developed by n What are the fundamental queries those judging performance and those n The utilization of performance infor- that must be answered in defining being held accountable. Some criteria mation in making policy and pro- such a system? for system development are: gram management decisions, and n What are the specific actions that n The amount of resources used. improving policies and programs. need to be taken in implementing n n The testing of policy premises. this kind of system? The baseline skill levels or problems of those served. These benefits of performance ac- n What are the problems and pitfalls countability, or performance manage- in making comparisons and report- n The intensity of a service or treat- ment. ment systems, are qualified by some ing performance? limitations. Again the authors take the n n How can accountability systems be The power of the incentives used to position that such systems are not capa- institutionalized within state govern- produce change. ble of determining the ultimate impact of programs and policies on people. ment? It is to the authors’ credit that per- Because impact evaluations are diffi- formance accountability systems, as The appendix of the guide suggests cult and expensive, they say, net impact a step-by-step progression toward com- they define them, are not the end-all in evaluations are “seldom used as man- judging policies or programs. Judging pletion of a design for an accountability agement tools.” Therefore perfor- effectiveness, they say, requires rigor- system on the basis of which judgments

Evaluation Forum n Issue 11 n Summer 1995 103 Evaluation Issues and Activities n At the National Level

mance management systems cannot states not to develop formal standards The National Institute for definitively pronounce policies or pro- prior to the application of a policy or Literacy’s Guide grams failures. The author’s persistence the operation of a program — a wiser While the Institute’s monograph is in viewing evaluation as something process is to develop and test initial directed toward developing accountable apart from performance management is standards against actual program per- state adult literacy systems, particularly an unfortunate and possibly misguided formance and adjust these standards the states that participated in the Acad- conclusion, since in general net impact based on experience with their use. emy on Performance Accountability evaluations produce the most accurate Apart from premature use, performance held in Washington, D.C. in 1993, the information for both policy and man- standards have been greeted with insights are widely applicable. The agement use, and are not always expen- mixed reviews because they have often guide is to assist states in establishing sive. been linked to funding decisions. Also, statewide outcome-oriented perfor- local economic and political conditions More to the point, accountability mance measurement, reporting and im- clearly affect performance but are not systems are considered by the authors provement systems, formally referred to to be limited by data quality and the considered realistically in the context as PMRISs. Their elements are generic. of uniformly-applied standards. accuracy of outcome measures, prob- A PMRIS measures outcomes, re- lems that have a significant influence Some Conclusions ports this information to key actors, uti- on evaluation activities as well. The au- lizes it to continually improve state pol- thors also recognize that accountability The authors are well aware that ac- countability systems can have short icies guiding programs and the quality systems can involve disincentives on of those programs. Its structural core is the part of service providers to respond lives if their advocates are not careful an interagency team composed of di- directly to the needs of clients given in designing and using them, and in mediating traditional objections to per- rectors of relevant statewide programs. their situations and circumstances. This The team is the vehicle for making sure is particularly true, they say, when ac- formance measurement. When account- that outcome information is collected countability systems are linked closely ability systems identify problems, there will be natural pressures to dismantle and reported, that it addresses policy to the allocation of resources. resource allocation, management and them. The Design of Systems service provider needs, and that coordi- A number of strategies are suggested nation across programs is maintained. The authors explain the design of for maintaining these systems. The cen- The authors outline a series of steps in performance accountability systems as tral theme is ensuring that key stake- constructing a PMRIS, which are pre- answers to five key questions: holders have access to and use account- sented in Figure 1. ability information: political leaders, n Are the goals and objectives of the In this scenario, a vision statement is policy staff, legislators, and legislative policy or program stated clearly? staff, program administrators, service a preferred future that explains the con- ditions and life quality to be achieved n How can progress toward these providers, consumers of service, the in the state. Benchmarks are measur- press and the public. goals be measured? able goals for the state as a whole, n How can incentives be accomodat- ed? Figure 1 n Accountability Essentials n How will the data be reported and

used? 1. Define vision and benchmarks. n Apply quality standards to existing 2. and new programs. n How can policy development and Define policy outcomes. 3. 8. program management be coordinat- Identify performance measures for Collect data & communicate results: ed to promote accountability? policy outcomes. n Define the audiences. 4. Identify program outcomes, related n Develop sample reports for differ- Realistic, measurable goal statements to policy outcomes. ent audiences. are an essential first step. Policy goals, 5. Identify program performance n Negotiate data collection protocols. in the guide, are essentially system- measures. n Collect the data. wide goals. Program objectives are 6. Identify target populations, related n Report the results. more specific statements with shorter to policy and program outcomes. 9. Use information from PMRIS to im- 7. timeframes. Developing indicators of Compare existing service delivery prove program interventions. progress toward goals and objectives is patterns with service needs: n Link inputs to processes. the most difficult step, according to n Determine what program interven- Brizius and Campbell. Such indicators n Identify constraints and enabling tions are necessary to achieve influences. should include both process variables outcomes for target groups. n Plan for continuous improvement. and outcome variables. Performance n Determine whether existing pro- 10. standards then rely on the selection of gram can achieve these out- Use information from PMRIS to these indicators. The authors caution comes. adjust resources and policies.

104 Evaluation Forum n Issue 11 n Summer 1995 Evaluation Issues and Activities n At the National Level

nn consistent with the vision. Benchmarks tions of performance systems is the ex- permit the tracking of progress toward tent to which they work in practice. Using Performance the vision over time. A rigorous defini- Historical attempts to coordinate sepa- Standards in tion of vision and benchmarks, howev- rately-funded programs providing com- er, is not viewed as a necessary condi- parable interventions to comparable Establishing tion for defining policy outcomes — client populations have had mixed re- Workforce System even though the latter need to be gener- views. The new view of performance ally consistent with these abstractions. management depends on such coordi- Accountability The authors are insistent, however, that nation. Relevant to this issue is the Na- There is a high level of agreement policy outcomes be quantifiable. tional Commission for Employment among workforce development experts In logical sequence, defining pro- Policy’s 1994 position paper “Guiding that a key element in establishing ac- Principles for Change,” by Anthony P. gram outcomes is a next step, consis- countability in new state-wide work- Carnevale, the Commission’s chair. tent with policy outcomes. These out- force preparation systems is the devel- comes are defined by the authors as the The paper explores some of the coordi- opment of performance standards nation problems sustained historically, statuses to be achieved by programs consistent with system goals and de- and makes the following recommenda- and their participants if policy out- sired outcomes. Unfortunately we lack comes are to be achieved. Performance tions for national-level efforts to increase a history of experimentation to rely on coordination in the context of national measures are seen as both quantitative for insights regarding the appropriate- and state efforts to build accountable and qualitative indicators of such out- ness of setting standards for “systems” comes. These measures are considered workforce preparation systems: of programs. Even our exploration of the central element of the PMRIS. Per- n A national dialogue is needed on the use of performance standards in formance measures are of three kinds, systemic reform of the national edu- separate major ongoing national pro- the authors say: results-oriented mea- cation and training systems. grams is recent. sures; measures of program quality; n The consolidation of programs serv- The longest running experiment is in and measures of participant character- JTPA. What we have learned about the istics. These process and outcome mea- ing similar target groups should be implications of using performance stan- sures are clearly the building blocks of endorsed. dards in JTPA should be enlightening any performance management and n Two interim steps for enhancing co- regarding the tradeoffs we may experi- evaluation system. ordination should be implemented: ence in applying standards to state Some Comments on Other 1) the establishment of a central workforce development systems. Performance Management waiver authority outside the Execu- One of the most astute analyses of Approaches of the PMRIS tive Branch, and 2) efforts to make the utility of developing and applying basic program requirements compat- performance standards is Burt Bar- National-level conceptualizations of now’s chapter on “The Effects of Per- performance management systems are ible. formance Standards on State and Local quite similar in their basic elements and n Broad planning and oversight re- Programs,” in Evaluating Welfare and meaning. They have given considerable sponsibilities at the federal, state and Training Programs edited by Charles direction to states at a time when states local levels should be promoted. themselves are recognizing the need to Manski and Irwin Garfinkel in 1992. The book began as separate but related integrate related programs and assure n Information about “best practices” papers commissioned for a national the accountability of multi-program regarding planning and oversight systems. Most performance manage- conference on evaluation methodology functions and tasks should be made by its cosponsors, the Institute for Re- ment concepts include measures of pro- available to state and local adminis- search on Poverty at the University of cess and outcome variables, even trators. though the systems are outcome-driven. Wisconsin-Madison, and the Assistant Secretary for Planning and Evaluation This moderates the pendulum swings of n More flexibility should be allowed in the U.S. Department of Health and the past several decades in which either in the development of state-wide process or outcome variables were the and local-level planning councils or Human Services. It is important to con- sider Barnow’s position on perfor- exclusive focus. This recognizes the boards. mance standards as we move closer to strong interrelationship between 1) pol- icy and program implementation struc- n Funds for state/local capacity-build- national employment and training poli- cy supported by performance standards, tures and processes, and 2) the shorter ing should be provided. and to state-level workforce systems at- and longer-term outcomes and impacts n Comprehensive cross-program man- of related policies and programs — and tempting to include standards in their agement information systems performance management subsystems. the significance of this relationship in should be developed. judging policies and programs. Some Background n A UI wage record database, includ- The test of these well-conceived and The mandate to develop perfor- logically explained ideal-type construc- ing all states, should be developed at the national level. mance standards for major national

Evaluation Forum n Issue 11 n Summer 1995 105 Evaluation Issues and Activities n At the National Level

programs, as a major tool in the ac- programs that meet or exceed stan- ness (does the program achieve its pro- countability kit, began in the 1980s. It dards — such as through more pro- grammatic goals for those it is intended now applies to JTPA, JOBS, and the gram flexibility or funding — and to serve?). Food Stamp Employment and Training sanctioning them for failing to do Performance management systems, Program. Performance standards were so. Sanctions may be mild, such as as theoretical constructs, have tended to to involve specific measures of desired simply an absence of praise, to quite involve three key “accountability” outcomes, tempered by lessons from harsh, such as a loss of funds. functions: program review (against previous program evaluations regard- plans), program monitoring (for com- ing the actual level of success employ- Barnow wants us to distinguish pliance with requirements), and pro- ment and training programs were hav- among three different concepts in talk- gram evaluation (to judge the efficient ing in increasing the employment and ing about performance standards: per- achievement of program goals). But all earnings of different participant groups, formance management, the achieve- of these constructs emphasize the con- — and in the case of JOBs and Food ment of economic efficiency, and tinuous feedback of evaluative infor- Stamps, in reducing participants’ de- evaluation. mation for continually improving pro- pendence on the welfare system. Performance management is con- grams, an emphasis missing till In the Manski/Garfinkel book, Bar- cerned with the achievement of an ar- recently in the evaluation research liter- now first defines E&T performance ray of program goals, Barnow says, in- ature. management, then outlines state and lo- cluding equity. It is an ongoing cal objectives and behavior vis-a-vis oversight activity that provides con- The Need for Performance standards, and ends with his view of tinuing feedback to policymakers, ad- Management Systems how performance is measured, rein- ministrators and planners about key Barnow’s rationale for developing forced and rewarded. In the process we program processes and shorter-term formal performance management sys- see some cautions emerging. program outcomes. Performance stan- tems is significant. Briefly, such sys- dards, within performance manage- tems 1) identify poor performers and The Performance Management ment, do not include impact measures, therefore permit corrections to be Concept Barnow states, and are less intrusive made, 2) locate high performers and re- Barnow defines performance man- than evaluations in terms of disturbing ward them, thus helping to define “best agement as “a system whereby pro- program operation. However, perfor- practices” and providing evidence that grams are systematically judged against mance management systems lack a there is organizational profit in apply- specific objectives.” The following are clear-cut “bottom line” (such as prod- ing them, and 3) maintain efforts in the the main interrelated “parts” within the ucts sold) similar to variables that can direction of accomplishing the goals performance management “system:” be monitored in private sector activi- originally intended for the program. ties. Identifying the ingredients in com- n Performance measures: indicators mendable performance also provides a of the performance of the entities Evaluation, Barnow proposes, fo- cuses on program impact through sin- practical basis for replicating those at- being judged. gle snapshots in time, using quasi-ex- tributes. Furthermore, sustaining con- n Performance standards: specific perimental or experimental research sistency with program goals reduces standards associated with each mea- designs. Economic efficiency remains natural tendencies to redirect program sure or indicator. undefined, and its relationship to per- activities toward other agendas, such as formance management and evaluation those responsive to political, cultural or n A standard-setting method: the is left unclear. organizational pressures. method for setting standards may be based on statutory regulations, man- These distinctions are not the tradi- Important Kinds of Performance tional ones. The concept of “perfor- agerial goals, performance levels in Measures to Consider mance management system” often cov- other programs, or on other factors ers both program monitoring and Barnow identifies a range of vari- — or on arbitrary decisions. The program evaluation, and involves some ables for which specific measures most useful, and perhaps the fairest, effort to use measures of economic effi- should or could be developed within Barnow claims, is a rational method ciency in the process. Program evalua- performance management systems: that is understood in the programs tion typically covers judgments of pro- Gross Outcomes being judged, even though standards gram organization and operation, can be set to take account of factors program outcomes, net program im- Gross outcomes are measures of a that vary with the characteristics of pact, and the tradeoff between costs program participant’s status at the time the program being judged. and benefits. In this sense it involves a he or she exits from a program, or study of both program efficiency (do shortly thereafter, vis-a-vis certain de- n A rewards and sanctions system: the program’s outcomes warrant its sired outcomes — educational, skill such a system is a way of benefiting various costs?) and program effective- training, employment, welfare reduc-

106 Evaluation Forum n Issue 11 n Summer 1995 Evaluation Issues and Activities n At the National Level

tion, etc. Exit status could mean at the education, preemployment, job-specif- properly combined with outcomes mea- completion of the services assigned, or ic), which are difficult to quantify. The sures, it may “promote underinvesting simply at the point participants leave advantage of these measures, however, in participants and serving participants the program for various identified and is that net earnings gains, can be con- who require less training.” Omitting unknown reasons. Barnow says that sidered to reflect the market value of costs from the performance manage- most of the outcomes measured in cur- varying kinds of training. ment system avoids a bias toward rent national E&T programs are gross shorter and less intensive programs. The problem is that different earn- outcomes, because they do not neces- ings gain measurement strategies may The tradeoffs are difficult to reconcile. sarily reflect program gains or losses. produce disparities in earnings gain es- Combining Performance Outcomes such as completion of ed- timates, leading to a mistrust of this Measures into an ucation, completion of skills training, version of program performance. Also, Overall Index placement in a job, and increased earn- using the determination of earnings ings in employment are standard pro- gains as the most prominent way to Barnow proposes that, in theory at gram outcomes in E&T programs. If judge a program’s achievements re- least, it is possible to develop a single we were not able to compare the quali- quires quasi-experimental or experi- overall measure of the economic gains ty of educational, skills training or em- mental net impact evaluations, which from E&T programs, such as “the ployment outcomes for program partic- are not always appropriate or possible. present value of all earnings gains re- ipants with comparable people who Furthermore, Barnow cautions that pri- sulting from the program, or the were not exposed to the program, we mary reliance on a quantitative eco- present value of earnings gains plus would designate these outcomes as the nomic measure of performance runs the welfare savings.” While useful, this di- program’s gross outcomes — meaning risk of precluding a consideration of rects attention to a single indicator of that we are unable to label these as bet- significant program outcomes that are performance, ignoring other outcomes. ter (a gain) or worse (a loss) than the less tangible but may be as important or To avoid excluding other program situation similar people would experi- more so. goals, such as service equity, perfor- ence in the absence of the program. If mance measures can be developed Inputs and Process Measures we were able to make the comparison which hold programs to more of their indicated, we could then calculate the Barnow views input measures, such promises — for example, measures of program’s net outcomes (by subtracting as the characteristics of participants, as the participation levels of client groups any loss or adding any gain). These net part of performance management sys- of special interest — which can be in- outcomes are usually referred to as the tems. Process variables describing how corporated within performance stan- program’s net impact. service delivery is organized, what pol- dards (as minimum levels of participa- The main thrust of Barnow’s argu- icies and practices are involved at each tion.) Outcome measures can be ment is that gross outcomes are rela- step in the service pathway, how many tracked for different groups, and a dif- tively easy to collect and analyze, and and what kinds of clients experience ferent set of minimum standards devel- therefore constitute an efficient part of what steps, and what the “culture” of oped for each. Employment retention any performance system. He qualifies the program is like (staff attitudes and may be a higher priority in some pro- this by pointing out that performance behaviors, for example) are clearly in- grams than earnings gains. The reduc- standards that rely on gross outcome cluded in a competent performance tion of welfare rolls may be more im- measures must adjust for differing cli- management system. As an expert in portant than any other outcome. ent profiles. Otherwise programs will designing and conducting net impact Performance measures, and ultimate be sanctioned for not meeting perfor- evaluations, it is understandable and standards, Barnow suggests, can be de- mance standards, even though the char- important that Barnow suggests that ex- veloped so as to permit different expec- acteristics of certain subgroups within clusive reliance on inputs and process tations for program performance. the client population make it unrealistic variables is a mistake. The challenge in the use of multiple to hold these subgroups’ gross out- Measures of Costs measures, he says, is in linking a clus- comes hostage to the same levels of ter of measures together such that an performance as one does the others. Performance management systems overall minimum level of performance must, Barnow believes, consider the can be established, and a reward/sanc- Net Outputs use being made of resources. Identify- tion system developed. A simple meth- Barnow is referring here to indica- ing the total dollars spent in achieving od is to count all performance measures tors of gains, such as in education, key outcomes is one approach. Or costs equally and sum the scores. But Bar- skills, work experience, wages, and job can be reported for each major out- now warns us that adding scores may retention. He claims that developing come, such as cost per entered employ- be undesirable. Some measures may be such measures will require a compari- ment. Separate cost measures give at- considered more important than others. son of different kinds of program inter- tention to unit costs, an approach Some may involve more variation than ventions, or treatments, such as differ- Barnow thinks is very useful. But he others, and therefore have to be weight- ent types of skills training (basic cautions that if the cost measure is not ed differently. An alternative is to

Evaluation Forum n Issue 11 n Summer 1995 107 Evaluation Issues and Activities n At the National Level

group performance measures and re- more difficult to deal with. Barnow for creaming, JTPA service delivery ar- quire programs to meet a certain num- says “if the gains in earnings accruing eas (SDAs) which gave priority to ber of standards in each group. to participants are at the expense of goals other than achieving high scores other workers, then not all the private on performance standards were able to A Rewards and Sanctions gains are social gains.” resist biasing the program enrollment System process. The most prominent of these Barnow says that giving substitution other objectives was the desire to serve Another important component of a effects due consideration involves ad- performance management system, re- dressing two types of equity issues: certain types of clients, and to respond ward/sanction systems, also has its to the needs and interests of employers. complexities. The least complicated 1. The value placed on successful out- Barnow believes that creaming is de- strategy is to offer praise for successful comes for particular subgroups of creasing with the increased use of state compliance through certificates or an clients. Earnings gains realized by adjustment models for developing stan- honor roll. A more energizing strategy certain kinds of participants may be dards, and with recent changes in both is to provide additional funding, ac- viewed as more valuable from a so- performance measures and adjustment companied by more flexibility in the cietal perspective than the same models, and the dropping of cost stan- dards. use of funds. A monetary reward is gains experienced by other clients. more consistent with private sector sys- But clearly there are questions re- 2. The value placed on a broad vs.a tems. maining. Barnow sees several worth more narrow service delivery re- policy attention: Inadequate performers, Barnow sug- sponse. Programs serving a large gests, could lose the privilege of oper- number of eligible people but with n Whether the JTPA performance ating a program, or the federal funds limited services may be considered standards system promotes the maxi- could be reduced. Or program adminis- more valuable than programs serv- mization of earnings and employ- trators could be required to accept re- ment gains and reductions in welfare medial technical assistance, as in JTPA. ing a smaller number but with a transfers. Tying the level of rewards to the level more intensive mix and sequence of at which standards are exceeded is an services. n Whether it achieves the goal of option already mandated in some feder- These issues complicate the devel- holding SDAs harmless for serving al legislation. However, funding re- opment of performance standards, since individuals with different character- wards and penalties may actually be they involve policy decisions laced istics. counterproductive, Barnow says, if with value judgments. Nevertheless, more funding leads to new responsibili- n Whether it should lead to indiffer- Barnow recommends that such issues ence on the part of SDAs regarding ties or declining funding denies servic- be studied in developing performance es to a burgeoning client population. the kinds of people served. management systems. The Achilles heel here is that a lack of Several sources of bias remain in the consideration of the characteristics of The JTPA Performance participants may encourage program JTPA performance standards adjust- Standards System ment models, Barnow says: 1) bias in- administrators to “cream” for those troduced by grouping data; 2) bias due most likely to succeed in the labor mar- The author sorts out the advantages ket. and disadvantages of an established to an incomplete set of explanatory performance management system that variables, and 3) bias as a result of the Performance Standards and has been fine-tuned over the 1990s — focus on participants rather than on the Measuring Economic Efficiency the system for JTPA. This system has eligible population. Some states have been successful, Barnow thinks, in addressed equity issues by adding stan- According to the author, judging a terms of its practicality, the level of ac- dards for the participation of particular program’s “return on investment” is a ceptance by state and local stakehold- groups, others for outcomes for such key objective at all levels of govern- ers, and the richness of its database. groups. But Barnow reminds us that ment. In discussing ways to address The best empirical evidence to date each new variable added to the models economic efficiency in performance suggests, however, that JTPA perfor- requires additional data collection, and management systems, Barnow focuses mance standards have led to efforts to as models become more complex they on two issues: reductions in transfer enroll those clients defined as easiest to become more difficult to use. Con- payments and substitution effects. Cost/ place in jobs. sideration should be given, he says, to benefit analyses have tended to treat a 1) approaches that take into account particular loss to clients as a particular However, he also points to an evalu- important participant characteristics not gain to taxpayers — that is, reduced ation of JTPA by SRI in 1988, spon- included in the models, and to 2) avoid- welfare transfers equal reduced taxes sored by the National Commission for ing basing reward/sanction systems for others. This tradeoff may sound Employment Policy, which indicated solely on regression equations. simplistic but it is implicit in most E&T that although the performance manage- legislation. The substitution problem is ment system created some incentives

108 Evaluation Forum n Issue 11 n Summer 1995 Evaluation Issues and Activities n At the National Level

nn Performance Standards Systems efits of providing program direction in Other Programs and accountability.” Performance man- Performance agement systems offer our best chance, The JTPA system for Title II is Management he believes, for dealing with conflicting clearly the most highly developed. Title Systems: Evaluation III uses only one standard, entered em- program objectives, and for countering the tendency of bureaucracies to act in ployment rate, and an optional wage Principles and a manner inconsistent with the achieve- standard. Title IV programs also have Standards performance standards systems, as well ment of economic efficiency. as the Employment Service and the System Performance Analyses of performance manage- Food Stamp E&T Program. The JOB ment concepts and systems have fre- Corps includes standards for the perfor- The trend in the mid-1990s is mov- quently failed to address the role of mance of private sector center opera- ing inevitably in the direction of devel- evaluation research within these sys- tors, with sanctions involving the loss oping performance measures and stan- tems. Sometimes evaluation is treated of a franchise. Standards are being de- dards for comprehensive workforce as an independent add-on oversight veloped for the JOBS program as well. preparation systems at the state and lo- function, rather than an ongoing inte- However, JOBS differs from JTPA in cal level. Can we apply Barnow’s is- gral element of performance manage- being a state rather than a local pro- sues, cautions and recommendations to ment. Yet the professional stature and gram, and in being jointly funded by these systems? Undoubtedly. The per- perceived utility of applied social re- federal and state governments, which formance management concept Barnow search in producing valid information affects the development of its perfor- defines fits the accountability needs of for “managing performance” in social mance management system. systems as well as programs. In this programs has grown enormously over concept, performance measures and JOBS administrators have been con- the past three decades. With the advent standards are to be based on goals, ob- of larger-scale workforce development cerned that standards may lead to jectives and a set of outcomes consis- systems, the failure to define a viable creaming, and that the gross outcomes tent with those goals. This is true for on which the standards rely are not role for evaluation research within per- programs, true for systems of pro- formance management systems is an highly correlated with net program im- grams. enigma of sorts, and represents a pacts. But Barnow proposes that these concerns may be more reflective of a Most of the kinds of performance missed opportunity. different service system philosophy measures Barnow identifies as part of This missing linkage is perhaps due than realistic anxiety about the influ- program-specific performance manage- most to the perception that evaluations ence of performance standards. ment systems are equally suited to sys- are one-time, more expensive snap- tem analysis. The exception is that shots of programs, and take a longer Barnow recommends that more data since there is no “comparison group” be collected on participants and their time than other strategies to yield use- for systems, net impact evaluations are ful results. There is even the view that outcomes, that a rewards/sanction sys- not appropriate. However, cross-state the results of the more competent eval- tem be developed only gradually, and comparisons may be possible, and net that administrators at the federal level uations are so well-qualified that they impact studies of consolidated pro- lose their utility. resist basing performance standards on grams within the system may be useful. state-specific programs — he suggests Longitudinal surveys that gather pro- This perception ignores the role of that they wait for the results of a na- process studies that look at ongoing cess and gross outcome information tional evaluation before setting stan- program implementation processes, and can also contribute to a knowledge of dards. the system’s overall economic efficien- longitudinal surveys which track cli- ents, services and results over time. cy and effectiveness. Conclusions in Barnow’s They ignore the simple institution of Analysis Barnow’s exploration of issues in- formal feedback expectations and volved in developing performance stan- At the end of his chapter there is an mechanisms within evaluation research admonition to readers that none of the dards, and his recommendation to col- contracts, which ensure that evaluation lect an adequate database prior to such existing performance management sys- findings are immediately absorbed into development, are important insights for tems are perfect, or even ideal. But policy and planning decisionmaking. there is also optimism. The author is the new coordinated workforce devel- They also ignore the basic need in all opment systems. Although the author’s convinced we can develop reasonable programs for genuinely accurate infor- chapter does not address the role of systems that are useful. The basic ques- mation, as opposed to information gen- tion he poses is whether the risks of evaluation research, per se, we turn our erated from more subjective sources. In attention to that subject in the next arti- providing inappropriate incentives due this respect, qualfications to evaluation cle. to imperfect models “outweigh the ben- research findings are required if what is learned about programs is to have any ring of validity about it.

Evaluation Forum n Issue 11 n Summer 1995 109 Evaluation Issues and Activities n At the National Level

Evaluation Research Goals, evolutionary process. The Committee The overriding purpose of evalua- Principles and Standards also recognizes that the general princi- tion is “to provide the best possible in- ples and standards of any era must re- formation bearing on the value of what- One perspective on the role of eval- main sensitive to the substantive fields ever is being evaluated.” This is in the uation research is reflected in a 1994 publication by Sage, The Program in which they are being applied. And context of Sanders’ definition of evalu- finally, the Committee sees evaluation ation as “the systematic investigation of Evaluation Standards by James R. standards as guides, not as binding the worth or merit of a program.” Fig- Sanders. Sanders is the chair of the American Evaluation Association’s rules. ure 1 provides an overview of the Task Force’s principles. Figure 2 is an over- Joint Committee for Educational Eval- The Evaluator’s Major Tasks view of the 1994 principles and stan- uation. Although the book focuses sub- stantively on the education field, it is The Committee believes that appli- dards which are given more substance in Sander’s book. offered as a much more general contri- cation of the standards must be judged in the context of key tasks evaluators bution to the understanding of evalua- Implications of the AEA’s perform: tion research, and is meant to apply Principles and Standards equally well to social policies and pro- n Deciding whether to evaluate. grams in other areas, such as employ- Setting thresholds regarding the ment and training. n Defining the evaluation problem. background and behavior of evaluators is much like setting performance stan- n The effort to develop principles and Designing the evaluation. dards for programs. It clarifies roles standards guiding evaluation research n Collecting information. and responsibilities. However, no in- practice goes back to the early 1970s centive/sanction system accompanies when twelve national professional as- n Analyzing information. the former, except the ability of cus- sociations began to explore evaluation n tomers to deny contracts to evaluators issues. These associations discovered, Reporting the evaluation. they do not feel fit the standards. This not to their total surprise, that there n Budgeting the evaluation. requires customers to be much better were serious, fundamental differences informed about the meaning and value in their viewpoints. The Committee n Contracting for the evaluation. of evaluation research than they are was formed to resolve such differences, n Managing the evaluation. currently. These principles and stan- which resulted in Standards for Evalu- dards should assist customers in signifi- ations of Educational Programs, n Staffing the evaluation. cant ways to be more sophisticated in Projects and Materials, published by n Developing evaluation policies. developing RFPs for research, and for McGraw-Hill in 1981. Through a pro- making sound selections among avail- gressive series of meetings and the so- The applicability and/or importance of a given standard, given these tasks, is able researchers for periodic process, licitation of ideas from a large number outcome and net impact studies and for of experts, the Committee later re- considered to vary with the nature of a ongoing evaluation activities such as viewed and updated the 1981 effort, given evaluation. longitudinal surveys. and reached a consensus on evaluation research principles and standards as of Evaluation Principles The principles and standards also the 1990s, which are contained in the There are a number of ways to look provide insights about the basis for Sanders book. at principles and standards. A 1992 re- quality assurance and continuous pro- gram improvement. As important goals The intent of this standards-setting port from the American Evaluation As- sociation’s Task Force on Guiding of performance management systems, process was to establish a professional quality assurance and continous pro- guide for evaluation practice that Principles for Evaluators gives us one gram improvement must necessarily would also educate evaluation custom- viewpoint. The goals for evaluation that underly these principles are: 1) the depend on the nature of the information ers and the public about the role of about programs, and their environ- evaluation research. The 1994 stan- improvement of “products, personnel, ments, that is generated from a range of dards represent the official position of programs, organizations, governments, consumers, and the public”; 2) more in- sources — involving qualitative and the American Evaluation Association quantitative information, objective and (AEA). formed decisionmaking and more en- lightened change efforts; 3) the actual subjective data — through ongoing or But the Joint Committee is keen to generation of change; 4) the empower- periodic data collection and analysis. remind readers and users that standard- ment of stakeholders through engaging The development of performance stan- setting is an ongoing process. Research and investing them in the evaluation dards is highly dependent on the quali- practice and the professional knowl- process; and 5) the incorporation of ty of this information. The AEA evalu- edge base are constantly undergoing new insights from research into public ation standards can, in that sense, make change, and such changes must be con- policy development, planning and a significant contribution. tinually reviewed and revised within an oversight.

110 Evaluation Forum n Issue 11 n Summer 1995 Evaluation Issues and Activities n At the National Level

Figure 1 n Guiding Principles

Systematic Inquiry Evaluators are expected to conduct systematic, data-based inquiries about whatever is being evaluated: n Should adhere to the highest scientific stan- n Should clarify the values, assumptions, theo- dards so as to increase information accuracy ries, methods, results and analyses that signifi- and credibility. cantly affect the interpretation of evaluation n Should communicate their approaches and findings. methods accurately and honestly, and in suffi- cient detail to allow others to understand, inter- pret and critique their work.

Competence Evaluators are expected to assure stakeholders of competent performance: n Should possess the education, abilities, skills n Should decline to conduct evaluations that fall and experience appropriate to the evaluation substantially outside those limits. tasks to be performed. n Should continually seek to maintain and im- n Should practice within the limits of their profes- prove their competencies. sional training and competence.

Integrity and Evaluators are expected to ensure the honesty and integrity of the entire evaluation process: Honesty n Should negotiate honestly with clients and that might pose a significant conflict of interest stakeholders concerning the costs, tasks, with their role as evaluator. methodological limitations, scope of results n Should not misrepresent procedures, data or and use of data resulting from an evaluation. findings. n Should seek to determine their own, their cli- n Should determine that if certain activities ents’ and stakeholders’ interests concerning the seem likely to produce misleading evaluative in- conduct and outcomes of an evaluation. formation or conclusions, these concerns are n Should disclose any roles or relationships they communicated to the customer. have concerning whatever is being evaluated

Respect for People Evaluators are expected to respect the security, dignity and self-worth of the respondents, pro- gram participants, clients and other stakeholders with whom they interact: n Should abide by current professional ethics n Should attempt to foster the social equity of and standards regarding risks, harms, and bur- the evaluation, so that those who give to the dens that might be engendered in those partici- evaluation can receive some benefits in return. pating in the evaluation regarding informed con- n Should identify and respect differences sent and informing participants about the scope among participants, such as in their culture, and limits of confidentiality. religion, gender, disability, age, sexual orienta- n Should conduct evaluations and communicate tion and ethnicity, and to be sensitive to the po- their results in a way that respects stakehold- tential implications of these differences when ers’ dignity and self- respect. planning, conducting, analyzing and reporting on evaluations.

Responsibilities Evaluators are expected to articulate and take into account the diversity of interests and values Re: the General that may be related to the general and public welfare: Public n Should consider including in the planning and n Should allow all relevant stakeholders to have reporting of evaluations the perspectives and access to evaluative information, and actively interests of the full range of stakeholders. disseminate it to them, tailoring this information n Should consider not only the immediate opera- to the interests of different stakeholders. tions and outcomes of the program being evalu- n Should maintain a balance between client ated but also the broad assumptions, implica- needs and other needs. tions and side-effects of that program. n Should protect the public interest and public good.

Evaluation Forum n Issue 11 n Summer 1995 111 Evaluation Issues and Activities n At the National Level

Figure 2 n Evaluation Principles and Standards

Utility Evaluations should serve the information needs of intended users and stakeholders. n The users and stakeholders, i.e., those involved should be made explicit, so that the basis for in and/or affected by evaluation activities, judgment is clear. should be identified and their needs deter- n Evaluation reports should clearly describe the mined. program being evaluated: its context, its pur- n Evaluators should be credible — that is, they poses, its methodology, and the findings of the should be both trustworthy and competent to evaluation. perform evaluations, and so perceived by users n Significant interim findings and evaluation re- and stakeholders. ports should be disseminated to users and n The information collected and analyzed should stakeholders to encourage timely use of emerg- address key questions about organizations and/ ing information. or programs of interest and concern to users n Evaluations should be planned, conducted and and stakeholders. reported in ways that promote follow-up by us- n The perspectives, processes and rationales un- ers and stakeholders. derlying the interpretation of evaluation findings

Feasibility Evaluations should be realistic, prudent, diplomatic and frugal. n Evaluation processes should be practical, to re- potential misapplication of evaluation findings. duce disruption of the programs being evaluated. n Evaluations should be efficient, and produce in- n Evaluations should be planned and conducted formation of sufficient value and use that the re- with respect for the positions of various interest sources expended on the research can be justi- groups, to obtain their cooperation, and to reduce fied.

Propriety Evaluations should be conducted legally and ethically, considering the welfare of those in- volved in the evaluations and affected by the results. n Evaluations should be designed to assist subjects of research activities, and the need to organizations and programs to address and protect these values. effectively serve the needs of all the partici- n Evaluations should recognize and record fully pants expected to benefit from organizational and fairly the strengths and weaknesses of activities and the receipt of program services. the program being evaluated, so that problem n What is to be accomplished, by whom, and areas can be addressed and strengths built when should be agreed to in writing by the par- upon. ties involved in an evaluation, to commit them n The formal parties involved in evaluations to the conditions of the agreement or to a for- should make certain that the set of research mal renegotiation of the agreement if that be- findings and limitations to the findings are avail- comes important. able to those affected by the evaluation, and to n Evaluations should be designed and conducted any others with expressed legal rights to this in- with a respect for the rights and welfare of the formation.

Accuracy Evaluations should reveal and convey accurate (unbiased) information about the key features of the organizations or programs being evaluated. n The organization or program being evaluated analysis methods) that guide evaluations should be described clearly and precisely. should be appropriate to the research questions n The setting and environment of the program selected for study. should be examined in sufficient detail to identi- n The evaluation process should be monitored fy those factors most likely to affect the organi- and assessed throughout the evaluation. zation or program. n The conclusions flowing from an evaluation n The adequacy of the sources of information should be made explicit and should be justified; used in an evaluation should be determined. and evaluation reports should honestly reflect n Data collection methods should be selected and the results of the evaluation and any qualifica- applied such that reliable and valid information tions to its findings. is the basis for interpretations of the findings. n The strengths and weaknesses of the evalua- n The research design (experimental or nonex- tion design, methodology, and research pro- perimental) including methodological decisions cess should be assessed at the completion of a (about measurement, sampling methods, and study. quantitative and qualitative data collection and

112 Evaluation Forum n Issue 11 n Summer 1995 Evaluation Issues and Activities n At the National Level

Editor’s Note Mohr, L. Impact Analysis for Program Stufflebeam, D. Metaevaluation. Evaluation. Thousand Oaks, CA: Kalamazoo, MI: The Evaluation Cen- For useful references on evaluation Sage Publications, 1992. ter, Western Michigan University, research, please see: Patton, M. Qualitative Evaluation and 1988. Alin, M. et al. Using Evaluation: Does Research Methods. Newbury Park, Worthen, B. and K. White. Evaluating Evaluation Make a Difference? New- CA: Sage Publications, 1990. Educational and Social Programs: bury Park, CA: Sage Publications, Guidelines for Proposal Review, On- 1979. _____ Utilization-Focused Evaluation. Beverly Hills, CA: Sage Publications, Site Evaluation, Evaluation Con- Cook, T. and C. Reichardt (eds.) Quali- 1986. tracts, and Technical Assistance. tative and Quantitative Methods in Boston, MA: Kluwer Academic Pub- Evaluation Research. Newbury Park, Rossi, P. and H. Freeman. Evaluation: lishers, 1987. CA: Sage Publications, 1979. A Systematic Approach. Thousand Oaks, CA: Sage Publications, 1991. Three useful journals are Evaluation Herman, J. Program Evaluation Kit. and Program Planning, New Directions Nine volumes on different aspects of Shadish, W. et al. Foundations of Pro- for Program Evaluation, and Evaluation evaluation research. Thousand gram Evaluation: Theories of Prac- Practice. Oaks, CA: Sage Publications, 1987. tice. Newbury Park, CA: Sage Publi- cations, 1991. Levin, H. Cost-Effectiveness: A Primer. Thousand Oaks, CA: Sage Publica- Strauss, A. Qualitative Analysis for So- tions, 1983. cial Scientists. New York, NY: Cam- bridge University Press, 1987.

Editorial Commentary n To assure high quality throughout of objective evaluative information the various aspects of individual pro- from various sources. This series of articles yields common grams and systems of programs. The inclusion of quality assurance, elements of performance management n To make assessments of systems continual program improvement, perfor- systems for judging programs, and and programs using performance mance standards and evaluation re- groups of related programs in work- measures and standards. search within the concept of perfor- force development systems. Quality as- mance management recognizes that a surance, continuous program improve- n To evaluate systems and programs range of strategies for judging systems ment, performance measures and regarding dimensions of their opera- and programs are, and must be, en- standards, and evaluation research are tion, the characteristics of their client compassed in the effective manage- key concepts. groups, the nature of their services ment of programs — if indeed the pur- and service delivery systems, their Contrary to some of the distinctions pose of accountability systems and the outcomes for participants and others made by the authors represented in this goals of management are to maintain a expected to benefit from them, and section, it would seem logical to view satisfactory level of performance or en- their net impacts. performance management (or account- hance it. None of these strategies ability) systems as the holistic concept , n To continuously improve systems should be cordoned off in a territory of and attribute to these systems a num- and programs through the timely use its own. ber of goals:

Evaluation Forum n Issue 11 n Summer 1995 113 Evaluation Issues and Activities n At the State and Local Level

At the State and Local Level

Editorial Introduction programs. But the changes recom- and effectiveness of workforce mended go significantly further than development. The interest in program coordination earlier concepts in knitting diverse edu- u Utilization of labor market and re- and service system integration is not cational, training and employment ef- lated information to identify avail- new. It even appears to be politically forts into a single “system,” particularly able jobs and occupations in de- cyclical. In the 1950s many communi- at the state level — with strong man- mand, and the educational and ties created local social service coun- dates regarding short and longer-term training services available to de- cils which brought program organiza- strategic planning and the use of a velop skills. tions together around common common set of measures for monitoring community goals. The missing link was and evaluating the progress of the sys- u Development of methods for co- a commitment to strategic planning in tem in meeting planning expectations. ordinating programs so that they the formal sense, and to oversight ac- Secretary Reich’s Reemployment Act of provide maximum coverage of tivities such as monitoring compliance 1994 was an important expression of the workforce, ensure equitable with formal plans and judging overall the new philosophy. access to all population sub- performance. Therefore, as a prelude to this series groups, and enhance the delivery In the 1970s there was another of articles on earlier and contemporary of services. wave of coordination efforts, under the coordination efforts, we summarize the u Review and approval of budgets title “human services integration.” This major features of the Act in the list be- for federal programs, and the de- was the era of the federal Joint Funding low. velopment of recommendations Simplification Act, which offered a strat- n The establishment of state human regarding the budgets of other egy for packaging federal funds for bet- resource investment councils or their programs operating within the ter coordinated, even though not yet in- equivalent. These councils are to be one-stop service areas. tegrated, human service activities. Most responsible for developing a com- u Monitoring of the implementation of the 1970 concepts remained policy prehensive state-wide workforce de- goals that were only sparsely tested of the strategic plan and the judg- velopment plan and system, and for ment of the overall performance and implemented, and continuing inter- overseeing the implementation of est group pressures yielded an increas- of the local workforce preparation this plan and the system’s perfor- system. ingly fragmented social service and mance. workforce preparation system. n The establishment of one-stop ca- n The identification of substate one- The Reagan Administration advocat- reer centers. stop service areas. ed what appeared to be a new version n The provision of a menu of common of coordination through block grants to n The development of a common opera- services through the centers. states. The consolidation of funding tional agreement among all the parties streams was viewed by some experts involved in the one-stop centers. n The participation of federal employ- as the only way to achieve true integra- ment and training programs in the n The creation of local workforce in- tion of existing program territories. But centers. vestment boards within the one-stop the goal at this time was only second- n center areas. The development of a quality assur- arily to pull social programs together ance and performance management more effectively. It was mainly a way to Composition: system. satisfy political commitments to states’ u Representatives of private sector rights and to cut the federal budget, The following series of articles is in- employers. Private sector em- since the down side of these block tended to shed some light on the histo- ployers are to be a majority on grants was increased state responsibili- ry of coordination, and on current ef- these boards. ty coupled with reduced funding. forts to pull an unwieldy cluster of u conscientiously-designed educational, The Clinton Administration acknowl- Representatives of organized la- training and employment efforts togeth- edged that multiple program organiza- bor and community-based organi- er into a more comprehensive, coher- tions had now developed a natural re- zations. ent, market-driven, user-oriented sys- sistance to integration. Separate u Representatives of educational tem. funding streams for programs and institutions. projects understandably reinforced the The first article looks at some of the bureaucratization of these organization- u Community leaders. history of coordination. The second fo- al empires. Concern about the impact Functions: cuses on the efforts of some states to of this development on costs, and on coordinate their employment and train- the results of service efforts on custom- u Provision of policy guidance. ing programs within state-wide sys- ers, led to another coordination move- tems. The third and fourth review im- u Development of a strategic plan for ment in the 1990s. portant conceptual and pragmatic workforce development programs reports supporting state coordination The reforms characteristic of the consistent with the state plan. efforts. The last explores NGA’s project present movement have continued to u to enhance accountability in new state preclude the comingling of program Identification of measurable ob- workforce development systems. funds and the actual consolidation of jectives for improving the quality

114 Evaluation Forum n Issue 11 n Summer 1995 Evaluation Issues and Activities n At the State and Local Level nn A Brief, Selected Clinton Administration recommended a Coordinated Delivery Systems consolidation of six programs for dislo- A number of strategies in the 1960s cated workers under a new governance History of held promise for integrating services. arrangement, with coordinated services Coordination However, planning inflexibilities and delivered through one-stop multi-ser- unrealistic goals reduced their effec- vice centers at the local level. Over- Efforts tiveness, according to Johnston. In sight would be the responsibility of lo- Tied as we tend to be to our own era 1972, the Comprehensive Manpower cal workforce boards under the Program was introduced. It consisted in employment and training, we often guidance of state Human Resource In- of a series of demonstration projects assume that the initiatives, or “re- vestment Councils or their look-alikes. forms,” indigenous to the life or our testing the coordination of services In the winter of 1994/1995, the new across political jurisdictions and was own careers must be new and different. Congressional leadership suggested constructed on earlier coordination ef- Within this perspective these reforms other alternatives, including a 1990s’ hold our attention and energize us to forts. But service contracting remained consideration of the block grant con- fragmented, and establishing linkages experiment with and adapt useful ideas cept which surfaced earlier in the Nix- among existing programs proved to be to the needs of our time. But it is also on and Reagan years. important for us to understand and difficult. However, a revision of the program appeared to be superior to pre- learn from the history that precedes The Commission’s Perspective vious attempts in terms of the decen- “our time.” on the History of Coordination tralization and decategorization de- A report yet in draft form at the time Johnston sees the evolution of na- sired, short of new comprehensive this article was written is a unique re- tional employment and training initia- employment and training legislation. source in this respect. It is the National tives between the Great Depression and Commission for Employment Policy’s the 1990s as federal experiments that Administrative Solutions monograph tentatively titled “Histori- “evolved from a collage of individual Although administrative reforms cal Efforts to Improve Coordination in contracts and grants administered by a were less attractive and dramatic to Employment and Training Programs” number of federal agencies to broaden planners than comprehensive planning and is authored by the Commission’s programs aimed at specific target and service delivery coordination, Nix- acting director, Janet Johnston. This groups.” The major question this evolu- on’s A-95 process was expected to de- excellent in-progress report tells us tion raises for the author is what the centralize programs, simplify federal much we need to know. role of the federal government should grant processes, standardize administra- be vis-a-vis state and local government tive rules, and integrate the administra- What Is The Problem? bodies. tion of grants to state and local govern- The report raises the issue of the This question leads Johnston to ana- ments — thereby providing a new U.S. General Accounting Office’s se- lyze the history of coordination within environment for coordination. This ries of 1994 reports on problems asso- four issue areas: comprehensive plan- change did streamline grantmaking and ciated with the proliferation of employ- ning, coordinated service delivery, ad- monitoring systems, and promoted ment and training programs, titled ministration and fund consolidation. comprehensive program management. Multiple Employment Programs. This series clarifies that there are now over Comprehensive Planning Fund Consolidation 150 programs devoted to workforce The 1967 Cooperative Area Man- In 1971, the federal government in- preparation, administered by 15 agen- power Planning System was the prod- troduced Planned Variations, an exper- cies, and funded at approximately 25 uct of four federal agencies working to- iment to give mayors more discretion billion dollars. The Commission itself gether on what Johnston views as a in spending federal grants. The test completed a comprehensive inventory grass roots-upward coordinated plan- yielded mixed results in terms of the in 1994 of 55 federally-funded employ- ning concept. This system was a way to power, sophistication and effectiveness ment and training programs adminis- enclose coordinated planning and re- of mayors in utilizing this flexibility. tered by 7 agencies, Understanding source allocation within an integrated Meanwhile, Nixon’s revenue sharing Federal Training and Employment intergovernmental system linking local was to replace categorical programs in Programs, which includes information communities with different levels of several broad issue areas — education on who sponsors and funds them, government. But the system was volun- and employment/training being two of whom they serve, how many are tary, the different governmental levels these — and gave grant recipients even served, their funding levels, and infor- disagreed over control issues, policies greater discretion in spending funds. In mation from research on their efficien- were not sufficiently supported by the process, it was to allow for a con- cy and effectiveness. The array is more specific objectives and implemen- solidation of programs at the subna- mind-boggling. tation strategies, and the results were tional level, consistent with the priori- What Is The Recent Response? not carefully evaluated. Consequently ties of state and local governments. But the system did not live up to expecta- the Congress became concerned about In response to earlier concerns about tions, Johnston concludes. the lack of monitoring and evaluation program and service coordination, the

Evaluation Forum n Issue 11 n Summer 1995 115 Evaluation Issues and Activities n At the State and Local Level

of the efforts made, and the idea of rev- and the entire set of a client’s needs, counseling. Agranoff called this a “bot- enue sharing faded away like an old rather than on a single problem, on toms-up planning approach.” Rather soldier. pressures from a single advocacy than relying on the functions of agen- group, or on a single categorical agen- cies in bringing things together, the Another Perspective on cy. planner organizes a service system Coordination around sets of client needs. Although The author suggested that human the imperative for coordination at the The Commission draft report chroni- services integration be defined in terms cles the important, if imperfect, coordi- of: federal, state and county levels was ad- nation efforts through the 1990s. But dressed, the emphasis was on a local- n before venturing past the 1970s, we A comprehensive policy that cuts level “human services subsystem”, should reminisce a moment about the across independent programs and which was to operate much like a com- larger human services integration classes of services. munity development or land use sub- system. Within this perspective, natural movement of the late 1970s. We can n A service delivery system designed catch a glimpse of this movement quite linkages were seen to exist between the to meet the needs of clients whose subsystem and a myriad of resources well in a large green tome used exten- problems go beyond a single prob- sively in social welfare courses in that within the entire community. lem, program or agency. era, titled Managing Human Services. But Agranoff was not naive about It was published in 1977 by the Insti- n A consolidated organizational struc- the ability of governments and commu- tute for Training in Municipal Admin- ture that supports the above con- nities to implement this concept. He ac- istration and the International City cepts. knowledged the weak supports for Managers Association. Its editors were moving in this direction. Yet this an eclectic group: Wayne Anderson The problem stimulating serious movement was receiving momentum from the U.S. Advisory Commision on thinking about integration, Agranoff from important people. Frank Carlucci, Intergovernmental Relations, Bernard said, was 1) the expansion of categori- a former Undersecretary of Health and Frieden of MIT, and Michael Murphy cal human service programs; 2) the Human Services, testified before the of the Management Association. multi-faceted role of government (per- Congress in support of this holistic ap- forming funding, regulation, purchase The book summarized, analyzed and proach. Social welfare experts such as of services, and other functions in addi- Alfred Kahn of Columbia University commented upon a considerable body tion to service delivery); 3) generic of managerial knowledge and experi- suggested a very simple service net- problems in service delivery (fragmen- work: “an access system, or numerous ence regarding human service policy tation, discontinuity, inaccessability development, delivery and evaluation. neighborhood information centers and unaccountability); 4) whether ser- charged with giving people information The content was focused on the chang- vice strategies are meeting their goals; ing roles of different levels of govern- and options, and allowing for selection and 5) the lack of policies seeking to and monitoring; generalist social ser- ment in assessing a range of human ser- resolve these problems. vice needs, developing human service vice offices that would offer general policies and the delivery of such servic- The author proceeded to provide an counseling and manage the case if spe- es, evaluating their results, and using overview of federal human service inte- cialized services are needed; and spe- this information to improve policies gration initiatives such as the Commu- cialized programs.” and their implementation. Sound famil- nity Action Agencies, Model Cities, fed- This human services integration iar? eral research and demonstration movement clearly has much to offer in projects such as the Services Integra- The last chapter of the book honed the current debate over how to coordi- tion Targets for Opportunity program, nate policy, planning, service delivery in on service integration. Authored by the U.S. Department of Health, Educa- Robert Agranoff, a political science and oversight in large areas of human tion and Welfare’s Partnership Grants services. professor, director of the Center for program, and state and county efforts Governmental Studies at Northern Illi- initiated with or without federal assis- More Recent History nois University, and chair of the Amer- tance such as the Flexible Intergovern- ican Society for Public Administra- mental Grant project in the northwest. For obvious contemporary cultural tion’s section on Human Resources These efforts were more extensive than reasons, “manpower planning” became Administration, the chapter gives us an we might have imagined. employment and training or workforce impressive overview and commentary system planning in the 1980s and about the thinking of that era regarding The concept of a service delivery 1990s. Such planning had benefited coordination. Agranoff defined service model serving the whole individual from the lessons learned in previous at- integration as a veritable movement. with multiple needs actually surpasses tempts to cooperate, collaborate, coor- The belief expressed in that movement current integration initiatives. It was dinate, integrate and consolidate. But was that the most useful way to address not tied to categories of need, such as territoriality continued to pose chal- people’s needs was an holistic one — to education, training and employment, lenges. We return to Janet Johnston’s that is, services should be based on the but broadened to include diverse ser- insightful analysis. needs of entire government systems, vices, income assistance, and personal

116 Evaluation Forum n Issue 11 n Summer 1995 Evaluation Issues and Activities n At the State and Local Level

The State Job Training Coordinating cated that close to 30% of all Private mation networks together, and a num- Councils and more localized Private In- Industry Councils felt state councils ber of other states have developed a dustry Councils in JTPA revised the had no impact on their activities or had core set of measures common to all notion of decentralized comprehensive actually created problems for them. Yet programs within state workforce devel- planning in the direction of privatiza- states have been innovative in the way opment systems. tion. Based on a Commission study of they have established councils in terms the PICs, Johnston suggests that their of their sensitivity to existing organiza- Consolidation at the main weakness has been a lack of train- tional arrangements across workforce National Level ing in planning and oversight for programs and institutions. Some have It is interesting that despite the ma- chairs, boards and council staff. incorporated the state job training jor efforts being made at the state level Secretary Reich’s Reemployment council, as in Oregon. Some have fold- to carry out ever-stronger decentraliza- ed in the State Council on Vocational Act took the council idea a step further tion and privatization mandates regard- Education, as in New Jersey. Some in recommending the formation not ing coordination, there is also a bub- only of statewide and regional councils have “umbrella councils” whose func- bling-up of support for a national tions are limited to high-level policy but also local boards that could oversee workforce development council or development, long-term planning and one-stop multi-service centers. Strate- board. The State Job Training Coordi- gic planning, budgeting and oversight oversight, as in New York. Meanwhile, nating Council Chairs’ Association, in Indiana exemplifies the kind of council were to be the cornerstone functions of their monograph Bringing Down the recommended by the Reemployment these new mechanisms for coordina- Barriers, has recommended the estab- tion. Again, experimentation with these Act. lishment of a Federal Human Resourc- innovations have encountered substan- And there are instances of states es Investment Board by Presidential ex- tial territoriality. braving the waters relatively alone. ecutive order. This mirrors the role of Johnston reports that the MASSJobs the National Commission, raising the Johnston points out that the JTPA Council has developed strong regional possibility of expanding the Commis- amendments gave governors the option sion’s composition and responsibilities, of developing Human Resource Invest- employment and training boards by providing genuine financial incentives as a more inclusive vehicle for nation- ment Councils as a way to expand the for their functioning as regional plan- al-level policy development, compre- stakeholder group and service cover- hensive planning and oversight. age. The Reemployment Act requires ning bodies, for their devising methods for measuring the board’s effective- this. Currently there are variations on Adding ideas to the hopper is the ness, for their developing an expanded the theme. Eighteen states have formed prestigious National Research Council, councils of the kind recommended in labor market and service information through its Committee on Postsecond- system, and for their having linked the the act. Some have established even ary Education and Training for the boards with economic development ef- broader-based bodies. But other state Workplace. The Committee’s recent councils have involved fewer constitu- forts. two-year study of workforce system in- novations, funded by the U.S. Depart- encies than mandated in the act. Many Information System Technology are moving in this direction but have ment of Education, concluded that a not yet arrived. In this age of information systems, new independent Office of Workforce we reluctantly recognize that manage- Development be created, much like the Since there are no sanctions current- ment information systems often drive National Science Foundation. This of- ly for failing to create comprehensive service integration, or at least support fice would apparently absorb some of state and local workforce system its development. There is increased in- the functions now performed by other boards or councils, and limited rewards terest in automating information for federal agencies. Given the current for doing so, states have made uneven customer use in one-stop centers — trend toward reduced federal govern- efforts, their coordinating bodies differ- both information about the labor mar- ment, this careful study was quite im- ing regarding leadership, composition, ket and information about service op- portant in confirming the continuing priorities, policies and implementation tions, across organizations. And there need for strategic planning and over- strategies. Bringing previously-sover- is a fluorescence of interest in comput- sight at the federal level. If we believe eign organizational entities together erizing more sophisticated and useful that cooperation and coordination around common goals, linking these monitoring and evaluation information among diverse parties is an essential entities with employers and unions and across these territories — high-tech in- characteristic of American social de- related stakeholders, has not been easy. formation systems now considered a mocracy, there is hope. However, given The mixed results of efforts to coordi- key element in coordination efforts. Congressional leadership sentiments, a nate JTPA and JOBS, whose client Few states have such systems in place, coherent national workforce develop- populations overlap, is an example of but many are moving in this direction ment effort may or may not be viewed the difficulties. despite difficulties in developing com- as an essential part of unrelenting ef- Another illustration is pinned down mon operational definitions and mea- forts over time to bring greater integra- in the results of a Commission-funded sures for key variables across pro- tion and rationality to workforce prepa- survey conducted by the National Alli- grams. States such as New York and ration. ance of Business in 1985, which indi- Wisconsin have gone far to pull infor-

Evaluation Forum n Issue 11 n Summer 1995 117 Evaluation Issues and Activities n At the State and Local Level

nn Establishing and methods could guide the produc- gree in system planning and over- tion of objective information for policy sight. Comprehensive and planning most effectively. n Nearly all were placing heavy stress Workforce States benefited from federal coordi- on strategic planning and a review nation efforts. Closer to the pressures of results against plans, across the Development for change, however, they were devel- entire workforce development sys- Systems: oping their own strategies. Having al- ready taken the initiative, many states tem. State-Level Efforts were further energized by the Clinton n Most were developing linkages in the 1990s Administration’s coordination priori- across programs through local work- ties and legislative proposals in support force boards, local planning, and There has been a gradual recogni- of workforce development systems. one-stop centers for integrated ser- tion over time that what the federal These priorities placed emphasis on vice delivery. government and state governments emerging state-level efforts to develop across the country wished could be de- comprehensive state-wide workforce n The majority were working toward a scribed as a nicely integrated set of ed- preparation systems that were bringing common core set of data elements ucational, training and employment ser- together individual programs and ser- and definitions, and a uniform set of vices for American workers was, in vices under one policy and planning performance measures and standards fact, a patchwork of separate service roof, and making the system’s services that were to apply to all workforce accessible through coordinated multi- programs, however well-conceived and development programs. designed as separate initiatives. The de- service centers. Paralleling this boost to velopmental history of each major na- coordination were efforts to address ac- n Most had not chosen to develop in- tional program, and a myriad of smaller countability issues in judging the per- centive/sanction systems related to state efforts, had resulted naturally in formance of these new entities. Now performance. diverse policies and planning priorities, the focus was on the outcomes and im- organizational structures, service deliv- pacts of systems of programs. n About half were giving serious ery procedures and practices, ways to consideration to achieving consis- This article provides a selective tency between system goals, skills monitor what was happening within the overview of state efforts in 1994 to ad- standards, certification systems and program and its results, and methods dress accountability issues in new state- for assessing the effect of influences wide workforce preparation systems. It accountability methods. imbedded in program environments. So is skewed toward states’ beginning n A slight majority was committed to concern about and enthusiam for find- conceptual efforts, which have likely ing solutions to the lack of coordinaton integrating data systems across the undergone considerable revision since workforce system, with the others has been building. originally conceived. moving toward coordinated or The President, the Congress and the linked data systems. federal agencies began to take the coor- Characteristics of Selected dination problem more seriously begin- State Workforce Systems n The vast majority were requiring ning in the late 1980s, with JTPA, Based on materials provided by ap- that the system’s programs be made JOBS and vocational education. A proximately nine states concerning accountable through multiple strate- number of studies of coordination were their state-wide workforce system’s gies, including evaluation research. conducted to determine “best coordi- missions and goals, the following at- There was substantial interest in nation practices.” At the same time, tributes were identified: both process and outcome evalua- there was an increased interest in social program accountability. There were n Most of the states were integrating tion, and net impact studies — i.e., many reasons for this, among them education, job training, employment, a formal evaluation system. public anxiety about the economy, gov- income support, related supportive n Few states had developed strategies ernment deficits, and the proliferation services, and economic development for disseminating and utilizing eval- of programs ahead of their evaluation. efforts. uative information for continually This “accountability movement” n Most were assigning planning, coor- improving the system. was occurring at all levels of govern- dination and oversight responsibili- ment, as well as in the private sector. ties for the entire workforce system Common Features of State Evaluation researchers were brought Workforce Development to a single central body — a state- into this debate and the search for solu- Systems wide council or board. tions because of the growing prestige A prevailing characteristic of these of applied research and the perception n About half of the states were involv- comprehensive systems was that they that utilizing social science principles ing business and labor to a high de-

118 Evaluation Forum n Issue 11 n Summer 1995 Evaluation Issues and Activities n At the State and Local Level

had moved toward a market and states extending this effort to program ment Initiative to promote the produc- customer-driven approach, away from implementation variables, and to a de- tion of information on efficiency and traditional program-driven approaches. scription of services as well as client effectiveness, replete with a single This involved the private sector in new characteristics. A few states were at- “performance index” for each program. ways, particularly in strategic plan- tempting to develop a single automated Florida attached the development of ning, coordination and oversight — service delivery “front-end” for intake, “industrial incubator partnerships” to and in the design of skills and certifica- client assessment and case manage- its workforce system, promoting inno- tion systems, as well as labor market ment. Most had been developing ex- vation within the system and creating a information systems. panded and more integrated labor mar- new area for evaluation. ket information systems to support It was a new era also in the emphasis Texas applied sunset rules to pro- states were placing on performance policy development and service deliv- grams within the workforce system, re- ery as well as monitoring and evalua- management systems oriented to out- quiring systematic evaluation. Provid- tion. come measurement and the develop- ers had to document the market demand ment of performance standards. In this In most states, outcome measure- for services, and there were incentives sense, states were adopting federal ment and performance standards devel- for “best practices” based on evalua- models and tailoring them to their cir- opment focused on a common set of tions. The state targeted money specifi- cumstances and needs. The stress on outcome variables: completion of edu- cally for performance management and performance supported a greater inter- cation or training; acquisition of certi- evaluation, and the training of profes- est in going beyond monitoring and re- fied skills; continuation of education or sional personnel needed for those func- porting, and the review of results training; placement in employment (by tions. It also created a formal Evalua- against strategic plans, to a commit- occupation, industry and relationship to tion and Performance Committee. ment to evaluating the efficiency and the training received); earnings in em- North Carolina planned to seek pri- effectiveness of programs within the ployment, job retention and cost per vate sector funding for evaluation ac- workforce system, and of the system it- positive outcome. tivities. Iowa was pursuing an interest- self. Some states had given attention to ing, useful chronology of steps in Few states had formal evaluation the economic development side of integrating the workforce system’s in- plans, but some had targeted funding workforce development: job creation, formation system, by beginning with for evaluation, had created advisory incentives for employers to retrain and structured interviews and focus groups committees for that purpose, and had retain workers, incentives for employ- with providers and stakeholders which developed professional evaluation ers to remain in the state, and entrepre- contributed insights to the IBM design strategies. Some had required consis- neurial projects emphasizing the devel- consultants. The design was then im- tency across system goals, measurable opment of high performance work- plemented in several phases that built objectives, certification system expec- places. on the state’s existing MIS investments tations, and performance measures, and allowed key operational and pro- linking these elements within an ac- Some Unique State Approaches gram issues to be resolved as technical countability system. Despite important common elements computer environments were created. Nearly all the states reviewed had across state-wide workforce systems, The Great Lakes Guarantee, which been interested in pulling their data each state had customized its approach sought to weld together into one coher- systems together more effectively to — to its own environment, needs, inter- ent system the accountability efforts of support expanded oversight interests ests and resources. Michigan, for exam- several states within a regional labor and commitments. Some were integrat- ple, had classified outcomes according market, existed at the high end of the ing existing systems into a single uni- to whether they served customers or ad- accountability continuum in proposing fied data system with common defini- ministrators. The state proposed com- the most integrated and comprehensive tions and measures of key program mon forms and procedures, the devel- approach to information production for elements. Some were developing new opment of common management policy development and strategic plan- data linkages across existing systems. standards, and the creation of inte- ning. grated local MISs. Some were creating cross-walks be- The combination of common ele- tween established MISs and data dic- New York’s annual strategic plans ments and state-specific strategies tionaries. Some had expectations for lo- required an evaluation component. sketched above reveals the growing cal-level data integration which would Oregon was interested in technical as- emphasis on accepting more uniform link state and local systems. sistance and training for personnel car- principles within and across states for In all cases, a first step had been to rying out accountability tasks, and in guiding the integration, consolidation reach a consensus on a common set of common standards for the design of in- and/or coordination of more inclusive data elements describing desired out- formation systems. The state estab- workforce development systems, while comes for the entire system, with some lished a formal Performance Measure- at the same time preserving discretion

Evaluation Forum n Issue 11 n Summer 1995 119 Evaluation Issues and Activities n At the State and Local Level

nn and flexibility in the way these princi- consolidation; and the conscious ples were applied. Contemporary shrinkage of business and industrial Coordination Efforts: workforces in an effort to preserve and The Implications of State-Level enhance profitability. System Development The Core Data At the same time, continuing tech- The renewed interest in 1995 in Elements Project nological change was contributing to block grant approaches, and a further In the late 1970s and in the 1980s, a an ever-widening gap between educat- devolution of federal monies to states ed, skilled and high-wage workers and number of states had seen the need for and local areas, will require — if im- the undereducated, underskilled and plemented — very serious and careful greater organizational and service de- livery integration across the many low-wage. Both greater program coor- strategic planning and coordination by dination and more precise accountabili- agencies and programs assisting indi- state governments, and stronger public- ty were becoming accepted ways of private partnerships at that level. The viduals in acquiring education, occupa- tional skills and sustaining employ- dealing with these new postindustrial experience of those states that have challenges. ment. The federal government’s moved rapidly on federal coordination initiatives, such as state-wide work- anxiety about coordination seemed re- Background of Recent newed as well. This was not unexpect- force system development, should be in National Efforts ed. The public, and therefore the Con- a far safer position than before to cope Consistent with the times, there with expanded decentralization and the gress, was concerned about the growing proliferation and fragmenta- were renewed efforts at the national responsibilities that accompany it. tion of education, training and employ- level to reduce duplication, become ment programs with its attendant costs, more efficient organizationally, and re- and about the preoccupation with pro- form service delivery systems in the gram inputs to the neglect of outcomes. user-friendly direction. Giving the inte- gration movement legitimacy was a In response to these concerns a new president who was actively en- “second wave” of interest in human couraging greater coordination not only services integration occurred in the from the bully pulpit but through new 1990s. Much of the force behind the Administration-designed legislation. second wave was a desire to reduce The purpose was to make coordination government expenses. Influential also happen. was the perceived need to reduce ser- vice consumer’s confusion and frustra- Relatively early in its tenure, the tion, make services more readily acces- Clinton Administration completed three sible, and assure greater service quality. interconnected proposals regarding hu- man resource development and coordi- The first wave had arrived toward nation: the Reemployment Act, which the end of the 1970s under the Carter proposed pulling programs for the un- Administration, but nothing much had employed and dislocated together; the come of this churning of the waters. Welfare Reform Act, which included a The service integration movement was greatly expanded and better coordinat- given new life a decade later due to the ed work/welfare system; and the extensive economic restructuring oc- School-To-Work Opportunities Act that curring in the latter half of the 1980s sought to knit schools, training institu- and the early part of the 1990s. New tions, apprenticeships and workplaces problems were emerging in an econo- together. In addition there was Ameri- my now imbedded in fiercer interna- corps. And beyond these reforms there tional competition. were new defense conversion and di- Of particular concern was the degree versification programs that addressed of displacement of skilled blue collar the dislocation issue with military per- workers, white collar middle managers, sonnel and with civilians and commu- and other categories of workers in oc- nities affected by military cutbacks. cupational sectors affected by trade These initiatives were laced with coor- imbalances; the movement of firms to dination mandates. areas of cheaper investment and labor; These changes and related proposals the downsizing of the military; the re- involved unprecedented collaboration configuring of corporations through and coordination at the highest levels mergers, buyouts and other kinds of

120 Evaluation Forum n Issue 11 n Summer 1995 Evaluation Issues and Activities n At the State and Local Level

of government — across the Depart- The second phase will concentrate willing, able translators and negotia- ments of Education, Labor, Health and on the development of a common set of tors. Unfortunately this ends in ineffi- Human Services, Agriculture, Com- measures and definitions describing cli- ciencies, and often in ineffective ser- merce, and Defense, in particular. The ent statuses in workforce preparation vice provision and outcomes for federal government was essentially pro- systems: the status of clients entering, workers and employers. viding a viable coordination model in progressing through, and leaving the In this context, the 1992 JTPA the way it was developing service inte- system. The workgroup is expected amendments required that “a core set gration alternatives. also to develop a common set of mea- of consistently defined data elements” sures and definitions for determining Service providers, now well social- be identified which could be used for eligibility across workforce systems. ized to thinking territorially, and essen- information-gathering across major tially rewarded for doing so by funding The Historical Rationale federal employment and training policies, did not always embrace these programs.The Departments of Educa- reforms. In the evaluation research Often viewed by policymakers and tion, Labor and Health and Human Ser- community, however, many social sci- program managers as a ‘technical task’ vices followed these amendments with entists were ecstatic. The public, the for computer specialists and therefore the Core Data Elements Project. They Congress, and the Administration somewhat removed from policy and were joined later by the National Occu- seemed to be speaking with a single management, information production pational Information Coordinating voice about workforce system evalua- concerning programs within the work- Council, the U.S. General Accounting tion, demanding that objective, science- force development system (and con- Office, the Census Bureau, and the Na- based judgments of new initiatives — cerning the system itself) actually re- tional Governors’ Association. that is, of both their implementation flects critical policy decisions about and impact — be synonymous with what program features are most impor- The Product tant. These priorities subsequently af- “establishing accountability.” The Project’s workgroup recom- fect the way programs are managed. mended operational definitions and The Common Data Elements Although information systems are measures, or indicators (data elements), Project meant to be a support for policies, in for key process and outcome variables Faithful to the JTPA Amendments, practice they often direct and redirect across Adult Education, Vocational Ed- policies. Consequently decisions about the Departments of Education, Labor, ucation, the Employment Service, what kind of information should and Health and Human Services, and Agri- JTPA Titles II and III, the Job Corps, culture jointly sponsored a two-phase can be gathered for various purposes JOBS, and Food Stamps. This accom- subtly channel program effort. project beginning in 1993 to develop a plishment was expressed in the form of core set of variables, operational defi- Therefore this unprecedented effort a sizable compendium, which was re- nitions and quantitative measures that to increase the coordination and inte- viewed by experts and practitioners at cut across multiple employment and gration of workforce systems through the federal, state and local levels. The training programs. The ultimate goal access to common information about review was undeniably positive. In fact was to integrate information production the programs within them is quite sig- it was the reviewers’ conclusion that by standardizing the “information glos- nificant. And such access greatly en- the project’s sponsors should expand sary” for the entire workforce prepara- hances the ability of administrators to the compendium to include variables tion system. A Workgroup on Common monitor and evaluate such systems for relating to client status and eligibility. Core Data Elements was established, policy purposes. In the compendium, common defini- involving an impressive number of rel- Each program has come off the Con- tions were provided for key terms used evant organizations and constituencies. gressional presses with its own linguis- in the array of employment and training The first phase of the project has fo- tic patterns, tailored to its legislative linguistic systems. These were to sup- cused on the identification of a com- era, the assumptions of its sponsors, port the use of the core data elements. mon set of data elements (measures) for and the nature of its program design. The latter were classified within five use across programs in workforce sys- Different linguistic patterns tend to groups: tems, and their common definition. In produce different program cultures and n Demographic elements. the summer of 1994, a draft report ti- environments. Programs can become tled Core Data Elements and Common hostage to that inevitability. Preserving n Services received. Definitions for Employment and Train- programs’ uniqueness, and protecting n Service completion. ing Programs was released for com- their rationales and intents from being ment and further revision. The discus- eroded, these linguistic barriers also n Program outcomes. sion in this article is based on this fight against coordination and integra- n Employment descriptors. initial draft report, which may undergo tion. Collaborative policy setting, plan- change as a result of the final approval ning, program operation and account- The core elements are listed in Fig- process. ability activities suffer from a lack of ure 1, excerpted directly from the

Evaluation Forum n Issue 11 n Summer 1995 121 Evaluation Issues and Activities n At the State and Local Level

Figure 1 n Core Data Elements* Streamlining and Integrating Human Resource Development Services for Demographic Elements Service Completion Adults, supported the need for a stan- Social Security Number Completion of Occupational Skills dardized information system glossary. It Date of Birth Training (non-OJT) also clarified that the benefits of coordi- Gender Completion of On-the-Job Training nation were greater than its risks — Race/Ethnicity (OJT) greater flexibility in using funds, in- Disabled/Individual Completion of Work Experience creased knowledge and communication, Education/Highest Formal Grade Completion of Pre-Employment Skills/ more efficiency in service provision, Level at Entry Job Readiness Training Highest Degree/Credential at Entry Completion of Basic Skills Education greater ability to use specialized exper- Labor Force Status at Entry through 8th Grade tise, better monitoring, and enhanced Scheduled Hours of Work Per Week at Completion of Basic Skills Education at performance. Entry (if employed) Secondary Level (9 through 12) The report on the Core Data Ele- Homeless Individual Completion of English as Second Lan- ments project also comments on the Veteran guage Public Assistance (federal, state, or lo- Completion of Postsecondary Academ- benefits of a more logical information cal) ic (non-occupational) Education collection approach, listing the follow- Long-term AFDC Recipient Reasons for Not Completing Planned ing major advantages: Service(s) n Removal of identified barriers to the Services Received Assessment/Testing Program Outcomes coordination and integration of ser- Counseling/Career Development Advanced to Higher Level of Education vices across employment and training Job Search Assistance or Training programs at the state and local levels. Occupational Skills Training (non- Attained Additional Degree/Credential OJT) Assessed Learning Gain in English as n Enhanced program planning and On-the-Job Training (OJT) a Second Language oversight capability at the federal, Work Experience Assessed Learning Gain in Basic Skills state and local levels. Pre-Employment Skills/Job Readiness Entered Unsubsidized Employment Training Entered Subsidized Employment n Removal of barriers to efficient cus- Basic Skills Education through 8th tomer service. Grade Employment Descriptors Basic Skills Education at Secondary Scheduled Hours of Work per Week Prologue Level (9 through 12) Earnings (Hourly or Annual) English as a Second Language Occupation The work of the Job Training 2000 Postsecondary Academic Education Performance Standards Subgroup in (non-occupational) *Common definitions have been devel- 1991 revealed considerable ambiguity Supportive Services oped for the italicized terms. in the goals of programs and the lack of outcome measures in a number of them. Not surprisingly, the Subgroup recom- workgroup’s draft report. Definitions with the corresponding element and its mended the establishment of a core set of the data elements are listed in the common definition. Project participants of performance outcomes. By 1993, the draft report’s Appendix. Detailed there are encouraged to add more detailed in- Government Performance and Results also are current differences between formation to their information systems Act was requiring agencies to establish programs regarding 1) whether data are consistent with the common defini- program goals, and performance mea- collected on individuals, cases or ag- tions. In this way, sponsors hope this sures consistent with them. At the same gregate characteristics, and 2) the num- convergence toward a common under- time, the NGA Performance Manage- ber and type of data elements collected. standing of terms and indicators will ment Project was pinning down some The report clarifies that programs may yield a common set of forms and inte- basic principles for establishing ac- adopt the common definitions but grated data systems. countability in coordinated state work- choose to make selections from among force development systems, emphasiz- the core elements, in moving toward The Benefits of Information ing a logical progression from system more comprehensive information sys- Consistency goals to system performance measures. A core cluster of standardized defini- tems. Part of the prelude to this innovative tions and measures was viewed as an es- The Project’s sponsors requested project was a 1991 NGA survey of state and local policymakers and pro- sential dimension of state E&T “ac- that if programs represented in this ma- countability systems.” jor effort collect information on a sub- gram administrators from 13 federally- ject included within the core set, they funded education, training and employ- Meanwhile, information system in- should do so in a way that is consistent ment programs. The survey report, novations at both the federal and state

122 Evaluation Forum n Issue 11 n Summer 1995 Evaluation Issues and Activities n At the State and Local Level

nn levels were beginning to capture infor- quate information for improving pro- mation on participants’ characteristics, Comprehensive State grams and suggesting major reforms. In services and outcomes across program Workforce Development that vein, they recommend the follow- territories, increasing the ability to ing: track clients in coordinated systems. Systems: Key n Simultaneous study of a program’s This was a significant achievement at a Elements or a system’s 1) organizational and time when 39 states had developed one-stop centers. As a first step in a 1993 National service delivery policies and pro- Governors’ Association project consid- cesses, and 2) its outcomes and So with this kind of evolution in ering the goals and strategies involved longer-term impact, or performance. thinking about the important role of in- in useful performance management sys- formation systems, the Core Data Ele- n Promotion of continuous improve- tems, NGA commissioned a monograph ments Project is looking at the compen- ment in the organization and opera- on these issues from the Center for Gov- dium, and the additions made to it in ernmental Studies at Northern Illinois tion of programs or systems through the next phase of the project, as more University. Authored by John Baj and the use of a combination of 1) pro- of a launching pad than a landing Robert Sheets in 1994, Building State cess standards and 2) outcome stan- place.They have recommended these Workforce Development Systems Based dards (performance standards). subsequent steps: on Policy Coordination and Quality As- n Use of a capacity-building (quality n The incorporation of the compendi- surance did more than provide a frame- assurance) system for adopting um of core data elements by addi- work for NGA’s Performance Manage- “best practices,” based on the results tional federal agencies, when report- ment Project. Its perspective on the of process and outcome studies, and ing requirements and new statewide coordination of policy and for the purpose of continuous im- management information systems service provision, and on the evaluation of state workforce systems, links evalu- provement. are developed, and by state councils ation research principles and methods The authors summarize their posi- and agencies involved in compre- with the principles of management and tion in stating that “the major challenge hensive workforce development sys- program operation, giving attention to a tems. in building cohesive workforce devel- range of accountability issues rarely dis- opment systems for labor market en- cussed simultaneously or merged in a n The expansion of the core set into a trants, unemployed workers and eco- single approach. comprehensive dictionary of com- nomically disadvantaged individuals is mon definitions across education, Inputs versus Outputs: The to improve the quality of programs and training and employment efforts, Authors’ Position service delivery throughout the system maintained and updated by OMB. while achieving greater accountabili- The authors begin with the conclu- ty.” Therefore the purpose of their re- n The use of such a dictionary in de- sion that we have lived for several de- port for NGA is to suggest a conceptual veloping new legislation. cades with an imbalanced focus on in- model that is responsive to that kind of puts (service “treatments”) to the It is to their great credit that the mandate. project’s sponsors hope to take their exclusion of adequate output (outcome) measures. We have essentially empha- The authors sketch the model in this mandate further, to the ultimate end of way. The outer boundary is a “broad sized the quality of services and service this exercise, namely policy consisten- policy coordination framework” de- cy. The focus on information systems is systems, failing to hold programs ac- countable for their results. But profes- fined in terms of a logical, internally to lead in this critical direction. consistent progression of priorities: the sionals are beginning to question wheth- system’s mission; its goals and objec- er the pendulum has swung too far to the output side. The belief that perfor- tives; its implementation strategies; its desired outcomes; its target groups, mance standards can lead sometimes to client eligibility criteria and types and selecting those most apt to succeed and can inappropriately redirect a program’s intensities of services; and its allowable costs. Quality assurance is seen as a intent, and that not meeting standards mechanism for maintaining and en- may torpedo programs important to pre- serve and refine, has led to new cau- hancing the structures and functioning of this coordinated system. Quality as- tions. surance systems are viewed as having a So Baj and Sheets take a middle po- performance management component sition on the swing of the pendulum. which guides the workforce develop- They say that an exclusive emphasis ei- ment system in the direction of its goals ther on inputs or outputs denies us ade- and objectives, and supports continu-

Evaluation Forum n Issue 11 n Summer 1995 123 Evaluation Issues and Activities n At the State and Local Level

ous improvement in system processes grams,” and should encourage better occupational licensing and certifica- and outcomes. communication between the public and tion, and private professional certifi- private sectors. We should investigate The performance management com- cation and recognition systems. this perspective on performance stan- ponent is expected to accomplish the dards. n Supplier certification and recogni- following: tion: training and technical assis- As mentioned earlier, the authors n Incorporate the principles of continu- tance related to the certification of distinguish between process, or design ous improvement. standards, and performance, or out- suppliers (service providers). n Judge the entire system’s perfor- come, standards: n Program and subsystem perfor- mance, as well as the performance of n Process standards: address specific mance management: monitoring and programs within it. characteristics of the service deliv- reporting performance based on de- sired outcomes and standards, and n Make these judgments using multiple ery process, such as internal organi- diagnosing performance problems. measures of outcomes, including ed- zational policies and procedures, ucational achievement, occupational assessment and instructional ap- Baj and Sheets insist that the devel- competencies, customer satisfaction, proaches, staff qualifications, and opment of common sets of standards is educational and employment contin- monitoring and evaluation strate- essential to performance management uation, employment retention and gies. and therefore to quality assurance at earnings, and industry-based skill any level of the workforce system: that n Performance standards: address the is, educational and occupational skill standards. results of service interventions, or standards and credentialing systems; n Use a common set of outcome vari- “treatments,” on those receiving performance standards; and supplier ables, definitions, measures and per- them, including desired changes in process and outcome standards and cer- formance standards in arriving at their academic and occupational tification requirements. these judgments. competencies vis-a-vis national and The authors are unequivocal about state skill standards. n Offer incentives and apply sanctions the role of evaluation. The improve- associated with performance stan- Monitoring and evaluation are es- ment of system performance, and the performance of programs within the dards, including capacity-building sential functions in arriving at objective system, “require a strong linkage be- assistance. judgments of the movement toward compliance with these standards. And tween performance management and Although the various concepts in this movement toward the standards is evaluation.” This linkage supports and 1990s “system”-oriented vocabulary — viewed as supporting a commitment to reinforces continuous improvement. quality assurance systems enclosing continuous quality improvement. The Evaluation, they suggest, plays several performance management systems, and authors seem to be saying that the dual roles: performance management systems en- emphasis on judging the quality of both n To validate quality standards. compassing continuous improvement — the design aspects of the workforce sys- are not always as clearly defined or as tem and its performance is a new devel- n To diagnose performance problems. well differentiated from one another as opment. However, a more equal em- n To establish continuous improve- one might wish, the monograph is enor- phasis on process and outcomes came ment as a supplier process standard. mously significant and useful. Let’s into vogue in the late 1980s, even Figure 1, excerpted directly from the look further into it. though it could only be applied to indi- report, gives us an overview of this part vidual programs in the absence of com- Quality Assurance prehensive coordinated workforce sys- of the authors’ model. The authors de- fine the traditional role of evaluation tems. What is new is this dualism in The establishment of quality assur- research as focusing on basic provision ance systems, as an element of adminis- passing judgment on the new networks of programs. decisions (who should receive what trative coordination, should respond, the type of services and at what cost), and authors say, to customer-defined out- The authors do not neglect, in their assessing the returns on high-cost ser- come standards. They claim that quality model, subsystem quality assurance. vices (such as case management) to tar- assurance systems require “the major They propose three major quality assur- geted populations. Evaluation research- restructuring and integration of all per- ance functions at the level of programs ers will likely quarrel with this view. formance standards, skill standards, and within workforce systems: As part of a management system, the related program accreditation and certi- n Trainee credentialing and recogni- authors feel evaluation should provide fication standards used by federal and decisionmakers with information on the state workforce development pro- tion: educational credentialing, state

124 Evaluation Forum n Issue 11 n Summer 1995 Figure 1 n Continuous Improvement Procedures and their Links to Evaluation and Research

Policy Coordination Framework

t Mission Clients Services Costs Quality t

t t Educational and Process Standards Supplier or Service Provider Subsystem and Program Occupatinal Skill Outcome Standards Outcome Standards t t Standards t

n Academic Continuous n Trainee Competencies n Trainee Competencies

Improvement t n Occupational Trainee Returns Trainee Returns Procedures n n Basic Workplace n Customer Satisfaction t n Develop an organiza- n Customer Satisfaction n tional plan to ad- Social Cost Reduction dress performance n

improvements. t t n Involve all staff and

all major functions and processes of t the organization. n Use recognized im- provement method- State and National ologies. Evaluation n Use valid and reli- able data with ap- n Validation of Quality Standards propriate analytical Best Practice t technologies. n t n Effectiveness of Services n Determination of Who Benefits

from What Services t t t t Figure 1 n Continuous Improvement Procedures and their Links to Evaluation and Research Evaluation Issues and Activities n At the State and Local Level

“validity and cost-effectiveness of all as to become effective indicators of net Information Systems major provision decisions.” Their ex- impacts, based on the differing charac- Without adequate data, quality as- tension of the role of evaluation re- teristics of client groups (unskilled vs. surance systems can accomplish little. search to validating process and out- skilled, for example). But effective and Therefore, the authors recommend that come standards stretches that role valid are two different things. Further- automated management information considerably, perhaps unrealistically more, weighting and adjusting are more systems at all levels must be capable of and inappropriately. appropriate in judging compliance with supplying the kind of information Evaluation research has been of performance standards than in inter- needed to implement the various over- preting program outcomes, it being greatest use in providing the least sight strategies suggested in their re- possible to disaggregate results for dif- biased information about the nature port. The emphasis is on a common set of service provision and its results for ferent client groups in interpreting data of quality indicators for process and analyses. clients. In the process, it can contribute outcomes. A related interest is in the information about the level of compli- Confusing also is the authors’ final development of standardized instru- ance with performance standards. But conclusion about the differences be- ments, meeting scientific standards, for the validation of quality standards has tween evaluation and management. conducting customer satisfaction sur- not been within its purview. Having advocated for evaluation as an veys. The authors also recommend the integral part of performance manage- development of cost-effective methods The authors claim that evaluations ment systems, they later differentiate for gathering and analyzing informa- of national programs and demonstra- tion projects have not been designed to between evaluation and performance tion at the point of service delivery, to management by pointing to the infre- serve supplier certification purposes improve the ongoing delivery or pro- quency of impact evaluation and the and to provide a basis for judging con- duction of workforce development ser- vices. This is open to debate. It is true, need for performance management sys- formance with quality standards. This tems that supply more immediate feed- should be paired, they indicate, with ef- however, that evaluations have not fo- back to administrators, program opera- forts to train suppliers in the continu- cused on different kinds of service pro- viders at the point of service delivery, tors and service providers. Therefore ous improvement process and provide they comment that “quality assurance ongoing technical assistance to them. which the authors feel must be a re- systems are largely concerned with out- quirement of performance management Major Recommendations systems within quality assurance sys- come and process questions that emerge from accountability rather than in the Report tems within workforce development impact issues.” What does that mean? systems, if continuous improvement re- There are three key recommenda- quirements are to be met. Are they saying that evaluation re- tions: 1) the encouragement of state search can contribute to performance experimentation and demonstration in Some evaluation researchers have policy coordination and quality assur- concluded that some of the program management but is not suited to an on- going oversight role? Performance ance; 2) the development of federal- outcomes typically measured are not management begins to look a great deal state initiatives for establishing a quali- necessarily good indicators of longer- ty assurance system within a national term impacts. But the authors take this more like an expanded version of com- pliance monitoring, rather than as a policy coordination framework; 3) the conclusion a step further by interpret- source of objective, science-based in- promotion of federal-state initiatives ing it to mean that some measures are for developing a national policy coordi- not valid indicators of program impact. formation about the critical relation- ships between client characteristics, nation framework beyond quality as- Indicator validity usually refers to the services, and outcomes. surance. The positioning of the recom- extent to which a particular measure or mendations reflects the authors’ belief group of measures accurately represent The authors also say that continuous that initiatives should begin in the the outcome variable being measured. improvement should involve ongoing states, and build toward a national Determining indicator validity is a sci- organizational self-evaluation through framework. entific process. One cannot simply leap internal review teams, as well as from position A — i.e., the conclusion through third-party process and impact What do these recommendations in- that measures such as job placement evaluations. And the basis for continu- volve? and off-welfare do not necessarily pre- ous improvement should be a clear def- n Recommendation #1: States are dict well to net earnings and welfare inition, they suggest, of what “im- identified as the location for experi- savings — to position B — i.e., the provement” is to mean, in the form of first two measures are not valid indica- criteria and standards. This definition mentation, but the federal govern- tors of the latter two. process is to be applied at all levels: the ment is expected to 1) encourage and assist states in experimenting, The authors suggest that outcome system as a whole, the programs within the system, and the service provider or- and in testing policy coordination measures be weighted and adjusted so ganizations involved. frameworks that include school-to-

126 Evaluation Forum n Issue 11 n Summer 1995 Evaluation Issues and Activities n At the State and Local Level

nn work, workforce entry, and worker The National Phase Two: The Six-State retraining and transitioning, and 2) Accountability Effort develop a national policy coordina- Governors’ Following the production of the tion framework for all publicly- Association monograph, in the spring of 1994 NGA funded workforce development pro- invited states with state-wide work- grams. Performance force boards or councils to apply for technical assistance to develop an ac- n Recommendation #2: Management Project Several na- countability model for their workforce tional initiatives should contribute to With funds from the Joyce Founda- development systems — i.e. to apply state experimentation: tion, NGA worked with the Center for for a second phase of the performance u Linking national skill standards Governmental Studies at Northern Illi- management project. nois University in 1993 to develop a to state quality assurance sys- Technical assistance was to be made tems. conceptual framework for improving available for fifteen months, the first services and accountability across twelve of which were to be devoted to u Restructuring public and private workforce programs, with a special in- the development of a state performance program accreditation and certifi- terest in the development of compre- management model. Funds for this as- hensive state workforce systems. This cation systems. sistance were to be provided by two framework was described in NGA’s u Establishing national/state capac- private foundations — the Joyce Foun- publication, Building State Workforce dation and the John D. and Catherine T. ity-building programs supporting Development Systems: The Critical MacArthur Foundation, and the U.S. quality assurance. Roles of Policy Coordination and Departments of Labor, Education, and Quality Assurance. This monograph u Linking evaluation to perfor- Health and Human Services. This phase was prepared in anticipation of a six- mance management within the was to be coordinated with the National state project sponsored by NGA, and quality assurance system. Institute for Literacy’s performance served as the major resource material management project. With respect to the last initiative, the for that project. authors suggest that the results of im- The states selected on a competitive pact evaluations should be used within Phase One: The NGA Working basis to receive assistance were expect- the quality assurance system to exam- Paper ed to design outcome-based account- ine policy decisions, their implementa- The concept proposed in the mono- ability models for 1) preparing youth for the school-to-work transition, tion, and the validity of quality stan- graph was a performance management 2) retraining and transitioning of other dards. Again, it may be more accurate system that incorporated quality assur- to talk about the appropriateness and ance and continuous program improve- workers, and 3) helping disadvantaged adults and those with multiple barriers utility of quality standards. ment in a model that allowed policy- to enter and re-enter the workforce. The makers and planners to judge 1) a Comments on the Report workforce system’s implementation six states were to serve as laboratories for establishing accountability across (how it was organized and made things Despite some fuzziness in distin- workforce development programs. The guishing among system-defining con- happen, including its level of coordina- tion); 2) its participants’ outcomes; and concepts of quality assurance and con- cepts, the overall model the authors tinuous program improvement were to 3) the system’s impact and net effects propose is admirable. It includes all the be dimensions of the models these major elements reflected in what we judged against its most significant goals. The idea of “continuous im- states were to develop for application have learned from experience, expert in their own state and for potential use provement” was considered to involve opinion and research about policy de- elsewhere. velopment, administration, planning, ongoing self-evaluation, and a redesign program operation, service delivery and of various elements of the workforce The RFP Process oversight, bringing these elements to- system in response to that evaluation. gether within a coordinated system. In The notion of “quality assurance” in- NGA’s project RFP made the as- developing this model they ably resist volved the development and use of per- sumption that many states were now in the temptation to ignore the different formance standards, educational and the process of restructuring their work- levels of human resource activity — the occupational skill standards, and stan- force development systems into what equally important national, state, pro- dards for judging the operation of the was described as “seamless, customer- gram-specific, and service delivery- system. The concept placed heavy em- driven delivery systems” in response to specific levels. State workforce devel- phasis on evaluation research and the these well-recognized problems: di- opment councils and boards, and innovation it was capable of encourag- verse and often contradictory federal program administrators operating with- ing. and state program goals, performance in comprehensive systems, will find expectations, funding incentives, oper- this report stimulating and useful. ational rules and regulations, and defi- nitions of program terms. The RFP considered the participating states to be

Evaluation Forum n Issue 11 n Summer 1995 127 Evaluation Issues and Activities n At the State and Local Level

developing performance management Figure 1 n Expectations for States Participating in the NGA Project models for much larger entities than those to which performance manage- n Create a plan for the development n Develop specific monitoring and eval- ment principles had been applied in the of the performance management uation methods for holding state and past — this time to comprehensive model that will outline the action local program managers accountable state-wide workforce systems, not steps to be taken, including the pro- for achieving system goals. cess to be pursued, the deliver- merely to individual programs or sets n Incorporate quality assurance and ables, the timelines, and the techni- of related programs within such sys- continuous program improvement cal assistance needed. tems. The design of requests for waiv- within the performance management ers to federal and state statutes and im- n Involve the state’s workforce board model. or council. plementing regulations were clearly to n Develop occupational standards for be part of participating states’ responsi- n Invest service providers in the use across programs within the sys- bilities. project. tem. n Designate an interagency policy n If desired, develop an incentive/sanc- More specifically, states applying team and a technical work group, tion system for rewarding successful for the project were expected to pro- and identify a coordinator who will system operation and performance, duce the results listed in Figure 1. The head the policy team and act as the or for providing technical assistance ultimate goal, in developing perfor- liaison with NGA. to resolve system problems. n mance management models, was to Focus on three priority groups of n Define federal, state and local over- support the survival and enhancement workers — youth, adults needing re- sight roles and responsibilities vis-a- of a single integrated workforce system training, and disadvantaged work- vis the system. ers — with the option of giving — one in which key system functions n Refine the model, working with na- greater emphasis to one or more of such as policymaking, coordination, in- tional experts. these. formation production, monitoring and n Participate in interstate coordination n Establish a set of state goals for the evaluation would be commonly defined efforts, and share information and state’s workforce system, for each problems with other states in the and applied across an array of work- priority group. force programs. This kind of standard- NGA project. n Identify quantifiable outcomes for n Participate in three project meetings ization was to result in a more orga- the system, for each priority group, sponsored by NGA. nized, efficient and accountable given these goals. organizational structure and service de- n Identify federal and state regulatory n Develop a common glossary of pro- barriers to applying the model. livery system. gram terms for the system. n Produce a state-approved report on n Develop a common set of opera- Clearly the RFP was asking a great the twelve-month design phase, tional definitions and quantitative deal. But states were receiving some- which details the dimensions of the measures for the key factors to be thing important in return — assistance state’s model and presents a plan for studied in judging the accountability from research experts at the national implementing it. of the system. level who were not otherwise available n Begin to implement the model devel- to them, validation of their approaches oped. to accountability from their peers, and a certain level of national notoriety for contributing to the dialogue on system plicant’s project organization plan Based on the above criteria, six coordination. appeared to be, and the strength of states were to be selected. The current the state’s commitment to attending status of each of these states’efforts to The Selection of Project the three national meetings. develop a performance management Participants strategy for its comprehensive work- n How clearly the applicant had artic- NGA’s selection criteria focused on force system was to be taken as the ulated technical assistance needs. the following issues: baseline for judging its progress over the life of the project. Each of the na- n How substantial the results of the n How the applicant would build on tional meetings was to further refine project were to be. and enhance current state efforts to each state’s model through input and develop comprehensive workforce Applicants who planned to focus on materials from the national consultants, policies, integrate local planning and all three groups of workers were given structured peer reviews, and the sharing service delivery, and develop and preference. In an effort to assure diver- of information on common issues. use common monitoring, perfor- sity in the models developed, other fac- States would be able to obtain technical mance and evaluation standards tors were considered, such as the com- assistance from the consultants and across programs. plexity of the applicant’s workforce from NGA staff on-site, and through system, the nature of the institutional conference calls and correspondence. n What priority groups would be em- environment of the system, the size of Representatives from any of the six phasized by the applicant. the state applying, and its geographical states could share information and ad- location. vice on-site with other states if this n How appropriate and feasible the ap- seemed useful.

128 Evaluation Forum n Issue 11 n Summer 1995 Evaluation Issues and Activities n At the State and Local Level

The Technical Assistance solidate at the federal level, the states NGA also provided participants with Offered selected were to have an opportunity to a Glossary of Performance Manage- work closely with relevant federal ment Terms which was an extremely These were the kinds of technical agencies. useful tool for sharing insights across assistance states could access in the the participating states. project: Project Orientation General Ideas in NGA’s 1. The provision of reports and work- To prepare the six states to under- Issue Paper ing papers from NGA’s Reinventing take a clearly-defined common mis- Government Task Force, appropriate sion, even though their accountability As a context, NGA staff reminded private sector representatives, and models would reflect the diversity of states in the paper that the quality of national experts. their circumstances and interests, NGA educational and other workforce devel- 2. The provision of information on held the first of three national project opment programs has been judged tra- meetings in Detroit in mid-June, 1994. ditionally on the basis of accreditation successful state practices. A number of working papers were pro- criteria, program approval processes, 3. The outlining of a process for dis- vided to give participants a framework and planning requirements oriented to seminating and utilizing research for carrying out the mission. These in- compliance with rules and regulations findings for local program planning. cluded the substantial report, Building concerning inputs. The pendulum had State Workforce Development Systems, swung, they said, in the direction of 4. The provision of information on a and the first draft of the report prepared judging outcomes — replete with in- set of process and outcome criteria by the Workgroup on Common Core centive/sanction strategies, affording for continuous program improve- Data Elements, Core Data Elements flexibility in the design and refining of ment and self-assessment. and Common Definitions. programs, and relying on competition to promote high quality. However, they 5. The provision of assistance with the The materials also included the Na- tional Institute for Literacy’s 1994 re- acknowledged that an emphasis on both development of waiver requests. inputs and outcomes was important, port, A Guide to Strategic Planning for and that a commitment to continuous 6. The provision of information on leg- Performance Measurement, Reporting islation, and reports on school-to- and Improvement Systems, an issue pa- improvement must involve an interest in both. work performance measures, adult per prepared by NGA for the first education quality indicators, com- project meeting, and an overview of The new wrinkle in this ongoing di- mon data definitions and measures, Texas’ process for developing core per- alogue about accountability, they pro- and the use of UI data. formance measures. posed, was the emergence of more NGA also outlined general expecta- comprehensive and coordinated service 7. The provision of a list of consultants systems that cut across individual pro- tions regarding the characteristics of specializing in performance manage- grams. This has generated interest and the accountability models the states ment. were to develop: concern about how to appropriately judge system-wide processes and out- The Selection of Project States n Measurable state-wide goals for : comes. Homogeneity of program terms Thirteen states applied to be part of 1) youth transitioning from school and goals is merely an essential preface to developing system-wide visions, the NGA project. The following six to work, 2) workers needing retrain- benchmarks, goals, objectives, outcome were selected: Illinois, Iowa, New York, ing and transition assistance, and North Carolina, Texas and Washing- 3) adults with multiple barriers to expectations, and performance stan- dards. ton. These states were considered to employment. have made significant progress in re- The movement is toward what NGA n A common set of measurable de- structuring their range of workforce de- terms system-wide shared accountabili- sired outcomes for the customers of velopment programs and projects into ty. This approach envisions a system high-quality, more customer-driven the workforce development system. with these dimensions: workforce preparation systems. In the n State and local performance stan- n A set of measurable policy goals for NGA project they were expected to dards consistent with the desired the entire state. take a subsequent step: to develop “the goals and outcomes. next generation” of performance man- n A set of measurable goals for the agement systems, which would incor- n A common information system workforce development system con- porate the concepts of quality assurance (variables, definitions and measures) sistent with state policy goals. and continuous improvement, and for reporting progress against the would stress joint accountability across goals. n A common set of measurable out- different programs and levels of gov- comes for all workforce develop- ernment. Given the Clinton Adminis- n Joint accountability across the sys- ment programs within the system, tration’s efforts to restructure and con- tem for achieving the goals and consistent with the goals for the sys- meeting performance standards. tem.

Evaluation Forum n Issue 11 n Summer 1995 129 Evaluation Issues and Activities n At the State and Local Level

n A single incentive/sanction strategy management information, monitor- velopment, with performance stan- that can be applied system-wide to ing information and evaluation re- dards phased in incrementally so as support the achievement of system sults? to take advantage of program infor- goals. mation which can inform the choice 5. Is it useful to focus on the different of standards. n A commitment to ongoing self-eval- populations served, or should there uation to determine the level of be a general accountability policy n Measures of clients’ access to the progress in achieving system goals. that applies to all customers? system should be included, as they This directs a very logical progres- 6. How should program outcomes be are important indicators of service sion of thinking and planning. Howev- linked to outcome expectations for provision equity and the level of re- er, it is questionable whether state-wide the system as a whole, and what sponse to key target populations. policy goals can be stated such that should the consequences be for n Key outcomes to track are: labor they are measurable in the same sense meeting, exceeding or failing to market outcomes, learning out- as system and program outcomes can meet system-wide performance ex- comes, and client/employer satisfac- be measured. Benchmarks are by nature pectations? tion outcomes. abstract and difficult to measure either qualitatively or quantitatively. The Survey Undertaken by the Survey respondents expressed cau- In discussing this progression, the Texas Council on Workforce and tions as well as making recommenda- writers of the issue paper distinguish Economic Competitiveness tions. They viewed “entered employ- between performance management ment rate” as a good measure if used The Performance and Evaluation tasks and continuous improvement with other measures such as “employ- Committee of the Texas council con- tasks. They suggest that goal/outcome ment retention,” the “appropriateness development is the former type of task, ducted a survey of eleven national ex- of the job,” and “the degree of self-suf- perts in the performance management and evaluation research and capacity ficiency.” However, they pointed out field. The survey instrument consisted building the latter kind. However, eval- that placement rates might not be ap- uation researchers would likely consid- of a questionnaire with ten discussion propriate for judging educational pro- questions. The experts were asked to er evaluation research activities to be grams or further training. They did not respond in writing and subsequently an integral part of a comprehensive think “training-related placement rates” performance management system through a telephone interview. Eight were useful in all cases because of responded. Some of the major insights which incorporates quality assurance, 1) the ambiguity of definitions of from the survey are summarized in the continuous improvement, and the iden- “training-related,” 2) the uneven im- tification of “best practices.” sections below. portance of this measure in educational and employment assistance vs. occupa- The issue paper poses some very im- Labor Market Outcomes tional skill training programs, and 3) its portant and useful questions: n Multiple quantitative and qualitative different meaning for longer-term spe- 1. What should be the roles and re- measures of key factors should be cialized occupational training than for short-term training. Nevertheless, in sponsibilities of federal, state and lo- developed to compensate for defi- some instances and with some training, cal governments and the private sec- ciencies in any one indicator, al- they considered the training-related tor in promoting and assuring shared though the system must remain man- measure very important. accountability? ageable. All agreed that “wage rate” mea- 2. Does there need to be shared respon- n Measures should not only include sures were important. However, they sibility for planning and oversight at sheer outputs, such as placement felt it was difficult to sort out their each level of the system, and what rates, but indicators that provide in- meaning vis-a-vis other factors such as kind of governance arrangements formation on changes over time, economic changes, or personal attitudes should be put in place to assure such as wage rates. and situations. They also agreed that UI shared accountability? Wage Record information was the best n Adjustments in performance stan- source of information on “earnings,” 3. How can the contribution of each dards, although to be kept to a mini- “employment affiliation,” and the char- funding stream to the collective ef- mum, should consider local eco- acteristics of firms where program par- fort be determined and judged? nomic conditions and the ticipants were placed. They recognized that tracking information specific to 4. Is it enough to have information sys- characteristics of participant popu- lations. particular occupations, developing indi- tems with common definitions and cators for the quality of jobs, and dis- measures of outcomes, or are inte- n Core measures to be used across a tinguishing between full-time and part- grated data systems needed which workforce system should be a first time employment in the wage records offer customer information, case step in performance management de-

130 Evaluation Forum n Issue 11 n Summer 1995 Evaluation Issues and Activities n At the State and Local Level

were a problem. They were ambivalent the six states and the history of their re- n The measures to be collected, repre- about client interviews as a way to en- cent efforts to coordinate multiple edu- senting these outcomes. hance the information on wages, be- cational, training and employment pro- The Illinois progress report followed cause of the general unreliability of grams to create a new workforce this logical sequence of decisions self-report data, but thought this strate- ‘system’. Figures 2 and 3 provide a gy was feasible if used in combination brief overview of the content of their closely. Twelve programs were includ- ed in the system. State policy goals en- with UI data. responses. compassed early childhood develop- All the experts surveyed recom- Some Observations About the ment, educational performance mended the use of “job retention” mea- RFP Responses standards, a school-to-work system, lit- sures used in combination with place- eracy and high school equivalency, ment indices. This could best be done, Of the six states participating in the professional and technical degree and they suggested, by using information NGA project, Washington State ap- certificate programs, worker retraining, from the UI files, by conducting cus- pears to have done the most thinking and economic independence and long- tomer service surveys, and collecting about the requirements of an account- term employment. Significant out- additional information on benefits and ability model that utilizes scientific comes were the acquisition of academ- career ladders. But there were caveats. principles and methods that are now ic, occupational and basic workplace It is difficult to track employment over well established evaluation research ap- skills; the realization of sustained em- time as people move in and out of jobs. proaches. Although Washington is con- ployment and earnings; customer satis- Also, the value of retaining a given job tinuing to explore the issue of general faction; and reduced welfare dependen- may not be high if it translates to a lack benchmarks and measurable goals, ob- cy. of mobility. jectives and outcomes, no other state proposed the kinds of scientific sur- Iowa placed specific timeframes Learning Outcomes veys, process evaluations, and net im- around these system elements utilizing pact studies recommended by the these guiding principles: a focus on the n Possession of a GED, a post-second- Washington workforce board. customer, an emphasis on system out- ary educational certificate, a license, comes, and a commitment to shared ac- or a degree are important indicators Mid-Project Progress countability, continuous improvement, of learning outcomes. greater collaboration in workforce de- In November 1994 the states partici- velopment planning and execution, and n Multiple measures are needed, how- pating in the NGA project met to report flexibility. ever, including “skill attainment”, on their progress. The informal reports “job placement” and “job retention.” they shared were guided by NGA’s pri- New York’s benchmark-develop- orities for covering the key elements of ment process consisted of three phases: There was unanimity about measur- performance management systems. In 1) planning, 2) data collection and ing skill attainment, but respondents general, these elements were the fol- analysis, and 3) reporting, building on acknowledged the difficulty of devising lowing: twenty key components derived from a measures in the absence of well-de- study of the research and business liter- fined credentials and licenses for most n The definition of “accountability.” ature, federal legislation, and other occupations. They were hopeful that n The mission of the workforce sys- states’ efforts. System goals gave spe- the National Skills Standards Board tem. cial attention to simplicity, quality and under Education Goals 2000 would re- accountability: duce these problems. Some experts n The system’s purpose. n Efficient and comprehensive plan- suggested that the first step should be n Long-term benchmarks. to define occupational and/or industry ning and policy direction. n skills standards needed for a given oc- Shorter-term policy goals. n Service integration. cupation and/or industry, and then al- n The system’s building blocks. n Targeting of education and training low those standards to shape training n The key features of the system. services to customers. curricula and programs. n Its participants and stakeholders. n Responsiveness to world-class edu- Project States’ Accountability cational and training standards. Concepts n Its set of roles and responsibilities: federal, state and local. n Resource allocation tied to the prior- The states that applied for and were itization of activities and their dem- n The target groups for service. selected to participate in NGA’s Perfor- onstrated effectiveness. mance Management Project proposed n The larger environment of the sys- North Carolina’s report outlined the similar concepts and tasks consistent tem. workforce system’s benchmarks, strate- with NGA’s framework. However, n The outcomes expected of the sys- there was a range of responses to the gic policy recommendations, policy tem, given its goals. RFP related to the unique situation of

Evaluation Forum n Issue 11 n Summer 1995 131 Evaluation Issues and Activities n At the State and Local Level

Figure 2 n Characteristics of Applications Submitted by States Selected for NGA Project

Illinois Previous Accomplishments Technical Assistance Requested

n Task Force on Human Resource De- n Establishment of subcouncil to define n Identification of models for support- velopment developed 7 goals: broad occupational clusters and skill ing continuous program improve- u To develop comprehensive early standards applicable across indus- ment. childhood development system. tries. n Identification of models for develop- u To establish standards for educa- n Development of educational guaran- ing system-wide definitions, mea- tional performance. tees for statewide community college sures, and incentives/sanctions. system. n u To institute comprehensive school- Development of strategies for ob- to-work transition system. n Development of Inventory of Ac- taining better data sources, coordi- countability Measures containing 35 nating automated data systems and u To develop system giving assis- tance to out-of-school youth and measures for judging students’ out- networks, and sharing data. disadvantaged adults failing to comes. n Identification of strategies for as- meet educational standards. n Development of operational frame- sessing program participants’ skills work for using performance mea- across different assessment instru- u To develop system for awarding technical/professional certificates/ sures/standards and evaluation strat- ments. degrees based on mastery of skills. egies within a continous program n Development of performance mea- improvement system for secondary sures for use at the local level con- u To urge employers to invest in re- and postsecondary vocational educa- sistent with state measures. training as major strategy for high tion programs. performance work organizations. n Development of employment/earn- u To provide access to coordinated ings measures for using UI data to system of education, training, case judge program performance. management and supportive ser- vices for youth/adults facing em- n Development of automated system ployment barriers. for tracking pre/post information on employment, earnings, UI receipt and n Development of coordination criteria job retention for all participants in involving all relevant agencies. state’s workforce preparation pro- n Requirement that single Compre- grams. hensive Workforce Preparation n Establishment of state/local partner- Agreement be developed for each ship to achieve State Goals for service area involving vision and Learning. measurable goals and objectives. n Agreement on twelve outcomes and n Development of multi-agency, state- their measures, for use in improving wide School-to-Work Transition Sys- workforce preparation activities. tem.

Iowa Previous Accomplishments Technical Assistance Requested

n Creation of Target Alliance estab- o Stimulation of efforts to influence n Development of benchmarking lished by governor and representing laws and regulations inhibiting col- process. six state agencies, to study the laboration and coordination. n Identification of “best practices” in workforce preparation system & de- o Implementation of a collaborative developing quantitative measures velop a theme and general ideas for process to develop a comprehen- consistent with benchmarks. achieving a more integrated system: sive State Human Investment Poli- n Consideration of fund search u Theme: greater attention to “hori- cy and appointment of a Work- strategies. zontal systemic connections” be- force Development Coordinator. n Development of strategies and tween programs and organizations n Development of funds for local work- methods for studying processes and to support local efforts to cross or- force development centers. outcomes re: continuous program ganizational territories in providing n Development of outcome-based improvement. services. School-to-Work Iowa Transition model. n Identification of performance u Ideas: n Creation of Council on Human Invest- measures used at federal level and o Consideration of establishing an ment to set long-term human invest- by other states. overall workforce development ment goals and benchmarks, and to council representing all programs. conduct customer satisfaction surveys. Figure 2 n Continued

132 Evaluation Forum n Issue 11 n Summer 1995 Evaluation Issues and Activities n At the State and Local Level

Figure 2 n Continued

New York Previous Accomplishments Technical Assistance Requested

n Governor’s Plan for Coordination of mends assessment methods for as- n Identification of sources of informa- Workforce Development and NYS suring continuous program improve- tion for reducing barriers to the de- Strategy for Workforce Excellence ment. velopment and use of common out- support a comprehensive customer- n ACCESS program provides compre- come definitions and definitions of oriented service system, skilled hensive educational and training/re- core competencies. workforce and high quality services. training services via one site using a n Identification of ‘best practices’ in Concepts involved are: common Client Assessment Frame- other states. u Development of measurable out- work. n Assistance with waiver requests. comes for entire workforce system. n GATEWAY initiative increases coor- u Redesign of education/training sys- dination of goals/objectives across tem. workforce preparation programs and u Enhancement of job development/ creates network of coordinated ser- placement. vices in 24 sites, and provides infor- mation about ‘best practices’ in inte- n Workforce Preparation Evaluation Act requires state agencies respon- grating customer services. sible for administering workforce n Career Pathways School-to-Work preparation programs to prepare an- Opportunities Initiatives test an inte- nual program plans, identify funding gration of workforce preparation with sources, and report actual program academic education. outcomes against stated program n Education for Gainful Employment le- goals and objectives. verages additional JOBS funds to ex- n Educational community’s New Com- pand the range of options for AFDC pact for Learning emphasizes prepa- clients regarding educational and ration for meaningful work, sets training alternatives. learning standards, and recom-

North Carolina Previous Accomplishments Technical Assistance Requested

n Governor’s Commission on Work- u Literacy Task Force is developing n Development of statewide goals and force Preparedness is responsible strategic state plan for increasing standards and program- level stan- for creating a coherent, comprehen- adult literacy, with assistance from dards, and the design of system- sive workforce development system the Corporation for Enterprise De- wide information systems to track with common vision and goals velopment. achievement. across all relevant programs. Is ex- u High Performance Task Force has n Development of quality criteria for pected to develop a 4-year strategic produced report on characteristics self-assessment and other evalua- plan and a 2-year plan which in- of high performance firms, assisted tive purposes. cludes state workforce goals and ob- by a national consultant. jectives. The Commission has a u Welfare Reform Task Force is de- number of task forces that are re- veloping a customer and outcome- sponsible for different dimensions of focused strategic plan for reforming the workforce system: the state welfare system. u School-to-Work Task Force is im- n Interagency Coordinating Council plementing statewide business/ed- provides technical advice to the ucation initiative, JobReady, to cre- Commission on policy, needs, infor- ate a school-to-work system, mation sharing and continuous pro- including development of outcome gram improvement. Has developed a strategies such as certificates of model for a comprehensive workforce mastery and competency mea- preparation system, Virtual Work- sures — with assistance from the force Preparedness Paradigm. University of Maryland’s Center for Learning and Competitiveness.

Figure 2 n Continued

Evaluation Forum n Issue 11 n Summer 1995 133 Evaluation Issues and Activities n At the State and Local Level

Figure 2 n Continued

North Carolina Previous Accomplishments Technical Assistance Requested n Commission on Standards and Ac- the application of “total quality man- countability is setting new standards agement” throughout state govern- and skill measures for public school ment. system. n North Carolina has also developed a n Commission for a Competitive North preliminary set of categories for out- Carolina is developing four-year stra- come and performance standards, tegic plan for economic development, and actual measures and standards a performance-based budgeting within this categories, across the en- strategy, and a plan for increasing tire system.

Texas Previous Accomplishments Technical Assistance Requested

n The Texas Council on Workforce u Development of uniform state-wide n Quality improvement process. and Economic Competitiveness (an client application and eligibility de- n Data system models. HRIC and a separate state agency) termination system for major work- n Integration of adult education. was established to unify policy de- force development programs. n Waivers. velopment, planning and evaluation u Development of intake/assessment of workforce development efforts. process that can distinguish be- The Council is responsible for devel- tween those for whom education oping a strategic state plan which and training are appropriate vs. ties funding to performance and in- those for whom basic labor market cludes goals, objectives and perfor- information, job search assistance mance measures for all workforce and placement are appropriate. development programs. u Development of state/local strategic The Council has an Evaluation and planning, evaluating and account- Performance Committee and a num- ability system to enable the state to ber of workgroups under it. Purpos- assess results achieved and “return es of the Council are to: on investment”, and to improve the u Develop industry-based skills stan- quality of programs and their deliv- dards/certification system. ery systems. u Develop comprehensive labor mar- u Improvement in individuals’ literacy, ket information system. basic education and basic work- u Design school-to-work transition place skills, including the proportion process. completing secondary and post- u Create local workforce boards and secondary programs. service centers. u Adoption of standard definitions and u Develop statewide system for eval- levels of achievement of workplace uating the effectiveness and effi- basic skills, and incorporation of ciency of all workforce develop- these standards into education and ment programs. training programs. n Texas Tomorrow is the current stra- u Preparation of youth for meaningful tegic plan for the workforce prepara- work and life-long learning. tion system, developed by the Coun- n Current strategic plan has two parts: cil. Goals and objectives include: the first is on vision, philosophy, mis- u Consolidation of workforce devel- sion, system goals, and program opment boards and creation of goals and objectives under each one-stop centers at local level. goal, and has been completed; the second is on core performance mea- sures which is in process.

Figure 2 n Continued

134 Evaluation Forum n Issue 11 n Summer 1995 Evaluation Issues and Activities n At the State and Local Level

Figure 2 n Continued

Texas Previous Accomplishments Technical Assistance Requested

n Related efforts: skill jobs and produce a quality u Workforce Development Initiative workforce through coordinating ed- for Youth is the Texas school-to- ucation and training. work project and will include per- u The Texas Assessment of Academ- formance standards. ic Skills involves quantifiable educa- u The New Texas Prosperity is a gu- tional objectives. bernatorial economic development u Texas community colleges now op- effort. erate within performance standards. u The Smart Jobs Plan is a compre- hensive governor-sponsored strat- egy to develop high-wage, high-

Washington Previous Accomplishments Technical Assistance Requested

n Investment in Human Capital study o To promote competency-based n Information about ‘best practices’ in identified employer/employee needs education and training with equal other states. and organizational barriers in state’s emphasis on acquisition of aca- n Identification of competent consult- workforce development programs demic and occupational skills. ants. and evaluated the performance of o To leverage and generate addi- n Federal waivers. these programs. It established a tional resources for an integrated standard for how workforce systems, n Federal developments relevant to state-wide education and training workforce development systems. as well as programs, could be evalu- system. ated scientifically. Major recommen- n Washington’s role in the National dations from the study were: Conference of State Legislature’s u Improve coordination. Jobs for the Future project: one of u Increase accountability. five states involved, focusing on u Create a state coordinating board school-to-work transition, skills up- for workforce education and train- grading, and coordination of work- ing. force system. n Creation of the Washington State n Washington’s new Performance Part- Education and Training Coordinating nership: brings governor, legislature, Board (WETCB) by the Washington state employees and private sector State Legislature, to be appointed by together to create a performance the governor. This board is responsi- management system for all elements ble for designing an accountability of state government. Will focus on system for workforce education, the development of measurable out- training and employment, as well as comes and continuous program im- for policy development, planning and provement. coordination re: a customer-driven n WETCB’s state workforce develop- system. Its vision and mission are: ment plan: establishes benchmarks u Vision: “to develop a globally com- and measurable goals and objectives petitive workforce supported by an for workforce preparation system. accessible, flexible, competency- u The Board has developed prelimi- based and technologically current nary Targets for Excellence for the education and training system”. workforce system. Utilizing the NGA u Mission: project, it has continued its analysis of issues surrounding the develop- o To establish new workforce part- nership among labor, business, ment of measurable system goals education and government. and performance standards, and the coordination of information sys- o To improve interprogram coordi- tems. nation.

Figure 2 n Continued

Evaluation Forum n Issue 11 n Summer 1995 135 Evaluation Issues and Activities n At the State and Local Level

Figure 2 n Continued

Washington Previous Accomplishments Technical Assistance Requested

u Accountability section within the six years on a range of issues re- new state workforce plan is consis- lated to workforce preparation. tent with Board’s accountability o A survey of employers every six mandate: years. o Assessment of state education o Surveys (and focus groups) to de- and training needs. termine customer satisfaction ev- o Establishment of an inventory of ery two years. programs. o Periodic conducting of a compre- o Evaluation of the outcomes of the hensive process and impact study workforce system. of programs within the workforce o Development of minimum stan- system: a process study of inter- dards for data collection and pro- program coordination, private sec- gram evaluation. tor participation and other issues of interest; a net impact and cost/ o Development of reliable, valid and accessible data base for the sys- benefit study of major new pro- tem. grams. Accountability section recommends u Matching of participants’ program the following: outcomes at approximately nine months and at five years beyond u A number of scientific surveys: program participation, with UI data o A state-wide household survey to determine earnings and obtain and data match with UI data every other information.

136 Evaluation Forum n Issue 11 n Summer 1995 Evaluation Issues and Activities n At the State and Local Level

Figure 3 n Other Characteristics of the States Selected for the NGA Project

Measurable Goals, Objectives, and Outcomes Monitoring and Evaluation Organizational Arrangements Illinois

n Will develop common set of measur- n Will develop alternative models for n Governor’s Task Force on Human able outcomes for each group, as achieving continuous program im- Resource Development is mandated well as standards for each, which provement, and make recommenda- to shape integrated, performance- apply across all programs. tions. driven human resource system, so will provide the leadership re: NGA project. Updates, focus groups, public hearings, articles in agency newslet- ters will be used to update all relevant agencies, employers, public.

Iowa n Will develop policy goals for work- n Will develop a common performance n A state Workforce Development force preparation system, and management system for all programs. Council will be created, associated benchmarks for achieving them for n Will develop a method for monitoring with the Human Investment Policy each population group, with the as- and judging performance across sys- Council. Meanwhile an existing inter- sistance of a formal Research tem, including customer satisfaction agency policy team will be responsi- Group from three state universities. and service quality. ble for the project, assisted by a n Technical Work Group. Will develop common performance n Will develop measures to assist ser- measures and definitions for each vice providers in implementing con- n Service providers, customers, com- population group, to be used at both tinuous program improvement. munities and others will be invested the state level and at the level of lo- in the process through symposia n Will develop self-assessment tools. cal workforce centers. and other forums and through a n Will develop an integrated automated Special Communications Group. information system to support inter- agency efforts. n Will seek to identify ‘best practices.’

New York

n Will identify system-wide bench- Will use Evaluation Act to support de- n NY State Job Training Coordinating marks for each population group us- velopment of common set of outcomes Council will coordinate existing state ing outcome-based measures and and strategies for continuous program workforce agencies, councils and re- process (quality) standards, and re- improvement. lated organizations via a new project lying on skills assessment/certifica- n Will seek to pinpoint “best practices.” team. tion system to be developed by Na- n Will develop a “feedback loop” to in- tional Skills Standards Board under crease utilization of program/UI data Goals 2000. for continuing program improvement. n Will develop MIS to collect, store, n Will develop economic incentives and and analyze outcome data (also UI sanctions to support the use of analy- data). ses to improve outcomes. n Will study barriers to implementing a comprehensive performance man- agement system. n Will develop strategic plan for work- force preparation system specifying goals, benchmarks, objectives, mea- surable outcomes, and information system, drawing on US DOL’s Core Data Elements and Common Defini- tions . n Will design pilot project to test viabil- ity of concepts in the state plan. Figure 3 n Continued

Evaluation Forum n Issue 11 n Summer 1995 137 Evaluation Issues and Activities n At the State and Local Level

Figure 3 n Continued

Measurable Goals, Objectives, and Outcomes Monitoring and Evaluation Organizational Arrangements North Carolina

n Will develop and define quantifiable n Will focus on a number of elements n Governors’ Commission on Work- state goals for each client group of a quality assurance system, such force Preparedness (NC’s HIRC) will across entire workforce preparation as continuous program improvement, have ultimate responsibility, but an system. program accreditation, and capacity expanded Interagency Coordinating n Will identify both outcome-based building. Council will oversee NGA project. measures, definitions and stan- n Will develop methods for tracking dards, and measures and definitions system progress in meeting state for assessing service delivery and goals. service quality for diverse clients. n Will develop common set of perfor- mance standards for all programs in workforce system. n Will rely on Secretary’s Commission on Necessary Skills and Goals 2000 in defining basic skills and compe- tencies. n Will recommend changes in legisla- tion and regulation re: waivers.

Texas n Will refine performance measures n Will link performance management n Texas Council’s Evaluation and Per- for state plan goals and objectives. system with system-wide continuous formance Committee will oversee n Will use the eight core performance program improvement and quality as- NGA project. measures developed in collaborative surance systems. n Committee’s Performance Measure- process by Texas Council on Work- n Will develop evaluation process for ment Workgroup will serve as inter- force and Economic Competitiveness continuous program improvement agency policy group. as general framework for NGA project. and quality service delivery. n Will view development and implemen- tation of performance management system as six-year project involving: u Creation of inventory of current in- formation systems and consider- ation of ways to connect them. u Development of plan for coordinat- ing information systems. u Reduction of barriers to coordina- tion/integration at all levels. u Formation of workgroups to devel- op implementation strategies.

Figure 3 n Continued

138 Evaluation Forum n Issue 11 n Summer 1995 Evaluation Issues and Activities n At the State and Local Level

Figure 3 n Continued

Measurable Goals, Objectives, and Outcomes Monitoring and Evaluation Organizational Arrangements Washington n Will establish benchmarks and mea- n Will further develop and refine evalu- n WETCB is coordinating the NGA surable goals, objectives, outcomes, ation system for judging both the project. Its established Interagency and performance standards. workforce system’s and its programs’ Committee will play a major role, u Will further refine Targets for Ex- progress against goals and objec- and a representative of governor’s cellence. tives based on scientific principles office will link project with state’s and methods. new Performance Partnership. A u Will consider more thoroughly the pros and cons of using perfor- n Accountability section of new state technical workgroup on accountability mance standards. workforce development plan will will have operational responsibility. serve as framework for further devel- n Data system coordination/integra- tion: efforts will be focused on a opment and refinement of methods more coordinated and integrated for ensuring accountability. Special workforce development information consideration will be given to: system, emphasizing: u Using skills standards and compe- tency-based assessments as part of u Common set of data elements and definitions. system-wide and program-specific evaluations. u Common reporting requirements. u Using program evaluations to sup- u Participant confidentiality protec- port continuous program improve- tions. ment and quality assurance. u Reduction of data duplication. u Developing reports for customers of u Information portability. the workforce system (workers, em- u Staff training needs. ployers and other stakeholders) to u State-of-the-art computer hardware increase informed choice among and software needed. education, training, and employ- ment options.

Evaluation Forum n Issue 11 n Summer 1995 139 Evaluation Issues and Activities n At the State and Local Level

goals, and more specific objectives. programs within the system, and the der over-arching state boards or coun- Accountability efforts were to focus on utilization of evaluation results for con- cils predated this shift, however. This retooling the existing workforce, pre- tinuous quality improvement. The poli- movement was encouraged by the paring the emerging workforce, and cy outcomes focused on employment, evolving positions of national employ- building the workforce infrastruc- earnings, skill competencies, worker ment and training organizations and ture — and on performance-based bud- productivity, the reduction of poverty, governors, and by incentives from the geting in the areas of social and eco- customer satisfaction, and the system’s U.S. Department of Labor as part of nomic well-being, corrections, health general return on the public’s invest- Secretary Reich’s effort to move the and safety, economic development, the ment. The Washington report advocat- employment and training system from a environment, justice, education, agri- ed the use of multiple measures for largely-fragmented unemployment sys- culture, cultural resources, and general each of the major desired outcomes, to tem to a more collaborative and inte- government. Measurable objectives increase the precision and validity of grated reemployment system. were being developed for each area of outcome evaluations. Motivated and funded quite differ- interest, as well as outcome measures ently by these disparate supporters of consistent with policy goals. Of partic- Some Comments on state-wide workforce initiatives, a com- ular concern were the quality of jobs, the NGA Project mon trend toward state authority and employment retention, and poverty re- A number of issues are involved in innovation in employment and training duction. Additional attention was being the six states’ efforts to develop an ab- (with or without substantial federal given to strategies for strengthening stract system concept, and to describe oversight) has come to pass. The NGA political support and marketing the sys- its potential implementation. Among project has demonstrated what states tem. them are these: can accomplish on their own, with spe- The Texas report focused on desired n How do planners move logically and cial assistance from national experts, system outcomes and quantitative mea- realistically from abstract missions, not only in developing workforce sys- sure for identifying them, based on five benchmarks and goals, to more spe- tems but in evaluating them using com- major system goals and a set of objec- cific objectives and measurable out- petent social science principles and tives under each. The goals emphasized comes? methods. 1) the development of a statewide sys- tem supporting local workforce devel- n Must all the outcomes be measur- opment centers and providing a net- able quantitatively? Editor’s Note work of services and information to n How should “continuous improve- The NGA references for this article customers, 2) the creation of a state/lo- ment” and “quality assurance” be cal strategic planning and evaluation are listed below. They were working pa- defined, in order for planners to re- pers provided to participants of the Per- subsystem for judging the performance alistically monitor and evaluate formance Management Project in June of the system, 3) the acquisition of these attributes of systems? and July of 1994. literacy, basic education and basic workplace skills by customers, 4) the n How much should be expected, Developing Comprehensive Perfor- pragmatically, of government/pri- mance Management for the Work- acquisition of occupational skills by force System: Project Description. customers that meet workplace require- vate sector partnerships in achieving ments for long-term sustaining employ- the more idealistic visions of work- Glossary of Project Terms. ment in high-skill, high-wage occupa- force systems promoted by the kind Issues in Developing Comprehensive tional areas, and 5) the preparation of of system planning efforts encour- Performance Management for Work- youth through obtaining knowledge, aged in the NGA project? force Development Systems. skills and behaviors necessary for the n Should new workforce systems be Texas’s Process for Setting Core Per- transition to high-skill, high-wage car- permitted to function for a time be- formance Measures. rers and through lifelong learning. De- fore crystallizing the outcomes to be sired outcomes included labor market, expected of them, and developing Attachment C of the report prepared by the Texas Council on Workforce and learning, customer satisfaction, and performance standards? system equity outcomes. Economic Competitiveness summariz- Although many such questions can ing the results of a survey of national Washington explained the state’s emerge from these impressive planning performance management specialists. performance management framework activities in states, the NGA project is in its report, as well as describing sys- particularly important in the context of tem policy goals and outcomes. This the renewed political interest in budget framework emphasized the intercon- balancing, greater state-level discretion, nections among components of the and the consolidation of programs and workforce system: state-wide policy service delivery. The movement to de- goals, measurable outcomes, the data velop state-wide workforce systems un- system infrastructure, the outcomes of

140 Evaluation Forum n Issue 11 n Summer 1995 Evaluation Issues and Activities n Editorial Commentary

Editorial Commentary ed program and service delivery coordi- their approaches, only minimal techni- nation under the aegis of broadly repre- cal assistance has been available. States have long recognized that sentative state-wide boards or councils Therefore, each state’s strategy for some workforce development policies whose responsibility was to set policy, workforce coordination has been some- and procedures have been conflicting or develop short and longer-term plans, what idiosyncratic, beyond the necessi- incompatible. This they knew was dam- promote coordination, and establish ac- ty to respond to state-specific needs aging service quality, precluding inter- countability for all state workforce-relat- and interests. And accurate, detailed in- agency collaboration, and reducing the ed programs and services. These enti- formation on the nature and status of ability of participants to move into and ties were becoming the major tool for workforce coordination efforts across between programs. Simultaneously, seeking better integration while reduc- diverse states has been hard to come states were appreciating the importance ing costs and making programs more by. Only time-limited, selective “snap- of encouraging the development of accountable for achieving broad, cross- shots” were available, which artificially high-skill, high-wage workforces, which cutting goals. froze states’ activities for a moment, ac- would demand a reorganization of the The emphasis was now on state tivities that were an important part of an workforce system. workforce systems, rather than on an evolutionary process that was constant- To pull education and skill training increasingly unwieldy collection of pro- ly changing. together better, many states introduced grams with diverse goals and regula- In the midst of continuing experi- educational reforms in the 1990s to tions. These systems were to involve mentation with coordination strategies, achieve a better linkage between aca- clear-cut benchmarks, goals, objectives there has been a strong steady move- demic knowledge and workforce skills, and performance indicators. Locating ment to ensure the “accountability” of stimulated by the National Education these tasks at the state level supported new more comprehensive and integrat- Goals established by the President and the decentralization of authority and re- ed workforce development systems. the governors in 1989. Some states cre- sponsibility launched in the 1980s. The methods for establishing such ac- ated new state boards or councils re- Such decentralization was to permit countability have moved ever closer to sponsible for the full gamut of workforce the federal government to restructure it- evaluation research standards. The programs and special projects. Some self, and later to “reinvent” itself, as a NGA project is an example of this com- were moving toward coordinated, some- leaner, more coordinated, and more mitment to making professional judg- times integrated, information systems efficient body. ments of the results of system policies that would support these new statewide Decentralization has meant, howev- and activities, and the impact of system systems and their evaluation. er, that states have had few national interventions for workers and employ- The JTPA amendments spawned models or guidelines to assist them in ers. Obtaining such objective results is some of these human resource boards. pursuing workforce integration. Apart now an accepted input to comprehen- Reflecting Congressional interest in co- from their own efforts to share informa- sive workforce policy development and ordination, these amendments promot- tion with other states and learn from planning.

Evaluation Forum n Issue 11 n Summer 1995 141 Resources

Editorial Introduction mainly on U.S.- specific issues; publica- double asterisk (**) those that deal tions providing insights about other more directly with worker dislocation In the first part of this section of countries; and a special bibliography on and displacement. Evaluation Forum we sample referenc- workforce development coordination is- The second part of the Resources es from the plethora of publications on sues. We have indicated with a single Section updates readers regarding economic change, restructuring and asterisk (*) the publications in the first some of the federal research conducted worker dislocation. The list of useful two categories which deal mainly with since the last issue of the journal. This books, articles, reports and profession- general economic issues, as a context is a limited overview, based on the re- al papers is loosely separated into for learning more about the context of sponses of relevant federal agencies to three categories: publications focusing worker dislocation, and indicated with a the editor’s request for information.

References on Economic Change, Restructuring and Worker Dislocation

Information about the United States *Abraham, K. and S. Houseman. Job **Baily, M. “Wages and Employment *Becker, G. “Investment in Human Security in America: Lessons from Under Uncertain Demand” in Re- Capital: A Theoretical Analysis” in Germany. Washington, D.C.: The view of Economics and Statistics, Journal of Political Economy, 70, Brookings Institution, 1993. 41, 1974. 1962. *Abraham, K. and L. Katz. “Cyclical *Baily, M. and A. Chakrabarti. Innova- **Bednarzik, R. and M. Lovell. U.S. Unemployment: Sectoral Shifts or tion and the Productivity Crisis. Labor Market Adjustment Pro- Aggregate Disturbances?” in Jour- Washington, D.C.: The Brookings grams. Paper prepared for the Presi- nal of Political Economy, 94, 1986. Institution, 1988. dent’s Competitiveness Policy **Aho, C. and J. Orr. “Trade-Sensitive **Balderson, K. M. Plant Closings, Council, 1991. Employment: Who Are the Affected Layoffs, and Worker Readjustment. **Bednarzik, R. Trade-Sensitive U.S. Workers?” in Monthly Labor Review Washington, D.C.: National Gover- Industries: Employment Trends and 104, 1981. nors’ Association, 1986. Worker Characteristics. Washing- ton, D.C.: Bureau of International **American Management Association: **Baldwin, S. Technology and Em- Labor Affairs, U.S. Department of 1993 Survey on Downsizing and As- ployment Policy. Washington, D.C.: Labor, 1991. sistance to Displaced Workers. New National Commission for Employ- York, 1993. ment Policy, 1987. **Bendick, M. The Role of Public Pro- grams and Private Markets in Reem- *Ashenfelter, O. “Estimating the Ef- **Barton, P. A Better Fit Between Un- ploying Workers Dislocated by Eco- fects of Training Programs on Earn- employment Insurance and Retrain- nomic Change. Washington, D.C.: ings,” in Review of Economics and ing. Washington, D.C.: National In- The Urban Institute, 1982. Statistics, 60, 1978. stitute for Work and Learning, 1986. **Bailey, T. “Changes in the Nature *Barnow, B. The Education, Training **Benus, J. and R. Byrnes. The St. and Structure of Work: Implications and Work Experience of the Adult Louis Metropolitan Reemployment for Skills and Skill Formation.” Re- Labor Force from 1984 to 1995. Project: An Impact Evaluation. port of the Conservation of Human Washington, D.C.: National Com- Washington, D.C.: U.S. Department Resources. New York, NY: Colum- mission for Employment Policy, of Labor, 1991. bia University, November 1989. 1985.

142 Evaluation Forum n Issue 11 n Summer 1995 Resources n References on Economic Change, Restructuring and Worker Dislocation

**Benson, D. Workforce Development: *Brooks, H. and L. Schneider. Poten- **______The New Jersey Unem- A Survey of Ohio Employers. Co- tial Impact of New Manufacturing ployment Insurance Reemployment lumbus, OH: Appropriate Solutions, Technology on Employment and Demonstration Project. Washing- 1993. Work in Industrial and Developing ton, D.C.: U.S. Department of La- Countries. Policy paper. Cambridge, bor, 1989. *Blackburn, M. and D. Bloom. The Ef- MA: John F. Kennedy School of fects of Technological Change on **______Process and Imple- Government, Harvard University, the U.S. Distribution of Earnings mentation Issues in the Design and 1985. and Income. Paper for Panel on Conduct of Programs to Aid the Re- Technology and Employment, 1987. **Bruno, L. Operating Effective Reem- employment of Dislocated Workers. **Blalock, A.B. “Analyzing Worker ployment Strategies for Displaced Princeton, NJ: Mathematica Policy Displacement in the 1980s: The Workers. Paper for U.S. Department Research, Inc., 1984. Landmark Reports of the Office of of Labor by CRS Incorporated, **______International Trade Technology Assessment,” in Evalu- 1986. and Worker Dislocation: Evaluation ation Forum, Issue #7, Summer *Carnevale, A. America and the New of the Trade Adjustment Assistance 1992. Economy. Washington, D.C.: Amer- Program. Princeton, NJ: Mathemati- ican Society for Training and Devel- ca Policy Research, Inc., 1993. *Blinder, A. Paying for Productivity: A opment and the U.S. Department of Look at the Evidence. Washington, **Creticos, P.A. and R.G. Sheets, Eval- Labor, 1991. D.C.: The Brookings Institution, uating State-Financed Workplace- 1990. *Carnevale, A. and H. Goldstein. Em- Based Retraining Programs. Wash- ployee Training: Its Changing Role ington, D.C.: National Commission *Bloom, H. Estimating the Long-Run and An Analysis of New Data. for Employment Policy and the Na- Effects of Job Training Using Longi- Washington, D.C.: American Soci- tional Governors’ Association, tudinal Earnings Data. Cambridge, ety for Training and Development, 1992. MA: John F. Kennedy School of 1986. Government, Harvard University, *Cyert, R. and J. March. A Behavioral 1982. *Chinloy, P.T. and E. W. Stromsdorfer Theory of the Firm. Englewood ______Back to Work: Testing Re- (Editors) Labor Market Adjustments Cliffs, NJ: Prentice-Hall, 1963. employment Services for Displaced in the Pacific Basin. Boston, MA: *Dertouzos, M., R. Lester and R. So- Workers. Kalamazoo, MI: W.E. Up- Klewer-Niuhoff, 1987. low. Made in America: Regaining john Institute for Employment Re- *Chubb, J. and R. Moe. Politics, Mar- the Productive Edge. Cambridge, search, 1990. kets and America’s Schools. Wash- MA: MIT Press, 1989. ington, D.C.: The Brookings Institu- *Bluestone, B. and B. Harrison. The *Duncan, G. et al. Private Lives and tion, 1990. Great American Job Machine: The Public Policies: Confidentiality and Proliferation of Low Wage Employ- *Cole, R. Work, Mobility, and Partici- Accessibility of Government Statis- ment in the U.S. Economy. Paper for pation. Berkeley, CA: University of tics. A report by the Panel on Confi- Joint Economic Committee, U.S. California Press, 1979. dentiality and Data Access. National Congress, 1986. Research Council. Washington, *Committee for Economic Develop- D.C.: National Academy Press, *______The Deindustrialization ment: An America That Works: The 1993. of America: Plant Closings, Com- Life-Cycle Approach to a Competi- munity Abandonment, and the Dis- tive Workforce, 1990. *Edwards, R. Rights At Work: Employ- mantling of Basic Industries. New **Cook, R. “Insights Concerning Suc- ment Relations in the Post-Union York, NY: Basic Books, 1982. cessful Strategies in Dislocated Era. Washington, D.C.: The Brook- *Boyd, W. and C. Kerchner (Editors). Worker Projects,” in Evaluation Fo- ings Institution, 1993. The Politics of Excellence and rum, Issue #2. *Ehrenberg, R. Labor Markets and In- Choice in Education. New York, tegrating National Economies. **Corson, W. et al. An Impact Evalua- NY: Falmer Press, 1988. Washington, D.C.: The Brookings tion of the Buffalo Dislocated Work- Institution, 1995. *Brock, W. and R. Hormat. (Editors) er Demonstration Program. Prince- The Global Economy: America’s ton, NJ: Mathematica Policy Role in the Decade Ahead. New Research, Inc., 1985. York, NY: W.W. Norton, 1990.

Evaluation Forum n Issue 11 n Summer 1995 143 Resources n References on Economic Change, Restructuring and Worker Dislocation

**Fechter, A. Forecasting the Impact *Hirschhorn, L. Beyond Mechaniza- *Katz, L. and B. Meyer. “Unemploy- of Technological Change on Man- tion: Work and Technology in a ment Insurance, Recall Expecta- power Utilization and Displacement. Postindustrial Age. Cambridge, MA: tions, and Unemployment Out- Working paper. Washington, D.C.: MIT Press, 1984. comes,” in Quarterly Journal of The Urban Institute, 1974. Economics 105, 1990. **Hollenbeck, K. Dislocated Worker **Fedrau, R. H. “Responses to Plant Human Capital Depreciation and *Kennedy, P. Preparing for the Twen- Closures and Major Reductions in Recovery. Working Paper. Kalama- ty-First Century. New York, NY: Force: Private Sector and Communi- zoo, MI: W.E. Upjohn Institute for Random House, 1993. ty-Based Models,” in The Annals of Employment Research, 1990. *Kochan, T. and H. Katz. Worker Par- the American Academy of Political **Horvath, F. W. “The Pulse of Eco- ticipation and American Unions: and Social Science, 475, 1984. nomic Change: Displaced Workers Threat or Opportunity? Kalamazoo, **Flaim, P. and E. Sehgal. “Displaced of 1981-1985,” in Monthly Labor MI: W.E. Upjohn Institute for Em- Workers of 1979-1983: How Well Review 110, 1987. ployment Research, 1984. Have They Fared?” in Monthly La- *Huang, W. (Editor) Organized Labor *Kochan, T. et al. The Transformation bor Review 108, 1985. at the Crossroads. Kalamazoo, MI: of American Industrial Relations. *Fretwell, D. et al. Employment Dimen- W.E. Upjohn Institute for Employ- New York, NY: Basic Books, 1987. sions of Economic Restructuring: A ment Research, 1989. **Kulik, J., D. Smith and E. Stroms- Review of Related Labor Policies *Hunt, H. and T. Hunt. “An Assess- dorfer. The Downriver Community and Programs in Industrialized ment of Data Sources to Study the Conference Economic Readjustment Countries. The World Bank, 1991. Employment Effects of Technologi- Program. Cambridge, MA: Abt As- *Graham, C. Safety Nets, Politics, and cal Change,” in Technology and Em- sociates, Inc., 1984. The Poor. Washington, D.C.: The ployment Effects, report of the Panel *Krugman, P. The Age of Diminished Brookings Institution, 1994. on Technology and Women’s Em- Expectations: U.S. Economic Policy *Guile, B.R. and H. Brooks. (Editors) ployment, National Research Coun- in the 1990s. Cambridge, MA: MIT Technology and Global Industry: cil, 1985. Press, 1990. Companies and Nations in the **Jacobson, L. et al. Earnings Losses *Lawrence, R. “Sectoral Shifts and the World Economy. Washington, D.C.: of Displaced Workers. Working pa- Size of the Middle Class” in The National Academy Press, 1987. per. Kalamazoo, MI: W.E. Upjohn Brookings Review, 3, 1984. Institute for Employment Research, **Hall, R. “The Importance of Lifetime 1992. *Lawrence, R. et. al. A Vision for the Jobs in the U.S. Economy” in Amer- World Economy: Openness, Diversi- ican Economic Review, 72, 1982. **Jensen, M. and J. Lederer: Workforce ty and Cohesion. Washington, D.C.: in Transition: A Guide to the Coor- **Hall, R. and E. Lazear. “The Excess The Brookings Institution, 1995. dination of State Dislocated Worker Sensitivity of Layoffs and Quits to Services. Washington, D.C.: Center **Leigh, D. Retraining Displaced Demand” in American Economic for Policy Research, National Gov- Workers: The U.S. Experience. Review, 69, 1984. ernors’ Association, 1992. Geneva, Switzerland: International **Hamermesh, D. “The Costs of Work- Labour Organization, 1994. er Displacement,” in Quarterly **Johnston, W. and A. Packer. Work- ______Retraining Displaced Journal of Economics 102, 1987. force 2000: Work and Workers for the Twenty-First Century. Indianap- Workers: What Can Developing *Hart, R.(Editor) Employment, Unem- olis, IN: Hudson Institute, 1987. Countries Learn from OECD Na- ployment and Labor Utilization. tions? The World Bank, 1992. *Kasarda, J. The Regional and Urban London, U.K.: Unwin Hyman, 1988. Redistribution of People and Jobs in *Leonard, J. Technological Change *Hartman, H. (Editor) Computer Chips the U.S. Paper for National Research and the Extent of Frictional and and Paper Clips: Technology and Council’s Urban Policy Workshop, Structural Unemployment. Paper for Women’s Employment. Washington, 1986. Panel on Technology and Employ- D.C.: National Academy Press, ment, 1986. 1987.

144 Evaluation Forum n Issue 11 n Summer 1995 Resources n References on Economic Change, Restructuring and Worker Dislocation

**Levitan, S. and F. Gallo. A Second **National Commission for Employ- *Rivlin, A. et al. Beyond Alliances: Chance: Training for Jobs. Kalama- ment Policy: Workers Dislocated by Global Security Through Focused zoo, MI: W.E. Upjohn Institute for Economic Change: Toward New In- Partnership. John D. and Catherine Employment Research, 1990. stitutions for Midcareer Worker T. MacArthur Foundation and the Transformation. Washington, D.C., Rockefeller Foundation, 1990. *Levitan S. and D. Werneke. Produc- 1982. tivity: Problems, Prospects, and *Robertson, D. B. “Political Conflict Policies. Kalamazoo, MI: W.E. Up- **______Displaced Workers: and Lesson Drawing,” in Journal of john Institute for Employment Re- New Options for a Changing Econo- Public Policy, 1991. search, 1984. my. Washington, D.C., 1983. *Rogers, E.M. The Diffusion of Innova- *Levy, F. Dollars and Dreams: The **______The Permanence of tions. New York, NY: The Free Changing American Income Distri- Dislocation: 1979-1983 Evidence. Press, 1983. bution. New York, NY: Basic Washington, D.C., 1986. **Romero, C. et al. The Potential Ef- Books, 1987. **______Assisting Dislocated fectiveness of the Employment Ser- *Lindberg, L. and C. Maier. The Poli- Workers. Washington, D.C., 1991. vice in Serving Dislocated Workers tics of Inflation and Economic Stag- Under EDWAA: Evidence from the **______Upward Mobility Pro- nation. Washington, D.C.: The 1980s. Washington, D.C.: National grams in the Service Sector for Dis- Brookings Institution, 1985. Commission for Employment Poli- advantaged and Dislocated Work- cy, 1991. *Lipset, S.M. Unions in Transition: ers. Washington, D.C., 1992. Entering the Second Century. San *New York Stock Exchange: U.S. In- **Secretary of Labor’s Task Force on Francisco, CA: Institute for Contem- ternational Competitiveness. Economic Adjustment and Worker porary Studies, 1986. Dislocation: Economic Adjustment *Nothdurft, W.E. Going Global: How *Litan, R. et al. (Editors) American and Worker Dislocation in a Com- Europe Helps Small Firms Export. Living Standards: Threats and petitive Society, 1986. Washington, D.C.: The Brookings Challenges. Washington, D.C.: The *Sleigh, S. (Editor) Economic Restruc- Institution, 1992. Brookings Institution, 1988. turing and Emerging Patterns of In- *President’s Commission on Industrial dustrial Relations. Kalamazoo, MI: *Magaziner, I. and R. Reich. Minding Competitiveness: Global Competi- W.E. Upjohn Institute for Employ- America’s Business. New York, NY: tion: The New Reality. Washington, ment Research, 1993. Harcourt Brace Jovanovich, 1982. D.C.: U.S. Government Printing Of- **Stromsdorfer, E.W. “Plant Closings **Mangum, S. and J. Tansky. Self Em- fice, 1985. and Worker Displacement,” in Eval- ployment Training As An Interven- **Podursky, M. and P. Swaim. “Job uation Forum, Issue #7, Summer tion Strategy for Displaced or Dis- Displacement and Earnings Loss: 1993. advantaged Workers. Management Evidence from the Displaced Work- and Human Resources Faculty, Ohio er Survey,” in Industrial and Labor *Summers, L. Understanding Unem- State University, 1987. Relations Review, 41, 1987. ployment. Cambridge, MA: MIT *Mincer, J. Schooling, Experience, and Press, 1990. *Reich, R. Education and the Next Earnings. New York, NY: National **Swigart, R.(Editor) Managing Plant Economy. Washington, D.C.: Na- Bureau of Economic Research, Closures and Occupational Read- tional Education Association, 1988. 1974. justment. Washington, D.C.: Nation- *______The Work of Nations: al Center for Occupational Readjust- *Mishel, L. and R. Teixeira. The Myth Preparing Ourselves for 21st Centu- ment, 1984. of the Coming Labor Shortage: ry Capitalism. New York, NY: Vin- Jobs, Skills and Incomes of Ameri- *Tan, H. Technical Change and Its tage Books, 1992. ca’s Workforce 2000. Paper of the Consequences for Training and Economic Policy Institute. Washing- *Rivlin, A. Reviving the American Earnings. A policy paper, Santa ton, D.C., 1990. Dream: The Economy, the States, Monica, CA: The Rand Corporation, and the Federal Government. Wash- 1987. **Moore, R. and D. Blake. Does ETP ington, D.C.: The Brookings Institu- Training Work? An Analysis of the tion, 1992. Economic Outcomes of California Employment Training Panel Pro- grams, Northridge, CA: California State University, 1992.

Evaluation Forum n Issue 11 n Summer 1995 145 Resources n References on Economic Change, Restructuring and Worker Dislocation

*Temin, P. Lessons from the Great De- *U.S. Departments of Defense, Com- Information About pression. Cambridge, MA: MIT merce and Energy: NASA and the Press, 1989. National Science Foundation. De- Other Countries fense Technology Conversion, Rein- *Thurow, L. “A Surge in Inequality” in vestment, and Transition Assistance. Books and Articles Scientific American, 256, 1987. Washington, D.C. 1993. **Abraham, K. and S. Houseman. Job *______Dangerous Currents: Security in America: Lessons from The State of Economics. New York, **U.S. Department of Labor: Employ- Germany. Washington, DC: The NY: Random House, 1983. ment and Training Administration. Economic Adjustment and Worker Brookings Institution, 1993. **Topel, R. “Inventories, Layoffs, and Dislocation in a Competitive Soci- ______Earnings Inequality in the Short-Run Demand for Labor” ety. Washington, D.C., 1986. Germany. Kalamazoo, MI: W.E. in American Economic Review, 72, **______Study of the Implemen- Upjohn Institute for Employment 1982. tation of the Economic Dislocation Research, 1994. *Ulman, L. et al. (Editors) Labor and and Worker Adjustment Assistance Ayub, M. A. and S.O. Hegstad. Public an Integrated Europe. Washington, Act. Washington, D.C., 1992. Industrial Enterprises: Determi- D.C.: The Brookings Institution, **U.S. General Accounting Office: nants of Performance. The World 1993. please see Part II of the Features Bank, 1986. *U.S. Congress: Congressional Budget Section for GAO reports. *Bamber, G.J. and R. D. Lansbury (Ed- Office. Economic Conversion: What itors) International and Compara- Should Be the Government’s Role. *Vroman, W. and D. Wissoker. Alter- tive Industrial Relations. London, Washington, D.C., 1980. natives for Managing Production Cutbacks. Report for the National U.K.: Allen and Unwin, 1987. **______Dislocated Workers: Is- Commission for Employment Poli- *Beauregard, R. (Editor) Economic Re- sues and Federal Options. Washing- cy. Washington, D.C., 1990. structuring and Political Responses. ton, D.C., 1993. *Walton, R. “From Control to Commit- London, U.K.: Urban Affairs Annu- **______Displaced Workers: ment in the Workplace” in Harvard al Review, 1989. Trends in the 1980s and Implica- Business Review, 64, 1985. *Best, M. The New Competition: Insti- tions for the Future. Washington, *Walton, R. et al. Innovating to Com- tutions of Industrial Restructuring. D.C., 1993. pete: Lessons for Diffusing and Cambridge, England: Polity Press, *U.S. Congress, Office of Technology Managing Change in the Work- 1990. Assessment: Information Security place. San Francisco, CA: Jossey- *Blanchard, O. et al. Reform in Eastern and Privacy in Network Environ- Bass, 1988. Europe. Cambridge, MA: MIT ments, 1994. *Waltman, J. and Studlar, D. (Editors) Press, 1991. **______Technology and Struc- Political Economy: Public Policies *Bosworth, B. and A. Rivlin. (Editors) tural Unemployment: Reemploying in the U.S. and Britain. Jackson, MI: The Swedish Economy. Washington, Displaced Adults. Washington, University Press of Mississippi, D.C.: The Brookings Institution, D.C., 1986. 1987. 1987. **______Trade Adjustment As- *Wilensky, H.L. and L. Turner. Demo- *Boyer, R. The Search for Labour sistance: New Ideas for an Old Pro- cratic Corporatism and Policy Link- Market Flexibility. Oxford, England: gram. Washington, D.C., 1987. ages: The Independence of Industri- Clarendon Press, 1988. **______Worker Training: Com- al, Labor-Market, Incomes, and *Brunetta, R. and C. dell’Aringa (Edi- peting in the New International Social Policies in Eight Countries. tors). Markets, Institutions and Co- Economy. Report by the Advisory Berkeley, CA: University of Califor- operation: Labour Relations and Panel on Workplace Training. nia Press, 1987. Economic Performance. London, Washington, D.C., 1990. England: Macmillan, 1990. *______U.S.-Mexico Trade: *Chitty, C. Towards a New Education Pulling Together or Pulling Apart? System: The Victory of the New Washington, D.C., 1992. Right? London, England: Falmer, 1989.

146 Evaluation Forum n Issue 11 n Summer 1995 Resources n Information about Other Countries

*Cline, W. International Economic *Helpman, E. and A. Razin. Interna- *______: New Technologies in the Policy in the 1990s. Cambridge, tional Trade and Trade Policy. 1990s. Paris, France, 1988. MA: MIT Press, 1994. Cambridge, MA: MIT Press, 1991. **______: Education and Training *Cukierman, A. et al. (Editors) Politi- *Hewitt, E. Reforming the Soviet Econ- After Basic Schooling. Paris, France, cal Economy, Growth, and Business omy: Equality Vs. Efficiency. Wash- 1985. Cycles. Cambridge, MA: MIT Press, ington, DC: The Brookings Institu- *______: OECD Jobs Study: 1992. tion, 1988. Facts, Analysis and Strategies. Par- **Elmeskov, J. and M. MacFarlan. *Hill, C. and J. Pickering. “Divisionalisa- is, France, 1994. “Unemployment Persistence” in tion, Decentralisation and Perfor- *Pavlin, I. Human Resources Under the OECD Economic Studies, 21, 1993. mance of Large U.K. Companies,” in Industrial Restructuring Paradigm in Journal of Management Studies, **Elmeskov. J. and K. Pichelmann. “In- Socialist Countries. Entrepreneurship 1986. terpreting Unemployment: The Role Development Institute of India, 1987. of Labour-Force Participation” in **Houseman, S. and K. Abraham. La- *Osterman, P. Employment Futures: OECD Economic Studies, 21, 1993. bor Adjustment Under Different In- Reorganization, Dislocation, and stitutional Structures: A Case Study *Finegold, D., L. McFarland, and W. Public Policy. New York, NY: Ox- of Germany and the U.S. Kalama- Richardson (Editors) Something ford University Press, 1988. zoo, MI: W.E. Upjohn Institute for Borrowed, Something Learned: The Employment Research, 1993. *Piore, M. and C. Sabel. The Second Transatlantic Market in Education Industrial Divide. New York, NY: and Training Reform. Washington, *Johnstone, D. Enterprise USA: Em- Basic Books, 1984. D.C.: The Brookings Institution, ployment and Training Through 1993. British Eyes. The Planning Ex- *Pirie, M. Privatisation. London, En- gland: Adam Smith Institute, 1985. **Finegold, D. “The Failure of British change, Glasgow, England. Training: Analysis and Prescrip- **Keep, “The Grass Looked Greener,” in **Postlethwaite, N.J. “Research and tion,” in Oxford Review of Economic International Comparisons of Voca- Policy-making in Education” in T. Policy, Autumn 1988. tional Education and Training for In- Husen and M. Kogan (editors) Edu- termediate Skills edited by P. Ryan. cation Research and Policy: How *Fischer, G. and G. Standing. Structur- UK: Basingstoke, Falmer, 1991. Do They Relate? UK: Oxford Press/ al Change in Central and Eastern Pergamon, 1984. Europe: Labour Market and Social *Kochan, R., H. Katz, and R. McKer- Policy Implications. Paris, France: sie. The Transformation of Ameri- **Prais, S.J. “Educating for Productivi- OECD, 1993. can Industrial Relations. New York, ty: Comparisons of Japanese and NY: Basic Books, 1986. English Schooling and Vocational *Freeman, C. Technical Change and Preparation,” in National Institute Full Employment. Oxford, England: **Mehaut, P. The Challenge of Compe- Economic Review, February 1987. Basil Blackwell, 1987. tence and Further Training Policies. **Prais, S.J. and K. Wagner. “School- *Gorbachev, Mikhail. Perestroika: New Nancy, France: G.R.E.E., University ing Standards in England and Ger- Thinking for Our Country and the of Nancy, 1994. many” in National Institute Eco- World. New York, NY: Harper and *Michalski, A. and H. Wallace. The nomic Review, May 1985. Row, 1988. European Community: The Chal- lenge of Enlargement. London, *Prokopenko, J. and I. Pavlin. Entrepre- *Grossman, G. and E. Helpman. Inno- England: Royal Institute of Interna- neurship Development in Public En- vation and Growth in the Global tional Affairs, 1992. terprises. Geneva, Switzerland: Inter- Economy. Cambridge, MA: MIT national Center for Public Enterprises Press, 1991. *Organization for Economic Coopera- in Developing Countries, International tion and Development: The Labour *Grubb, D. and W. Wells. “Employ- Labour Organization, 1991. Market in Poland. Centre for Coop- ment Regulation and Patterns of eration with the Economies in Tran- **Rose, J. and F. Pottier. Training and Work in EC Countries,” in OECD sition. Paris, France: OCED, 1993. the Labour Market: The Use of Data Economic Studies, 21, 1993. in Decision-making. Berlin, Germany: *Hashimoto, M. The Japanese Labor *______: Education and the European Centre for the Development Market in a Comparative Perspec- Economy in a Changing Society. of Vocational Training, 1993. tive with the U.S. Kalamazoo, MI: Paris, France, 1989. W.E. Upjohn Institute for Employ- ment Research, 1990.

Evaluation Forum n Issue 11 n Summer 1995 147 Resources n Information about Other Countries

*Rose, R. “What’s New in Youth Un- *U.S. Office of Technology Assess- *National Commission for Employ- employment?” in University of ment: U.S.-Mexico Trade: Pulling ment Policy: The Employment Ef- Strathclyde’s Studies in Public Poli- Together or Pulling Apart? OTA fects of European Economic Inte- cy, 177, 1989. Report Brief. Washington, D.C., gration: Background Study. 1992. Washington, D.C., 1993. *Scott, B. and G. Lodge (Editors) U.S. Competitiveness in the World Econ- *______: Competing Economies: *Organization for Economic Coopera- omy. Boston, MA: Harvard Business America, Europe, and the Pacific Rim. tion and Development: The OECD School Press, 1985. OTA Brief. Washington, D.C., 1991. Jobs Study: Facts and Analysis **Schoppa, L. Education Reform in Ja- *Villeval, M. “Labour Market Restruc- Strategies. Paris, France: OECD, pan. London, England: Routledge, turing and Deprivation Processes,” 1994. 1990. in Labour and Society, 16, 1991. *Reis, F., F. Rychener. Training and the Labour Market: The Use of Data *Sengenberger, W. , G. Loveman and **Williams, S. Youth Without Work: in Decision-Making. Nancy, France: M. Piore (Editors) The Re-Emer- Three Countries’ Approach to the University of Nancy, Research gence of Small Enterprises: Indus- Problem. Paris, France: OECD, 1981. Group on Education and Employ- trial Restructuring in Industrialised *Wood, S. The Transformation of ment, 1993. Countries. International Institute for Work? Skill, Flexibility and the La- Labour Studies. Geneva, Switzer- bour Process. London, England: Additional Articles and Book land, 1994. Unwin Hyman, 1989. Chapters of Potential Interest **Senker, P. International Competi- Elmeskov, J. and K. Pichelmann. “In- Working Papers and tion, Technical Change and Train- terpreting Unemployment: The Role Research Reports ing. Papers in Science, Technology of Labour Force Participation,” in and Public Policy. University of *Abraham, K. and S. Houseman. Earn- OECD Economic Studies. Paris, Sussex, England, 1988. ings Inequality in Germany. A staff France: OECD, 1993. working paper. Kalamazoo, MI: *Steenbergen, J. Change and Adjust- Elmeskov, J. and M. MacFarlan. “Un- W.E. Upjohn Institute for Employ- ment: External Relations and Indus- employment Persistence,” in OECD ment Research, 1994. trial Policy of the European Com- Economic Studies. Paris, France: munity. Boston, MA: Kluwer Law **Bennet, R. et al. Training and Enter- OECD, 1993. and Taxation Publishers, 1983. prise Councils (TECs) and Voca- Grubb, D. and W. Wells. “Employment tional Education and Training: *Steiner, J. European Democracies. Regulation and Patterns of Work in Conference Papers. London, White Plains, NY: Longman Pub- EC Countries,” in OECD Economic England: London School of Eco- lishers, 1994. Studies. Paris, France: OECD, 1993. nomics, July 1989. *Storey, D. and S. Johnson. Small and *Houseman, S. and K. Abraham. Labor Houseman, S. “Job Security v.Labor Medium-Sized Enterprises and Em- Market Flexibility: Is There a ployment Creation in the EEC Adjustment Under Different Institu- tional Structures: A Case Study of Tradeoff?,” in Employment Re- Countries. Brussels, Belgium: Euro- search. Journal of the W.E. Upjohn pean Commission, 1987. Germany and the United States. A staff working paper. Kalamazoo, Institute. Volume 1, No. 1, Spring **The Training Agency: Training and MI: W.E. Upjohn Institute for Em- 1994. Enterprise Councils: A Prospectus ployment Research, 1994. Sengenberger, W. The Role of Labour for the 1990s. London, England: Standards in Industrial Restructur- **Leigh, D. Retraining Displaced Training Agency, 1989. ing: Participation, Protection and Workers: What Can Developing Promotion. Discussion Paper No. *Turner, L. Democracy At Work: Countries Learn from OECD Na- 19. International Institute for Labour Changing World Markets and the tions? Washington, D.C.: Popula- Studies, 1990. Future of Labor Unions. Ithaca, NY: tion and Human Resources Depart- Cornell University Press, 1991. ment, The World Bank, 1992. Villeval, M. “Labour Market Restruc- turing and Deprivation Processes,” *Ulman, L, B. Eichengreen, and W. **Mehaut, P. The Challenge of Compe- in Labour and Society. Journal of Dickens (Editors) Labor and an In- tence and Further Training Policies. the International Labour Organisa- tegrated Europe. Washington, DC: Nancy, France: University of Nancy, tion, 1991. The Brookings Institution, 1993. Research Group on Education and Employment, 1994.

148 Evaluation Forum n Issue 11 n Summer 1995 Resources n State-Level Workforce Coordination, Integration and Accountability

State-Level Brizius, J. and M. Campbell. Getting Ganzglass, E. (Editor). Excellence at Results: A Guide for Government Work. Washington, D.C.: National Workforce Accountabiliity. Washington, D.C.: Governors’ Association, 1992. Council of Governors’ Policy Advi- Coordination, Gardner, S. Roles for General Purpose sors, 1993. Integration and Governments in Service Integration. Carter, R. The Accountable Agency. Rockville, MD: Project SHARE, Accountability Beverly Hills, CA: Sage, 1983. 1976. Agranoff, R. “Human Services Admin- Condelli, L. and M. Kutner. Quality In- Henton, D. The Feasibility of Services istration: Service Delivery, Service dicators for Adult Education Pro- Integration: An Evaluation Pre- Integration and Training” in Human grams: Lessons Learned from Other pared for the HEW Interagency Ser- Services Integration edited by T.J. Programs. Report to the Office of vices Integration Project. R&D Task Mikulecky. Washington, D.C.: Vocational and Adult Education, Force, 1975. American Society for Public Admin- U.S. Department of Education. Institute for Educational Leadership: istration, 1974. Washington, D.C.: Pelavin Associ- An Overview of Skill Standards in ates, Inc., 1992. American Evaluation Association: Education and Industry. Washing- Guiding Principles for Evaluators. Council of Governors’ Policy Advi- ton, D.C., 1993. Report of a Task Force on Evalua- sors: Getting Results (by J. Brizius Hallman, H.H. Pilot Comprehensive tion Standards, 1994. and M. Campbell). Washington, Manpower Program: Implications D.C., 1991. American Society for Quality Control: of Its Experience for Local and State Quality Systems — Model for Quali- ______Experiments in Systems Manpower Organizations. Washing- ty Assurance in Design/Develop- Change (by J. Chynoweth, et al.). ton, D.C.: Center for Government ment, Production, Installation and Final report to The Ford Foundation Studies, 1973. Servicing, and Guidelines for Using and United Way of America. Wash- Hansen, J.S. Preparing for the Work- Quality Assurance Principles in Ed- ington, D.C., 1992. place: Charting a Course for Post- ucation and Training. Milwaukee, Council of State Governments: Human secondary Training Policy. Policy WI, 1991. Services Integration: State Func- paper. Committee on Postsecondary Baj, J., R. Sheets, and C. Trott. “Im- tions in Implementation. Washing- Education and Training for the proving the Performance and Coor- ton, D.C., 1974. Workplace, National Research dination of Workforce Preparation Council. Washington, D.C.: Nation- Crawford, D., A. Johnson and A. Sum- Programs for Youth Through the In- al Academy Press, 1994. mers. Schools and Labor Market tegration of Federal-State Perfor- Outcomes. Philadelphia, PA: Na- Hoachlander, G. and M. Rahn. Perfor- mance Standards Systems,” in In- tional Center on the Educational mance Measures and Standards for vesting in Youth. Washington, D.C.: Quality of the Workforce, Universi- Vocational Education: 1991 Survey National Governors’ Association, ty of Pennsylvania, 1993. Results. Berkeley, CA: National 1992. Center for Research in Vocational DeWitt, J. Managing the Human Ser- Barnow, B. “The Effects of Perfor- Education (NCRVE), 1992. vice “System:” What Have We mance Standards on State and Local Learned from Services Integration? Ink, D. “The Origins and Thrusts of the Programs,” in Evaluating Welfare Project SHARE, Human Services New Federalism,” in The Adminis- and Training Programs edited by C. Monograph, Series 4, 1977. tration of the New Federalism: Ob- Manski and I. Garfinkel. Cambridge, jectives and Issues edited by L.H. MA: Harvard University Press, Friedlander, D. Subgroup Impacts and Grosoenick. Washington, D.C.: 1992. Performance Indicators for Selected Welfare Employment Programs. American Society for Public Admin- ______“Thirty Years of Changing New York, NY: MDRC, 1988. istration, 1973. Federal, State and Local Relation- Interstate Conference of Employment ships in Employment and Training Gans, S.P. and G.T. Horton. Integra- Security Agencies: Building An Ef- Programs,” in Publius, 23, Summer tion of Human Services. New York, fective Workforce Development Sys- 1993. NY: Praeger Publishers, 1975. tem. A discussion paper. Washing- ton, D.C., April 1993.

Evaluation Forum n Issue 11 n Summer 1995 149 Resources n State-Level Workforce Coordination, Integration and Accountability

Jarosik, D. and L. Phelps. Empowering National Association of Counties: Hu- Ruttenberg, S. H. and J. Gutchess. Accountability for Vocational-Tech- man Services Integration at the Manpower Challenge of the 1970s: nical Education. Berkeley, CA: Community Level: A Six County Re- Institutions and Social Change. Bal- NCRVE, 1992. port. Washington, D.C.: Research timore, MD: Johns Hopkins Press, Foundation, 1974. 1970. Jennings, E.T. and N.S. Zank. Welfare System Reform: Coordinating Fed- National Association of State Job Sanders, J. and the Joint Committee On eral, State and Local Public Assis- Training Coordinating Council Standards For Educational Evalua- tance Programs. Westport, CT: Chairs: Bring Down the Barriers: tion: The Program Evaluation Stan- Greenwood Press, 1993. Policy Recommendations. Washing- dards. Thousand Oaks, CA: Sage Kahn, A. J. Social Policy and Social ton, D.C., 1993. Publications, 1994. Services. New York, NY: Random National Commission for Employment Sheets, R. Building a National Skill House, 1973. Policy: Developing Effective JTPA Standards and Certification System Performance Standards Incentive in the U.S.: The Role of Existing King, C. Cross-Cutting Performance Policies. Washington, D.C., March Public and Private Occupational Management Issues in Human Re- 1990. Regulation Systems. Paper for Na- source Programs. Research report. tional Center on Education and the Washington, D.C.: National Com- National Governors Association: De- Economy. Washington, D.C., 1993. mission for Employment Policy, signing Program Evaluations (by L. 1988. Bilheimer). Center for Policy Re- Smith, G. and D. Stevens. Beyond Ac- search, Washington, D.C., 1989. countability. Denver, CO: Commu- Levitan, S.A. and J.K. Zickler. The nity College of Denver, 1994. Quest for a Federal Manpower ______An Action Agenda to Re- Partnership. Cambridge, MA: Har- design State Government. Reports Spencer, L. M. “Planning and Organiz- vard University Press, 1974. of the State Management Task Force ing Human Services Delivery Sys- Lien, C. A. Design for a Multi-Service Strategy Groups. 1993. tems” in Proceedings of a Seminar Delivery System. A report prepared ______Building State Workforce on Human Services Integration. for the Washington State Depart- Development Systems Based on Pol- Denver, CO: Social Welfare Re- ment of Social and Health Services icy Coordination and Quality Assur- search Institute, 1973. by Stanford Research Institute, ance (by J. Baj, R. Sheets, and C. Stevens, D. and J. Shi. Beyond Perfor- 1973. Trott). 1994. mance Standards. Berkeley, CA: National Center for Research in Vo- Linn, L.E. “Organizing Human Servic- ______Performance Management cational Education, 1994. es in Florida,” in Evaluation, Num- Project: Glossary of Project Terms. ber 3, 1976. Resource Papers series. July, 1994. Sundquist, J.L. Making Federalism Work: A Study of Program Coordi- Lucas, W. et al. The 1975 Census of ______Performance Management nation at the Community Level. Local Services Integration Projects. Project: Federal Core Data Ele- Washington, D.C.: The Brookings A paper prepared for the U.S. De- ments and Common Definitions. Re- Institution, 1969. partment of Health, Education and source Papers series. (Draft report) Welfare, 1975. July 1994. U.S. Congress, Office of Technology Lucas, W.A. Aggregating Organiza- ______Performance Management Assessment: Report Brief for No- tional Experience with Services Inte- Project: Texas’ Process For Setting vember 1993, reviewing OTA’s gration: Feasibility and Design. Core Performance Measures. Re- “Making Government Work: Elec- Santa Monica, CA: Rand Corpora- source Papers series. July 1994. tronic Delivery of Federal Services.” tion, 1975. Washington, D.C. National Institute for Literacy: A Guide ______Wage Record Information Mangum, G. L. The Emergence of to Strategic Planning For Perfor- Systems. Background paper, May Manpower Policy. New York, NY: mance Measurement, Reporting And 1994. Holt, Rinehart, and Winston, 1969. Improvement Systems. Washington, D.C., February 1994. Miller, R. et al. Developing Client Out- comes Monitoring Systems. Wash- ington, D.C.: The Urban Institute, 1985.

150 Evaluation Forum n Issue 11 n Summer 1995 Resources n State-Level Workforce Coordination, Integration and Accountability

U.S. Department of Education, Nation- U.S. Department of Labor, Office of Great Lakes States al Center on the Educational Quality Inspector General: Services Integra- Sheets, R. and C. Trott. Creating The of the Workforce: Crosswalk of Na- tion: A Twenty-Year Retrospective. Guarantee: A Great Lakes Work- tional Data Sets Focusing on Work- Washington, D.C., January 1991. force Development Strategy. Center er Training and Statistical Compan- ______Employment and Training for Governmental Studies, Northern ions to the Crosswalk of National Administration: Coordination of Illinois University, 1993. Data Sets Focusing on Worker Housing and Job Training Services: Training. Washington, D.C., June Council of Great Lakes Governors: A Review of Best Practices in 12 and April 1992. New Directions in Great Lakes Cities. Washington, D.C., 1992. Workforce Development Policy. Au- ______Creating Responsible and ______Employment and Training gust 1993. Responsive Accountability Systems. Administration: Summary of State Report of the Office of Educational Reports on Coordination Between Iowa Research and Information. Washing- Vocational Education and JTPA. ton, D.C.: U.S. Government Printing Washington, D.C., 1993. Iowa Target Alliance: Integrated Em- Office, 1988. ployment and Training Management Zank, N. Coordinating Federal Assis- U.S. Department of Health and Human Information System. November tance Programs for the Economical- Services: A Review of the Conceptu- 1992. ly Disadvantaged. Washington, al Foundations and Current Status D.C.: National Commission for Em- of Services Integration. Office of Indiana ployment Policy, 1991. Human Development Services, Indiana Workforce Legislation, 1993. 1983. Zornitsky, J., H. Bloom et al. Establish- ing a Postprogram Performance Indiana Department of Workforce De- U.S. Department of Labor: An Assess- Management System for Employ- velopment: Workforce Development ment of the JTPA Role in State and ment and Training Programs. Cam- Summary, 1993. Local Coordination Activities. bridge, MA: Abt Associates, Inc., Washington, D.C., 1991. 1985. Maine U.S. General Accounting Office: Wel- Maine Human Resource Development fare Simplification: Projects to Co- References for the Coordination Council: Second Annual Maine Hu- ordinate Services for Low-Income Efforts of Individual States man Resource Development Plan. Families, 1986. Connecticut July 1989. ______Integrating Human Ser- Connecticut Employment and Training ______Report on the Status of the vices: Linking At-Risk Families With Commission: Connecticut Employ- Governor’s Second Human Re- Services More Successful Than Sys- ment and Training Commission: source Development Plan, July-De- tem Reform Efforts. September People, Employment and Training, cember 1989. March 1990. 1992. 1992. U.S. Department of Health, Education ______Human Resources Invest- Massachusetts and Welfare: Integration of Human ment Goals for the 21st Century. Services in HEW: An Evaluation of National Commission for Employment July 1993. Services Integration Projects. Social Policy: Assessing State-Level Job and Rehabilitation Services, 1972. Training Coordination: A Survey Florida Design and Methodology Based U.S. Department of Housing and Urban Florida State Job Training Coordinat- Upon the Massachusetts Experience. Development: Coordinating Federal Washington, D.C., 1992. Assistance in the Community: Use of ing Council: Forum on Enhancing Selected Mechanisms for Planning Florida’s Economy Through Job National Governors’ Association: and Coordinating Federal Pro- Training Workforce 2000 Confer- “Building the Capacity of Regional grams. Community Development ence. A White Paper, 1993. Employment Boards in Massachu- Evaluation Series No. 8. Washing- setts” (by S. Spinner) in Labor ton, D.C.: Government Printing Of- Notes, 80, January 1993. fice, 1972.

Evaluation Forum n Issue 11 n Summer 1995 151 Resources n State-Level Workforce Coordination, Integration and Accountability

Michigan Oregon Issue Paper on Accountability. January, Michigan Job Training Coordinating Oregon Workforce Quality Council: 1994. Council: Creating A Human Invest- Shared Information System. May Patterns Emerging from the Research ment System, 1989. 1992. on Dislocated Worker Programs. Michigan Human Investment Fund and ______Information System: Sum- Spring 1994. Michigan Job Training Coordinating mary (by T. Lynch and J. Westine), Information from Research on Dislo- Council: Michigan Opportunity Sys- 1992. Oregon Department of Edu- cated Workers: Some Beginning tem: Report from Mackinac Island. cation: Workforce Development in Policy Issues. Spring 1994. October 1989. Oregon 1988-1994. “State Efforts to Assure Accountability Michigan Human Investment Fund: in Comprehensive Workforce Sys- Michigan Human Investment Sys- Texas tems: With Insights from Other tem. Working paper. March 1989. National Governors’ Association: Countries.” Spring, 1994. “Quality Workforce Planning in New Jersey Texas: A Market-Driven Initiative” Washington Training and New Jersey State Employment and (by J. Boyd) in Labor Notes, 80, Education Coordinating Board: Major Publications Training Commission: Annual Re- January 1993. Workforce Development Directory: port, 1992, 1993. References for Washington State Workforce Preparation Programs ______A Unified State Plan for Coordination Efforts and Services in Washington State. New Jersey’s Workforce Readiness March 1993. System, 1992. Washington Training and Education Coordinating Board: High Skills, High Wages: Washington’s Working Papers New York Comprehensive Plan for Workforce Some Beginning Ideas About Establish- Training and Education. September New York State Job Training Partner- ing Accountability: Oversight Issues 1994. ship Council: Creating a Vision: and Policies. Winter, 1994. The Workforce Preparation System Some Beginning Thoughts About What Office of the Governor of the of the Future. February 1990. Helps Define Accountability. Win- State of Washington: Office of ______New York State’s Invest- ter, 1994. Financial Management ment in a Quality Workforce: A Cat- Thinking About Accountability: Making Evaluation of Training Programs. alog of Workforce Preparation and Working paper. Forecasting Divi- Related Programs, 1993. Assessments of Activities and Pro- grams. Winter, 1994. sion, 1993. ______Workforce Preparation Thinking About Accountability: Devel- Evaluation Act. February 1990. oping Policy Questions to Anticipate Washington State Superintendent of Public Accountability Issues and Needs. North Carolina Instruction: Assessment and Winter 1994. School Information Division Governor’s Commission on Workforce Steps on the Way to Establishing Pro- Preparedness, 1992. Proposed Educational Outcomes and gram Accountability. Winter 1994. Measurement System, 1993.

152 Evaluation Forum n Issue 11 n Summer 1995 Resources n Federal Research: An Update

Federal Research: An Update

U.S. Department of Labor Employment and Training Administration: Office of Policy and Research Reports Completed in 1994

Evaluation of the EDWAA Job Creation The School-to-Work/Youth Apprenticeship A Guide to Well-Developed Services for Demonstration. Demonstration: Preliminary Findings. Dislocated Workers. Evaluation of the Defense Conversion Ad- From the Farm to the Job Market: A Guide Employment and Training for America’s justment Demonstration: Interim Report to Employment and Training Services Homeless: Report on the Job Training on Implementation. for Farmers and Ranchers. for the Homeless Demonstration Pro- gram. Service to Dislocated Farmers: An Evalua- Training and Employment Report of the tion of the EDWAA Farmers and Ranch- Secretary of Labor. Evaluation of the JTPA Title IV Migrant ers Demonstration. and Seasonal Farmworker Program.

Bureau of Labor Statistics: Office of Economic Research Selected Reports Completed,1994

Retirement in a Family Context: A Structur- Transitions from School-to-Work: A Survey al Model for Husbands and Wives. of Research Using the National Longitu- dinal Surveys.

Selected Projects in Progress

An examination of School-to-Work Transi- Education, Experience and Employer Incentive Pay, Earnings and Information: tion in the NLSY. Learning about Worker Productivity. Evidence from the NLSY. Wages, Fringe Benefits and Savings: Inter- High School Employment: Consumption of Technological Change and On-the-Job actions and Implications for the Determina- Investment. Training of Young Workers: The Intergen- tion of Labor Market Outcomes Analysis erational Consequences of Family Structure Labor Market Activity and Earnings: A with the NLS. on Wages and Educational Attainment. Comparison of NLSY and SIPP.

Bureau of International Labor Affairs: Foreign Economic Research Division Selected Recent Reports

The Labor Market Impacts of NAFTA. Employment Dimensions of Economic Re- Changes in the Industrial Structure of Job structuring: A Review of Related Labor Displacement: Evidence from the Dis- U.S. Employment Effects of a North American Policies and Programs in Industrialized placed Worker Surveys. Free Trade Agreement: A Survey of Issues Countries. and Estimated Employment Effects. Labor Market Adjustment and Job Dis- Evaluations of Programs in Industrialized placement. Returns to Classroom Training of Dis- Countries to Assist Workers Displaced placed Workers. The Economic Consequences of Jobless by Structural Change. Duration for Displaced Workers. Is Job Stability Declining in the U.S. Econ- Structural Change and Labor Market Ad- omy? Guidelines for Assisting Trade Displaced justment: A U.S.-Japan Comparison. Workers. Analysis of the Dislocated Workers’ Educa- Competition and Complementarity between tional Training Program. Designing Trade Adjustment Policies U.S. Imports from Development and U.S. Labor Market Adjustment Programs. Newly Industrializing Countries. Does Training Work for Displaced Work- ers? Trade-Sensitive U.S. Industries: Employ- The Wage Structure by Occupation: Skill ment Trends and Worker Characteris- Level and Skill. Labor Adjustment Policies and Practices in tics. Europe. The Impact of Technology, Trade and Out- The Quality of U.S. Jobs. sourcing on Employment and Labor Human Resources and Corporate Policy. Composition. Labor-Management Conflict and Coopera- tion.

Evaluation Forum n Issue 11 n Summer 1995 153 Resources n Federal Research: An Update

U.S. Department of Health and Human Services Selected Recent Reports

JOBS Evaluation: Early Lessons from Sev- Saturation Work Initiative Model in San Di- Summary of Final Evaluation Findings en Sites. ego: A Five-Year Follow-Up Study. from FY 1988-89 Demonstration Part- nership Program Projects. Welfare Reform in 1993: State JOBS Pro- Working Toward Jobs: A Series of State grams and Waivers. Case Studies. Responsibility, Work and Pride: The Values Underclass Behaviors in the U.S.: Measure- of Welfare Reform. ment and Analysis of Determinants. Families on Welfare: Focus on Teenage Assessing JOBS Participants: Issues and Mothers Could Enhance Welfare Re- Trade-Offs. form.

Selected Work/Welfare Waiver Projects Approved in 1994/1995

Arkansas Welfare Demonstration Indiana Indiana Manpower Place- Oregon JOBS Plus Program Project ment and Comprehensive Pennsylvania Pathways to Independence Training Program California Work Pays Demonstration Project Project Michigan To Strengthen Michigan South Carolina Self-Sufficiency and Paren- Families Colorado Colorado Personal Respon- tal Responsibility Program sibility and Employment Mississippi A New Direction Demon- South Dakota Strengthening of South Da- Program stration Program kota Families Initiative Connecticut A Fair Chance New York Jobs First Demonstration Wisconsin AFDC Benefit Cap Dem- Florida Family Transition Program North Dakota Early Intervention Program onstration Project Hawaii Creating Work Opportuni- Oklahoma Learnfare Program ties for JOBS Families

U.S. Department of Commerce Selected Current Research Projects

A project to develop a skill-based labor strategies that provide economic and job A project to determine the feasibility of de- market information system reflecting the creation benefits and meet sustainability ob- veloping a plan to utilize private industry occupational restructuring underway and jectives. councils to attract high-growth high-wage changing job requirements. industries to distressed areas. A project to stimulate technological innova- A project to develop information on initiat- tion in small businesses. ing and executing economic development

National Commission for Employment Policy Selected Recent Projects

An investigation of labor market trends be- An analysis of the Panel Survey of Income An exploration of employment prospects tween 1979 and 1994. Dynamics on the labor market status of Af- for low-income youth and young adults, in rican-American workers. A study of racial inequality in employment relation to lessons from the 1980s. and earnings among young men between A study of the employment effects of the A review of employment and training pro- 1979 and 1989. Administration’s programs re: Technology grams for youth and young adults. for a Sustainable Future.

National Commission list continues.

154 Evaluation Forum n Issue 11 n Summer 1995 Resources n Federal Research: An Update

National Commission list continued Selected Recent Projects

An analysis of JTPA Title II training pro- A series of studies on the relationship be- A review and commentary on evaluation re- grams for in-school and out-of-school tween UI and dislocated worker programs ports produced by the U.S. DOL on con- youth, including the effect of local labor in terms of measuring performance in temporary employment and training pro- market characteristics on the JTPA post- JTPA, JOBS, and Secondary Vocational grams for an array of target groups. program outcomes of out-of-school male Education. An assessment of the ability of three deliv- youth. An evaluation of the impact of Title III on ery systems to meet workforce development A study of successful and promising earnings and employment prospects, using needs: education, government job training school-based dropout prevention programs UI data. and placement programs; and employer- and strategies. sponsored education and training efforts. A study of the impact of alternative eligibil- A series of work/welfare studies to explore ity requirements for training stipends in the A number of studies concerning earnings the implications of time-limited welfare re- Reemployment Act of 1994. inequality: history, causes, strategies to ad- form initiatives on employment and training dress. A study of the kinds of training workers un- systems; employers’ roles in welfare re- dertake. A study of the changing role of labor force form; the role of JTPA; patterns of partici- statistics for economic policies in the pation in programs; and other issues. An historical review of comprehensive 1990s. planning and programming in employment and training in the U.S.

U.S. Congress Office of Technology Assessment Selected Recent Reports

Testing and Assessment in Vocational Edu- Performance Standards for the Food Stamp Technology and Work-Based Learning. cation. Employment and Training Program. Wage Record Information Systems. Technology, Jobs, and Productivity in a American Industry and the Environment: Multinationals and the U.S. Technology Service Economy. Implications for Trade and U.S. Com- Base. petitiveness.

Evaluation Forum n Issue 11 n Summer 1995 155 In Memoriam

At the end of this issue of Evalua- work of community learning centers He made extremely important contri- tion Forum we honor the memory of to skill our youth, or a few thousand butions to policy understanding in a Sar A. Levitan, who died in May 1994. public service jobs to move people broad range of areas. He believed in Sar was a unique and remarkably influ- from welfare to work. But since these evaluations, learning by experience ential contributor to the national dia- are not an offer, I am honored to ac- and experimentation. I think one of logue on employment, training and an- cept this award. If right and rationali- the important lessons that he tried to tipoverty policy in the U.S. over his ty continue to be recognized, these teach everyone was never to assume long career as an activist economist. critical education, employment and that what you are doing will work au- Author of fifty books and hundreds of earnings policies may eventually tomatically, because if everything articles on employment issues, Sar was come to pass, to realize this nation’s you tried to do worked it probably the recipient of the Secretary of La- promise of greatness. was not worth doing. . . . He had a bor’s Lifetime Achievement Award, strong belief in democratic institu- A memorial service was held for Sar presented to him by Secretary Robert tions, in the ability of informed opin- in June 1994 in the Education and La- ion to get things done. . . . He be- B. Reich. bor Committee Hearing Room of the lieved that there was a role for The text of this coveted award read: U.S. House of Representatives. Those competition in politics as in other “In recognition of your numerous con- commenting about his life, career and things, and that the federal govern- tributions to the Department of Labor achievements were Ray Marshall ment could in fact do a great deal to over several decades, you have set the (former Secretary of Labor), Gordon improve the lives of people. And the highest standards in your dedication to Berlin (formerly with The Ford Foun- reason he thought that is because it is work, the American worker, and the dation and now with MDRC), Marion true. fight against poverty. Your scholarship, Pines (The Johns Hopkins University), wisdom, wit and kindness have in- Jon Weintraub (House Committee on Sar actively supported the develop- spired us all.” Education and Labor), Isaac Shapiro ment of Evaluation Forum as an educa- (Office of the Secretary, U.S. Depart- tional tool for informing professionals In accepting the award, Sar replied, ment of Labor), Cliff Johnson (The in the field about the results of major My scribblings over the years have Children’s Defense Fund), Andrew policy analyses and research projects in aimed to combine right and rationali- Sum (Northeastern University), and employment and training, and willingly ty to improve social welfare. This Garth Mangum (University of Utah). offered many useful suggestions that improved its quality and utility and award, from those who share this The following excerpts are from Ray helped direct its further evolution. goal, is gratifying recognition of Marshall’s comments at the memorial these labors. I would gladly trade it service: for an indexed increase in the mini- mum wage to reduce hardship, a net-

156 Evaluation Forum n Issue 11 n Summer 1995 Material contained in this publica- Evaluation Forum is a national jour- Editor and Reviewer tion is in the public domain and may nal for program professionals in re- Ann Bonar Blalock be reproduced, fully or partiallly, lated human service fields: basic Admiralty Inlet Consulting without permission of the Federal education, vocational education, Hansville, Washington Government. Source credit is re- employment and training, welfare, Guest Reviewer quired only to reproduce any copy- labor market and labor force Ernst W. Stromsdorfer righted material contained herein. change, and economic develop- Department of Economics ment. The purpose of the journal is Washington State University This material will be made available to stimulate interest in policy analy- Pullman, Washington to sensory impaired individuals sis, policy research and program Graphics and Technical Services upon request. evaluation as useful tools for Kathleen Shankman Voice phone: 202-219-5782 policymaking, planning and the im- Center for Governmental Studies TDD* phone: 1-800-328-2577 provement of social programs. It is Northern Illinois University understood that the ideas and opin- DeKalb, Illinois *Telecommunications Device for the ions expressed in the content of the Deaf. journal are those of the respective writers and do not necessarily rep- resent those of the funder or those of the contributing public interest organizations. Sponsor and Funder Office of Planning and Research Employment and Training Administration U.S. Department of Labor Contributors to Journal Distribution National Governors’ Association Washington, D.C. National Association of Counties Washington, D.C. National Alliance of Business Washington, D.C. History of the Journal The New Evaluation Forum Please Note Evaluation Forum was devel- The new Evaluation Forum, of Each issue addresses a general oped as part of a five-year national which this is the second issue, is theme in its Features section, but demonstration project, The JTPA being sponsored and funded by the also covers a range of subjects un- Evaluation Design Project, funded Office of Planning and Research, der the section Evaluation Issues between 1986 and 1991 by the Na- Employment and Training Adminis- and Activities. Therefore, readers tional Commission for Employment tration, U.S. Department of Labor. are encouraged to send materials Policy, the U.S. Department of La- Its purpose is to disseminate useful to the editor on a wide variety of is- bor, the IBM Corporation and The information on evaluation issues, sues for possible review in the jour- Ford Foundation. The purpose of activities and results to those re- nal. the project was to increase the in- sponsible for administering, plan- Please send materials and sugges- terest, knowledge and sophistica- ning, managing and overseeing tions to: tion of employment and training employment and training programs professionals at the state and local across the United States. Again, Ann Blalock Editor and Reviewer level regarding program evaluation: each issue of the journal will be Admiralty Inlet Consulting its scientific principles and meth- theme-oriented, and will follow the PO Box 409 ods, its practical applications in bet- same organizational format as in 40554 Skunk Bay Road ter understanding program imple- the past. However, each issue will Hansville, WA 98340 mentation and impact, and its utility feature reviews by the editor and 360-638-2159 as a practical tool for adjusting poli- guest reviewers rather than articles The editor would appreciate your cies and improving programs. The written by practitioners and re- providing the following information project’s series of evaluation guides searchers. with your materials: name, address, can be obtained through the ERIC phone, organizational affiliation, po- system in public libraries, or in book Reader Participation sition in the organization, and major form from the W. E. Upjohn Institute The content of the new version interests re: evaluation issues and/ for Employment Research. of the journal will cover the same kinds of materials as in the past: or activities. Evaluation Forum is a national commentaries, policy analyses; The Features section of the next journal growing out of the five-year program reviews; reports on evalu- issue of Evaluation Forum will focus project and developed specifically ation planning, reports on analyses on issues surrounding youth em- for state and local program practi- of monitoring data; research reports ployment and training. tioners in the employment and on program evaluations or policy training field. Its major objective is research; books and articles on to communicate information on policy issues, programs and re- policy issues and research activi- search; and other materials related ties at the national, state and local to evaluation that would be of inter- level which can inform judgments est to practitioners. about the design and effectiveness of various employment and training policies and strategies. Eight theme-oriented issues of the journal were published as part of the dem- onstration project.