(OTIS). Final Report

DOCOSENT RESUME

ED 098 969 IR 001 352

TITLE Evaluation of the Occupational Training Information System (OTIS). Final Report. INSTITUTION Macro Systems, Inc., Silver Spring, Md. SPONS AGENCY Economic Development Administration (DOC), Washington, D.C. PUB DATE May 74 NOTE 224p.

EDRS PRICE MF-$0.75 HC-$10.20 PLUS POSTAGE DESCRIPTORS *Information Systems; *Job Market; Labor Force; Labor Market; Manpower Development; *Manpower Needs; Manpower Utilization; Program Evaluation IDENTIFIERS Kentucky; *Occupational Training Information System; Oklahoma; OTIS

ABSTRACT The Occupational Training Information System (OTIS) , which is in operation in Oklahoma and in adevelopmental stage in Kentucky, was evaluated. The principle objective of OTISis to provide the information necessary to formulate educational, manpower,, and economic development plans and policies. OTISconsists primarily of a manpower demand subsystem, a manpower supplysubsystem, and an -interface subsystem. These areas were evaluated for their effectiveness in the two states. A resurvey was made todetermine the accuracy of OTIS in predicting manpowerdemand. In Kentucky the forecasts for manpower demand and supply are inaccurate andthe cost' of the personal interview census is excessive;however, it is recommenderthat system development be continued if these problems are corrected. The favorablefactors in Kentucky are the widespread support OTIS has and the type ofinformation it provides. (RU) jiZ 3 5- ffr6

BLST Cr.;1-1AVAILABLE

EVALUATION OF

THE OCCUPATIONAL -TRAININGINFORMATION- SYSTEM ON

TWIN TOWERS. 1110 FIDLER LANE SILVER SPRING. MARYLAND 20910 I® TELEPHONE (301) 588-6484 MACRO SYSTEMS, iNC. Wostolostoo Naw York

1147114f,1 AJO3 12:11 Crik

Cr` GQ

C) FINAL REPORT C=3 EVALUATION OF THE OCCUPATIONAL 1.4.1 TRAINING INFORMATION SYSTEM (OTIS)

Prepared For: Economic Development Administration U. S. Department of Commerce Frederick 13. Dent, Secretary. William W. Blunt, Assistant Secretary For Economic Development

Prepared By: Macro Systems, Inc. Suite 300 1110 Fidler Lane Silver Spring, Maryland 20910

May, 1974

u S DEPARTMENT OF HEAL TN EDUCATION AWELFARE NATIONALINSTITUTE OF EDUCATION THISDO-ImENT HAS 8Fe AtRE Pk(., Du( ERr"%eTtyAsarcrivrDxlaGnA 'HsAFRII0)41 OR ORGANtrATiON t. 1 It/r, IT POINTS OFv,eW OR OPINIONS %TATED DO NOTNEGESSAall Y REPRE `.ENT OFFICIAL NATIONAL INSTITUTE OF EDUCATION RC'S'ci..N Oa POLICY This study was accomplished by professional consultants under contract with the Economic Development Adminis- tration. The statements, findings, conclusions, recommendations, and other data in this report are solely those of the contractors and do not necessarily reflect the views of the Economic Development Administration.

4 TABLE OF CONTENTS

Page Number

EXECUTIVE SUMMARY

I. INTRODUCTION 1

II. BACKGROUND 3

III. CURRENT OPERATIONS AND FINDINGS 16

Manpower Demand Subsystem 17

Manpower Supply Subsystem 46

Interface, Student Follow-Up, Cost, And Educational 54 Resources Subsystems

Overall System And System Impact 59

IV. RECOMMENDATIONS 68

Appendices

A. CONCEPTUAL DIAGRAM OF MANAGEMENT INFORMATION SYSTEM FOR VOCATIONAL EDUCATION

B. SOURCES AND DERIVATION OF OKLAHOMA OTIS COST ESTIMATES

C. SOURCES AND DERIVATION OF KENTUCKY OTIS COST ESTIMATES

SAMPLE OKLAHOMA AND KENTUCKY QUESTIONNAIRES TABLE ,OFCONTENTS(Continued)

E. RESURVEY METHODOLOGY F. RESURVEY DATA ANALYSIS

G. DISTRIBUTION BY REGION FOR EACH STATISTIC

H. RESURVEY SUMMARY DATA

I. OTIS SUPPLY ADJUSTMENT PROCEDURES J. BLS EMPLOYMENT DATA COLLECTION AND FORECASTINGPROGRAMS INDEX OF EXHIBITS

Following , Pau

I OVERVIEW OF EARLY OTIS MODEL 5

II. ORIGINAL OTIS CONCEPTUAL MODEL 5

III. SAMPLE PAGE CYCLE FIVE REPORT 6

IV. OTIS KENTUCKY CONCEPTUAL MODEL 11

V. CURRENT STATUS OF OTIS IN OKLAHOMA AND 14 KENTUCKY

VI, OTIS DEMAND DATA COLLECTIONPROCEDURES 18

VII. OKLAHOMA AND KENTUCKY DEMAND SURVEY 18 ATTRIBUTES

VIII. DISTRIBUTION OF ERROR STATISTICS INOKLAHOMA 25

IX. SUMMARY OF OKLAHOMA FORECASTSBY TYPE 27 AND EMPLOYMENT SIZE

X. SUMMARY OF OKLAHOMA FORECASTSBY INDUSTRY 29

XL OKLAHOMA OCCUPATIONAL FORECASTSBY REGION 30

XII. FORECAST ACCURACY BY CLUSTER INSELECTED 31 REGIONS IN OKLAHOMA

XIII. ALL REMAINING EMPLOYMENTFOR OKLAHOMA 32

XIV. DISTRIBUTION OF ERROR STATISTICSIN KENTUCKY 33

XV. SUMMARY OF KENTUCKY FORECASTSBY TYPE AND 36 EMPLOYMENT SIZE INDEX OF EXHIBITS(Continued)

Following Page

XVI. SUMMARY OF KENTUCKY Fo CASTS BY 37 MANUFACTURING INDUSTR

XVII. KENTUCKY OCCUPATIONAL FORECASTSBY REGION 38

XVIII. EFFECT OF CLUSTERING IN I.ENTUCKY 39

XIX. SUMMARY OF EMPLOYER FORECASTRESPONSES: 42 KENTUCKY

XX. DEMAND DATA SUBMISSION DATES BYREGION: 43 KENTUCKY

XXI. OTIS STUDENT ACCOUNTING SYSTEMDATA COLLECTION 48 PROCEDURES

XXII. COMPOSITION AND SOURCE OF OTIS OKLAHOMA 51 SUPPLY DATA

XXIII. COMPARISON OF LMI/OTIS INFORMATIONNEEDS 59

XXIV. OKLAHOMA VOC-ED PLANNING PROCESS 61

XXV. OTIS DEVELOPMENT AND OPERATINGCOSTS IN 62 OKLAHOMA AND KENTUCKY

XXVI. OTIS KENTUCKY DEMAND DATACOLLECTION COST 66 ESTIMATES

XXVII. OTIS REPLICATION COST ESTIMATES 66 iv

BEST CO liVi;11.41511

EXECUTIVE SUMMARY

The Occupational Training Information System (OTIS) first.wa's developed in Oklahoma during 1968 to provide continuous, timely informktion for statewide vocational education planning and economic development. Subsequently, development of a more comprehensive version of the system waa undertaken in Kentucky with Economic Development Administration (EDA) and statefunds. MACRO SYSTEMS. INC. (MSI), was employed by EDA to evaluate this developmental effort.Since the Kentucky system was still under development, it also was decided to examine ongoing OTIS operations inOklahoma. The purpose of this evaluation was to identify the strengths and weaknesses of OTIS and to recommend improvements where necessary. These findings andrecommenda- tions are summarized below.

I.FINDINGS Although OTIS is comprised of several subsystems, the nucleusof the entire system is the interface of projected manpower demand withestimated

I. manpower supply.Because of the criticality of the demand/ supply interface, the evaluation focused on the demand, supply, and interfacesubsystems and on the overall system impact.

1. DEMAND SUBSYSTEM Employer forecasts provide the basis for manpower demandprojections in both Oklahoma and. Kentucky. However, the proceduresutilized by each state to collect these data differ Inarkedly. Oklahoma collects baseline data from a sample of all state employers every four years. Only the largest employers are contacted personally; theremaining firms receive

o KSI CL;17,"

questionnaires in the mail and are contacted personally only if an adequate response for a particular cell is not received. The baseline data are updated annually to include new firms and significant employment changes for listed firms. %, Kentucky collects data through a personal interview census. During the initial year, only manufacturfng employers were contacted. \ MSI conducted a resurvey of a sample of theemployerparticipating in the current OTIS surveys. The resurvey was conducted at thend of the forecast period in both Oklahoma and Kentucky so that forecastem\loyment could be compared with current employment.

1 (1) Oklahoma Employer Forecasts 1 The forecast of manpower demand made by employers participating in the Oklahoma resurvey was underestimated by 1,342 positions.This understatement represents an error of 4 percent when related to total actual employment and 96 percent when related to the actual change in employment. An analysis of the forecasts made by individual firms indicated that: Over 60 percent of the employer forecasts did not increase the accuracy which would have been obtained with a simple no-change extrapolation. When compared to actual employment, nearly 30 perdent of the firms made forecast errors of over 20 percent; the average error was over 26 percent. Over half of the employers had a forecast error equal to or greater than the actual change in employment. The average error was nearly 25 percent greater than thechange in employment. Additional analysis indicated that the type of forecast and employment size did not affect accuracy substantially. I (2) Oklahoma Regional Forecast The previous analyses were concerned with theability of an employer to forecast manpower demand. However, the data produced andutilized by OTIS are regional summaries of net manpower requirementsby occu-

.. pation and/or occupational cluster. Thus, the accuracy of theregional forecasts of manpower demand also was assessed. The results of these analyses indicated that: When compared to actual employment, the regional errors ranged from nearly 2 percent to over 17 percent. Theaver-' age error for all regions was approximately6 percent. When compared to change in employment, the regional errors ranged from over 8 percent to 67 percent. The averageerror for all regions was slightly over 50 percent. Combining the occvmational data into vocational education clusters did not in.. move accurac3). substantially. >. The forecast for "All Remaining Employees" did not affect the accuracy of the totalforecast adversely.

(3) Kentucky Employer Forecast Th' forecast of manpower demand made by employersparticipating in the Kentucky resurvey was overestimated by 1,100 positions.This overstatement represents an error of 2 percent when compared toactual employment and 98 percent when related to actual change inemployment. An analysis of the individual forecasts indicated that: Over 75 percent of the employer forecasts did not increase the accuracy which would have been obtained with asimple no-changeextrapolation.

When compared to actual employment, over29 percent of the employers made torecast errors of Aver 20 percent; the average error was over 30 percent. REST

Over 50 percent of the firms made a forecast error equal to or greater than the actual change in employment. The average error was nearly 90 percent greater than the change in employment. Additional analyses indicated that forecast accuracy was not affected by type of forecast or employment size.

(4) Kentucky Regional Forecast An analysis of the regional aggregations of OTIS data indicated that When compared to actual employment, the regional forecast errors ranged from less than 0.2 percent to a high of over 11 percent. The average error for all regions was 4. 7 percent. When compared to actual change in employment, the forecast error ranged from a low of 2. 4 percent to a high of over 300 percent. The average error for all regions was over 100 percent. In addition, combining the occupational data into clusters did not significantly improve forecast accuracy.

2. SUPPLY SUBSYSTEM In Oklahoma, supply totals are comprised primarily of graduates of formal training programs and registrants with the Oklahoma Employment Service Commission (OESC). Kentucky plans to,obtain supply data from similar sources. With the exception of company-trained individuals, persons receiving formal vocational training appear to be adequately covered. However, individuals grad- .uating from nonvocational schools or changing jobs are not included in supply. Quantifying the extent of these omissions is extremely difficult.However, the results of numerous studies indicate that the number of people available for jobs from nonvocational schools and other occupations is considerable. Ijndoubtedly, some of the persons omitted currently from OTIS supply data are included in the ()ESC registrants, which compriseapproximately 50 percent of manpower supply.However, despite the relatively high Employment Service

-iv- liLS1'Lug; penetration raies in Oklahoma, it is likely that many of these individuals are not included in the OESC totals.Thus, the published OTIS supply figures are understated.

3. INTERFACE SUBSYSTEM The interface statistic is the principal output produced by OTIS. This statistic is computed by comparing estimated manpower supply for each occupation within a selected program cluster with projected manpower demand for the same occupations. As noted earlier, some of the OTIS forecasts of manpower demand are inaccurate and additional sourcesof manpower supply are required. However, even if manpower demand data were accurate and manpower supply data complete, the interface statistic mustbe used with caution for the following principal reasons: Manpower requirements data are aggregated statewide and by region.However, commutation patterns can affect the results. For example, an excess of supply for a particular occupation statewide or in a particular region may be offset by workers commuting to another state or region, e.g., nearly 36, 000 more Kentucky residents commute to Ohio than Ohioresidents commute to Kentucky. Net migration may alter the net manpower requirement projections over the course of the forecast period by decreasing or increasing manpower supply. Many trained individuals qualify for jobs outside the scope of their training, e.g., many workers have multiple skills and demonstrate a high degree of occupational mobility.Thus, supply for a particular occupation can change with personnel shifts.

4. SYSTEM IMPACT AND REPLICABILITY The principal objective, of OTIS is to provide the information necessary to formulate educational, manpower, and economicdevelopment plans and policies. Our analysis indicates that:

-v- BEST COP) ti,.;: OTIS output influences directly the Oklahoma state vocational education plan. Kentucky is incorporating OTIS data into the planning process. The annual operating costs of OTIS Oklahoma are approximately$30, 000, including approximately $19, 000 for the demand subsystem.In Kentucky, OTIS annual operating costs are estimated to be $185, 000, including over $110,000 for the demand subsystem. The relatively low annual operating cost inOklahoma reflects the fact that benchmark demand data are -,:llected through a sample of employers every four years.In addition, Kentucky utilizes personal interviews exclusively and Oklahoma contacts large employers personally andsmall employers by mail and telephone.In result: Oklahoma obtained data from employers at a field data collection cost of approximately $6 per employer. Kentucky obtained data from employers at a field data collection cost of approximately $28 per employer. Thus, the total field cost of collecting demand data from a censusof Kentucky yr, employees is $1.2 million.Conversely, a 10 percent sample similar to that used in Oklahoma would cost from $29, 000 to $120,000, depending on the extent of personal interviews. Therefore, the additional cost ofconducting a personal interview census could be more than $1 million.Clearly, the cost of replicating the Kentucky OTIS demand data collection methodology is excessive,when compared to the cost of conducting a sample. II. RECOMMENDATIONS In general, OTIS Kentucky has widespread supportand is providing the type of information required for manpower andvocational education planning. However, as noted above, some of the forecasts of manpowerdemand are inaccurate, manpower supply isunderstated, and the cost of a personal-interview census is excessive.Consequently, we recommend that system development be continued, if the following four improvements are incorporatedto alleviate these deficiencies.

-vi- ri1;1111.6311 BESTCOI 1. UTILIZE THE OCCUPATIONAL EMPLOYMENTSTATISTICS SURVEY TO COLLECT CURRE:.r EMPLOYMENTDATA The Occupational Employment Statistics(OES) survey program now being implemented in Kentucky should be utilized tocollect current employment data. Although the OES program in Kentucky is notfully operational, in our judgment, this survey can meet the informationneeds of OTIS users, if the following refinements are made. Develop jointly a sampling procedure thatwill permit compilation of.reliable, local-level data. The currentemploy- ment data previously collected by OTISshould be useful in developing an appropriate sample. Conduct the entire OES survey annuallyfor three years, rather than surveying manufacturing,nonmanufacturing, and trade employers in three-year cycles. These survey data should be utilizedimmediately to develop regional industry-occupation matrices for Kentucky.Subsequently, these matrices can be utilized in the projection of manpowerdemand by occupation.

2. UTILIZE THE BUREAU OF LABORSTATISTICS (BLS) METHODOLOGY TO FORECAST MANPOWERDEMAND The National /State Industry-OccupationMatrix System developed by BLS should be utilized to forecast employment.During the first year, the BLS census-based interim industry-occupationmatrix for Kentucky can be utilized. Subsequently, the state regional matrices canbe utilized with the National/ State Matrix System to project manpowerdemand by occupation.In addition, the Employment Service shoulddetermine*appropriate forecast periods in consultation with major data users; e.g., 3-,5-, and 10-year projections with annual updates.Subsequently, the accuracy of theseforecasts of manpower demand should be evaluated. BEST CONAV;,1,1:31.i

3. IMPROVE THE ESTIMATES OF MANPOWER SUPPLY The OTIS estimates of manpower supply should be improved by: Adjusting the manpower supply figures to include informally training individuals, e.g., individuals from nonvocational schools or individuals who have become proficient through training or experience on the job. Identifying the extent of geographic mobility. Net migration in Kentucky is not significant; however, intra- and interstate commutation do involve a substantial proportion of the work force. These improvements require additional research on occupational and geographic mobility. Additional follow-up studies are necessary to obtain this information.In' our judgment, these follow-up studies should be conducted two to five years after graduation and should include graduates from non-occupational training programs. The results of these studies will provide the data with which to assess the long- term impact of vocational training and to determine the occupational and geographic mobility of graduates from all types of schools in Kentucky.

4. EMPHASIZE THE QUALITATIVE ASPECTS OF VOCATIONAL EDUCATION TO INCREASE RAPPORT WITH EMPLOYERS One of the important intangible benefits of OTIS is the rapport obtained with employers.In our judgment, the level of rapport can be increased by emphasizing the qualitative aspects of vocational education, rather than data collection. For example, under current procedures, rapport is sought by asking the employer for current and forecast employment by occupation and emphasizing the benefits of OTIS. This is done within the tight time constraints necessary to meet data collection requirements. Consequently, scheduling and conducting interviews are difficult for both vocational education personnel and employers. Using the OESIBLS demand data collection and forecasting methodology, the industrial coordinators will receive the OTIS forecasts for their region. Sub; sequently, as part of their normal functions, the coordinators can discuss these IL1E occupational projections and related information with employers. More importantly, they can take the time to discuss the qu.ditative aspects of vocational education, e.g., the characteristics of a typical joband/or worker. Job characteristics include duties, working conditions, hours, wages, shifts, fringe benefits, and opportunitiesfor advance- ment Employee characteristics include skills, attitudes, physical requirements, educational and training requirements, and work experience In addition, current vacancies for placement purposes, potentialtechnological changes, such as the introduction of new machines or products, and the employer's recent experience with vocational education graduates should be discussed. In our judgment, bringing a forecast to the employer, obtaining hisideas on its validity, and discussing how vocational education can meet his particularneeds will result in a higher level of rapport than exists currently. I. INTRODUCTION

The Occupational Training Information System (OTIS) was first developed in Oklahoma in 1968 to provide continuous, timely information for statewide vocational education planning and manpower development. Subsequently, the Bureau of Vocational Education of the Kentucky State Department of Education and the Economic Development Administration (EDA) of the U. S. Department of Commerce became interested in the system. Bureau personnel believed that the system would satisfy needs for new and more specific information to support planning efforts. EDA was interested because the additional information provided by OTIS and resultant planning improvements could (1) maximize the utilization of EDA-funded educational facilities and (2) provide the skilled craftsmen necessary to sustain economic growth. Consequently, development of OTIS in Kentucky was undertaken with state and EDA funds.1/ The original OTIS conceptual model was comprised of six interrelated subsystems.Subsequently, this model was considerably revised and a more comprehensive version formed the basis of the Kentucky model.In addition, the scope of the Kentucky model was broadened by the addition of several new subsystems to give greater emphasis to manpower planning. However, the original objectives of the system were not changed substantially. Currently, OTIS in Kentucky is under development, and only four subsystems have been implemented. Further, only one subsystem, the demand component, is totally operational, but data have been collected and processed for manufacturing industries, only. Bureau personnel in Kentucky want to

1/ In Kentucky, OTIS is known as the Kentucky Information for Training and Education System (KITES). proceed with this developmental effort and several other states are also interested in installing similar systems, particularly if federal funding is available. Consequently, EDA requested an evaluation of OTIS prior to committing additional funds for system expansion or replication. MACRO SYSTEMS, INC. (MK), was employed by EDA to conduct an evaluation of the OTIS developmental effort in Kentucky and to examine ongoing OTIS operations in Oklahoma. The purpose of this evaluation is to ascertain the strengths and weaknesses of the system and to recommend improvements.where necessary. Our evaluation focuses on OTIS Kentucky because the systemcchlk-N cept is more comprehensive and the development partially funded with EDA monies. The results of this study are presented in the following chapters. Chapter II describes the background, concept, and current status of OTIS in Oklahoma and Kentucky. Chapter III describes current OTIS operations and costs and presents our findings regarding the operational effectiveness of the various subsystems and the overall impact of OTIS, including system replicability, Chapter IV presents our recommendations regarding system improvements and/or replication. Relevant background material and detailed information supporting ouranalyses are included in the appendices. II. BACKGROUND

OTIS was first implementedin Oklahoma. Sinceimplementation, the system has been revisedsubstantially.Currently, a more comprehensive version of OTIS is un4ierdevelopment in Kentucky. Thischapter presents our understanding of the conceptualand operating modelsimplemented in each state. OKLAHOMA IN 1968 1. THE OTIS CONCEPTWAS DEVELOPED BY The ultimate objectiveof vocational education, asspecified in the 1968 amendments to the VocationalEducation Act of 1963, isemployment for program graduates.Equally important, trainedindividuals are necessary to sustain economic growth.Realization of these objectivesrequires accurate forecasts of manpowerdemand and supply.Accordingly, forecastrequirements were the focus of twomajor studies completedin 1967 by Ling-Temco-Vought, Inc.,1/ and Oklahoma State University.z1 The results of, theseoccupational studies indicated a needfor an operational data systemto provide continuous, timely information for statewidevocational education planningand manpower policy development inOklahoma. Subsequently, representativesfrom the Research Departmentof the Depart- Oklahoma IndustrialDevelopment and ParkDepartment, the Oklahoma Security ment of Vocationaland Technical Education,the Oklahoma Employment Commission (OESC), andthe Manpower Researchand Training Center at

Ling-Ternco-Vought, Inc. (Prime Contractor),Systems Management Services, Vocationaland Technical 1/ Commission, December, 1967. Skills and Literacy Systems. ReportII: Washingt-m, D. C.: Ozarks Regional in Oklahoma: 2/ Braden, Paul V. and Roney, Maurice W. ,Occupational Education Beyond the High School An analytical study withrecommended= for a statewide system for manpowerdevelopment. Stillwater, Oklahoma: The Research Foundation,Oklahoma State University, January, 1968.

-3 Oklahoma State University met to study the need for a data system to provide

1 information necessary to support the state's vocational training and economic development programs.In response: a 'proposal to develop a statewide manpower data system was prepared by the ManpowerResearch and Training Center. The results of the initial research emphasized the establishment of system parameters, including: Potential user population Information and user linkages Information input processes Report formats for data analysis and dissemination Information feedback and system modification procedures User involvement strategies to ensure system effectiveness Based upon the specification of overall user requirements and other pertinent system parameters, data were collected and the first anneal OTISCycle Report was published in January,l969.11 The first report was necessarily incomplete due to the short timeperiod between project approval and completion. However, this report did furnish the requisite operating experience to enable system development tobe continued. As a result, a second Cycle Report contained significantrefinements.Further, the success of the efforts over the two-year period led to the permanentdesig- nation of system operating responsibility. The State Departmentof Vocational and Technical Education was selected as the operating agency,based on the assumption that as the prime user of OTIS, this department wouldbe the logical focus of operations ti' assure maximum effort and interestduring implementation. Currently, this agency exercises full operatingresponsibility for OTIS within the context of agency planning and analysis functions. The majorshare of these

lf The OTIS Final Report gives credit for the system concept to Dr. Pat Choate (then Director of Research) and Mr. George Appley of the Industrial Development and Park Department, and for system development, to Dr. Paul V. Braden of the Manpower Research and Training Center who served as Project Director.Dr. Braden was assisted by researchers Dr. Krishan Paul and James L. Harris. Braden, Paul V. , et al, Occupational Training Information System (OTIS) Final Report, Research Foundation, Oklahoma State University, Still- water, Oklahoma, 1970. operating activities is performedby the Research, Planning,and Evaluation Division. However, manpowerdemand data are collected andedited by the OESC. 2. THE ORIGINAL OTISCONCEPTUAL MODEL CONSISTEDOF SIX INTERRELATED SUBSYSTEMS OTIS was developed over atwo-year period. Exhibit I,following this page, presents anearly version of the OTISmodel. The early model included only four subsystems:supply, demand, interface, andcost/benefit.Exhibit following Exhibit I, presents anoverview of the OTIS model as it wasdesigned ,for implementation inOklahoma. This exhibitdepicts the functional relationships among the sixmajor conceptual subsystemsdescribed in the OTIS Final Report. E The principal objectivesof the original OTISmodel were to provide, on local, regional(intrastate), and state bases, theplanning and operating information necessary to: Permit the formulationof manpower planningstrategies and assist in manpowerdecision-making Enhance the formulationof viable economicdevelopment policies and programs Establish needed vocationaland technical training programs Enhance program graduateemployment potential Facilitate development of programsfor specific groups such as thehandicapped Meet legislative andcommunity accountabilityrequirements 4.

Hence, this overview was developedby A conceptual diagram was notincluded in the OTIS Final Report. MSI, based on the descriptionof the system contained in the report.

-5- *"..17,1*

EXHIBIT I

COPYAV.101311 Economic Developmetit Administration BEST OVERVIEW OF EARLY OTIS MODEL

Analysis Demand Data _loorSupply Data of Supply and and Analysis and Analysis Demand Data (Interfacing) A

Recommendations to Decision- Makers

%.0klahoma Other Manpower Considerations Policy

Industrial Growth Planning

MS Source: Paul, Kristian K. "AnAssessment of The Oklahoma PrivateSch pl Occu tional Thinin Oklahoma State University, with Implications for StatewidePlanning." Unpublished Masters Thesis. May, 1970. EXHIBIT II

iggiCOMAVNLABLE Economic Development Administration ORIGINAL OTIS CONCEPTUAL MODEL

Manpower Manpower Interface Supply Donald

A A 1 Recommendations to Decision- Follow-,Up Makers

Cost

Underdeveloped Human Resources

Other Analyses Decisions(4 Change in Manpower

---

Implementation Program Economic By Schools Changes Development It was determined that theseobjectives could be met through thedevelop- ment of a model based onthe collection, analysis, and comparisonof manpower supply and demand data.In concept, the OTIS model functions asfollows: Manpower demand data by occupation(DOT code) are collected from employers. These datarepresent the estimated, number of jobs that will result fromthe expansion or reduction of manpower needs.Subsequently, these data are adjusted for replacement and summarized onregional and state levels. Manpower supply data are collected onplanning areas and statewide bases. Manpower supply isdefined as the total number of prospective employeesreceiving training in various vocationaleducational curriculum clusters orthose cer- tified by the employment service. Net additional manpewerrequirements are calculated by region for each cluster.The nucleus of the model is the comparison of the demand andsupply figures for each cluster. The net result, either ademand or supply surplus, is summarized in a report calledthe OTIS Cycle Report. Exhibit III, following this page,pregents a sample page from e the Oklahoma report. These net demand data areused with information fromother subsystes to f.cilitate the formulation ofvocational education policies.The entireanpower decision-making process takesplace within an environmentof socio-political involvement, i..,policies are made within thecontext of community- and agency-wide paripatiefn.Thus, obtaining broad-basedparticipation in both the formulation andimplementation of plans and policiesis fundamental to OTIS. The six major subsystemscomprising the initial OTIS model aredescribed

-below.

(1) The Manpower Supply SubsystemWas Designed To Furnish Labor Market Entrant Information The objective of the manpowersupply component of OTIS is to provide information concerningthe number of trainedindividuals entering the

40. -6- BEST COPY AVAILABLE

. a a a a a 6 a a X ).a a X) ,- I- Oft a 4J in Ift 0. 43 40 at x - X 16 fa C.' au ie N ft 0- I. at X - o e0 Kt 10 0. W 2 i a P. =P 0 00 w * ..0 a IP a 04 ii* V 42. E 1, * l 4 a 0C" ou ir- . vtIt ,tt .,, 43 * * a 1 C'' 4 *.$414 > 0 e VP ..e in -0 * Mn 101 43 -.. a. N )1 LX .2e. I a X ;* a ii 4 4 * an omme ... mow al. 0. 411. 4.11. a ti a 3 4... .r.: a 4.- r3 a a *. .1. a 0 t.4 A * V. * - .- 4 .X ..t a 1. e A S. us a -.. a-t" i CI A 1" '.Re 4 r L : SI [ la ... - - - -. -- 4. r a sm. .41. * * ii i SA. IS. * .5 2 * 4 0 e. at se ). 11. lb- ...... - . c * .1 * 4 8 r..... - ;s * * 4.:3 1 0 i" o I. O ...... m...em. .. * .. woo IP * >' * 41. fo. 1111. on. vo. 1 E C.0 J 4, .0 : e. A en 0 -, .4 7 1.., IL, . ... 1 4 %. 4 4. Ca 0 Is s I: .., s ).tr.. II O I,I. 1 0 O .1 1 AI 0,1 etdi e V 0 r 1 rs, t. 4 r v 0 r. 0 0 S.J 6 6. * * VS 4 ...... - ..-...... -- n it I VI dt- n V e^ et sa ft. a 4.. ... 4 .41 r. 11 * 1.61 I. tt . S S. 0 4 II s .....al. 5 gm. ... 0 WI. Opow. 0 eV e, O 4.

- 4 4 * M1 g A 4" AI 44 fi M ftti nis ** et ' t 0 *

0...... , ,.. -I 4, W 4.. 11 C. re e 4 tou ..< < ft :.- tif a le ... 44.? 1%. -. t.6. I. ' td 0 e. V C C - . I. e. ... s 0* & e. 1 fir t^ ti C. 4" .. sfr -. . I.' ow' - . 'ft, ea ft C am. 'v.IA c ....i 8r IL. C . C. r C a 5. ft:7 r a - - Crt N us rS -1 p- U. W Oa as Its .44 is- M a .. 0 4. (:/ 4, 0. 4- L'. If` 0- i .JN C. iv, d) ft fA 11.1 V a a4) labor market in any forecast year.In the initial model, mostof these individuals were graduatesfrom publicly funded vocationaland, technical education programs. Acorollary objective of the supplycomponent was to collect data necessaryfor a detailed evaluation ofthe vocational education curriculum. Thus,in addition to a numerical countof students enrolled in a variety of skill programs,the following data were collected; Student names and addressesfor follow-up Student characteristics data Economic status Career goals Ethnic and social background Physical or mental handicap These data, particularlystudent characteristicsinformation, were to be collected jointly fromstudents and teachers, andutilized in the preparation of a, summarymanagement report. The student accountingsystem was selected asthe primary mechanism for the collectionof the supply data.However, other sources of supply data have beenrecognized and are included inthe total supply data component. Theadditional sources of supply data andthe student accounting system are discussedin detail later. Furnish A (2) The Manpower DemandSubsystem Was Designed To Short-Term Forecast OfLabor Market Requirements The OTIS demand componentwas conceived toyield a short-term forecast of manpowerdemand, where demand equalsavailable new jobs plus estimated replacementpositions. This component processesshort- term employer estimatesofmanpower demand by occupation(DOT. code) and aggregates these data onstate and regional bases.In turn, these occupational demandforecasts are used to determinethe net demand for specific vocational educationtraining skills.

-7- The principal objectivesof the initial demandsubsystem were to: U.Zlize the benchmarkdemand data for nonfarm wageand salary employment collectedpreviously by the OESC Develop specific methodologiesfor collecting agricultural manpower demanddata and generatingacceptable agricultural forecasts Collect occupational manpowerdemand data to update the OESC benchmark data Develop a significant rapportbetween educators and representatives of industry The initial demanddata were collectedby contacting personallyall manufacturing employers inthe state.x/Most of the data collectors were occupational educators orindustrial coordinators.This technique was selected to foster maximumpersonal contact and rapportbetween educators and the privatesector to:(1) ensure employerparticipation; (2) assist in the developmentof the occupational forecast;and (3) enhance placement prospects forvocational/technicalgraduates.Y The demand data procedures currentlyutilized in both states aredescribed in Chap- ter III. Demand For Each (3) The Interface SubsystemComputes Net Manpower Cluster Of Training Programs The interfacesubsystem compares manpowersupply and demand. Forecast demand andsupply are listed for mostoccupations associated with selected training programclusters.Total manpower demandand total manpower supplyfigures for each cluster arecomputed by adding the figures for eachoccupation.Subsequently, the anticipatedsupply of

the initial system development but were notincluded in the Cycle Cne Report. ji These data were utilized in Founda- at Braden, Paul V. , et Occupational Traininia Information System(OTIS) nnal Report, Research tion, Oklahoma State University,Stillwater, Oklahoma, 1970. trained personnel is subtracted fromforecasted demand to obtain the net manpower requirement orsurplus for each training cluster.Exhibit III, following page 6, presents the resultsof this process. The net requirements calculation is the nucleusof the OTIS model and provides requisite information for vocational educationand manpower planning.

(4) The Initial Cost Data Subsystem WasDesigned To Collect Information Concerning Costs Per Student The cost data subsystem wasdesigned initially to collect the information necessary to conduct programcost /benefit analyses. The results of these analyses were intended toprovide the basis for resource allocation and a breakdown of the totalexpenditures for vocational education into specific program areas.The average cost per student is computed by dividing the costs for a specific programby the number of students enrolled in the program. Although specific objectives forgathering detailed analytical program cost data are not presentlydefined, it is apparent that such datacould provide significant input to decisionsregarding curriculum mix. For example, .cost information mightindicate that a proposed program,which would benefit only a small groupof students, requires inordinatelylarge amounts of capital andpersonnel. Thus, a low-priority rankingcould be accurately ascribed to the program.

(5) The Initial Graduate Follow-UpSubsystem Was Designed To Collect Postgraduate Job Placement Data ForProgram Evaluation The graduate follow-up subsystem wasconceived to collect information from program graduates.These data, in turn, were intended to assist curriculum plannersin the development of a data basefor evaluating program impact.Specifically, this subsystem was designedto produce three major evaluative reports.

-9- Mobility Report- -This report wasintended to present information concerning thevocational changes of program graduates over time.Based on the comparison of training and employment siteZip Codes, a series ofthese reports would describe the movementsof individual program graduates and discerniblepatterns for the group. Subsequent Employment OfGraduates Report--A major diffi- culty in assessing the impactof training involves determining the impact of specifictraining and skills upon the typeof employment obtained by a programgraduate. Accordingly, the objective of this report wasto correlate and presenttrain- ing and employmentdata. The data collected inthe follow-up subsystem were aimed at providing some,indication of training/ job linkages. Graduate Salary StudyReport--A major issue invocational education and manpowerplanning is the extent to which pre- employment skills trainingincreases the entrance wageand long-term earning potentialof an enrollee. Thus,training- related earnings increasesrepresent a significantevaluative indicator of programeffectiveness. Accordingly,the Graduate Salary Report wasdesigned to display thesalary levels of program graduatesand identify salarydifferentials among graduates of differenttraining programs. Initially, the graduatefollow-up subsystem wasdesigned to collect data on 20,000students annually throughdirect mailings. As many as three follow-up mailings perstudent were contemplated.At the same time, follow-upmethods utilizing instructorcontact with graduates were tested. After comparingboth methods, the teacherfollow-up method was selected as"faster and less costly." However,neither approach produced the anticipatedresults.1/Consequently, the graduatefollow-up subsystem has beenincorporated into the studentaccounting system.

ResoarchCoordinating Unit, Project Report #7, Oklahoma State Department of Vocational and Technical Education, Stillwater, Oklahoma, p. 11.

-10- (6) The Initial Underdeveleped Human ResourcesSubsystem Was Des fined To Identify Special Requirements OfSpecific Labor Market Entrants The purpose of underdeveloped human resources subsystem was to identify specific population groups in needof special or supplementary services and training experiences. This targetpopulation is defined generally as unemployed, underemployed,and/or disadvantaged, and it includes culturally deprived as well as physicallyand mentally handicapped individuals. Initially, attempts were made to collect generalpopulation profile data through a test survey in the Tulsa area.Approximately 285, 000 questionnaires were distributed randomly in the street.However, only 5,045 usable responses were obtained.Subsequently, the idea of identifying the underdeveloped human resources inthe general population was -dropped and an alternative approach ofcollecting data on training program enrollees through the student accounting system wasadopted.

3. THE OTIS CONCEPTUAL MODEL HASBEEN MODIFIED EXTENSIVELY FOR REPLICATION IN KENTUCKY OTIS has evolved continually sinceits development began in Oklahoma in 1968. A more extensive version of theoriginal OTIS model led directly to the formulation of the Kentucky OTIS model.The basic objectives of OTIS do not appear to have undergonesubstantial, change in the Kentucky model. However, several subsystems have been modifiedand the addition of new subsystems has broadened the scope of the Kentucky model toinclude greater emphasis on manpower planning, of whichvocational education is a significant part.Exhibit IV, following this page, displays the overallKentucky OTIS model. The four additional subsystems developed forKentucky are summarized in the following sections. Economic Development intuarr Administration FollowUp OTIS KENTUCKY CONCEPTUAL MODEL Manpower Supply Interface Manpower Demand Training Resources Task Analysis Training Costs Other Data Analyses Socioeconomic CharacteristicsUnderdeveloped Characteristics I waxier Training ProcessCharacteristics YL 1 Information Systems 1 Mao Delivery Needs AIM. gime 01 L-_- -I 1, 1 ,System Strategy Economic Imp'emenr-ation DecisionsTraining rI Prablems J li tvlanpowerPrograms for ,-rI OPP01' - -_--J tunities -- -I I Development Policy Coals and Objectives Development/I SOCIO-POUTICAL INVOLVEMENTS r Source: Braden, P. B., An Overview of The OTIS (OTIS, SoutheasternTraining Economic DevelopmentInforrozuc AdminiStratiOn (FDA) Initial Inc.Six Month, Mey 1972),Contract page And 5; AA Prelim' Region, Occupational Training 111 InformationTechnical System*, Plan Inc. For ;Frankfort,The Kentucky, April, :4 For ration In tation And Installation Of The Oc ,.ational r tateS 1972. n BEST COPY 117.11,1311

(1) The Goals And ObjectivesSubsystem Was Designed To Focus Attention On Key HumanResources Development Issues The objective of the goalsand objectives subsystem is toestablish procedures for formulating"realistic goals and objectives for agencies which provide manpower-relatedservices... "I/ This subsystem was designed to link conceptuallythe manpower supply and demandcomponents and public policy with theplanning and decision-making processes.This approach to data organizationhas the potential to provideinformation for considering planning strategy,goals, and objectives.However, the actual definition of this componentis not defined by available documents. Moreover, the focus of thegoals and objectives subsystemdescribed in the OTIS literature differsfrom that described by otherresearchers working on the Kentuckysystem.ifExhibit IV, following page 11, presents the original KentuckyOTIS conceptual model. Therevised model described by other researchersis included in Appendix A.

(2) The Educational ResourcesSubsystem Was Designed ToIdentify Human And Capital ResourcesFor Allocation By Planning Decision-Makers The educational resourcessubsystem represents an effort to develop capital and plantrequirements data and personnel resources data for inclusion in theplanning process.Shifts in educational policy, resulting from an examinationof manpower supply anddemand data and other labor market andeconomic condition indicators,have an impact upon theutilization of the total resourcesof the educational system. Thus, an inventory of these resourcesis necessary.This would enable j/ Braden, Paul V., Proposal For ResearchAnd/Or Related Activities; Adaptation And InstallationOf The Occupational Tr Infomation in OT In Selected St- Districts Within The Southeastern ERA Region, Occupational TrainingInformation Systems, Inc.: March 1972, p. 9. Findlay, D. C.and Braden, P. V. , e Comprehensive DitsstrstemFor Occupational Slucation; (The Center for Research andLeadership Development in Vocational Education, TheOhio State University, unpublished paper --N. D.) pp. 3-5.

-12- the planner to identify precisely the level of resources required to implement proposed educational changes.However, the development of this potentially important component has not progressed beyond the develop- , ment of an inventory file.

(3) A Training Process Subsystem Was Formulated To Monitor Critical Training Variables For Program Upgrading Exhibit IV, following page 11, shows training process characteristics as input information into the manpower supply component. Qualita- tive curriculum information would be useful to anticipate not only the types of skills being imparted to enrollees, but also how well these skills were were being taught. This component appears to call for the identification of specific training techniques or curriculum outputs in terms of their causal relationships with enrollee employment success. At a minimum, therefore, a variety of impact evaluation studies to test for significant curriculum variables would be required.It appears that some of this requisite employment experience data could be collected throughthe student follow-up subsystem. However, neither the operating linkages nor a generalized procedural approach have been formulated.

(4) The Deliver S stem Strateg Selection For Manpower Development Programs Subsystem Was Formulated To Establish Allocation Priorities The delivery System strategy selection for manpower development programs subsystem was conceived to collect requisite system inputsand utilize these data to allocate resources. The objective of this component is to organize all data collected during the OTIS process and present a set of feasible operating strategies for review within the framework of the socio-political environment. Within this environment, the exact relationship among the elements of this subsystem, the goals and objectives subsystem, and a variety of other data or information inputs is not clear. Elements such as needs, opportunities, and problems appear to constitute

-13- the cognitive input of plannersdeveloped from the informationcollected by the OTIS subsystems.These data contribute to theformulation of policy-level goals and objectivesand, thus, to the developmentof the delivery system strategy.

4. THE OTIS CONCEPTHAS NOT BEEN TOTALLYIMPLEMENTED Extensive interviews withpersonnel in Oklahoma andKentucky have indicated that neither OTISconceptual model has beenimplemented fully.Exhibit 1r, following this page,presents the current statusof development of the sub- / Systems comprisingthe original conceptualmodel in each state. Asindicated, the subsystemsimplemented in Oklahoma are: Manpower Demand Subsystem Manpower Supply Subsystem Interface Subsystem

In addition, the follow-upsubsystem has been implementedby and incorporated into the state's studentaccounting system; underdevelopecihuman resources data are collectedby the student accounting system;and an extremely limited version of the cost subsystem wasdeveloped as part of OTIS.Subsequently, the OTIS, cost subsystem wasdropped and a completely new cos:system developed and implemented as part of theOklahoma Department ofVocational Education Man- agement InformationSystem. On the other hand, inthe OTIS Kentucky model,only the demand subsystem of the 10 subsystemscomprising the model isimplemented. However, four other systems are underdevelopment. The manpower supplysubsystem is being developed bystate personnel. The interface subsystem isbeing developed and programmed in conjunction withEDA personnel. EXHIBIT V a Economic Development AdministrationCURRENTOKLAHOMA STATUS AND OF KENTUCKYcens IN 1 2 °US Old 4h.3 , - uirtilte 4 5 6 Conceptual Model ManpowerDemand Manpower Supply Interface follow-Up______1_, Cost Data developedResourcesUnder-Hurfnan Operating Model Implemented Implemented Implemented Inc arporatedAccotutingIn Student ReplacedMotherSystem With IncorporatedAccountingIn StudentSystem System Sub 1 3 OT IS Kentu Under- 6 . Resource 10 Conceptual Model - ManpowerDemand______SuManpower w, Interface Follow I. Cost Data developed Humanesources CbCoals ectives And Educational esources TrainingProc --, AllocationStrate:4 Operating Model Implemented Develop- Underment Develop- mentUnder Develop- Undermeat Not?Dented YetImple- Implemented Not Yet Implemented Not Yet File StartedInventory mentedNot Imple-Yet Implemented Not Yet The follow-up subsystem is being developed by the Center for Vocational and Technical Education at Ohio State Univer- sity. An inventory file of educational resources has been started.

This chapter has presented a discussion of the conceptual OTIS models and has described the current status of the systems actually implemented in Oklahoma and Kentucky. The next chapter describes current system operations and presents the findings resulting from ouranalysis of OTIS operations and utilization. III. CURRENT OPERATIONS AND FINDINGS

The purpose of this study is to evaluate OTIS Kentucky and to examine ongoing OTIS operations in Oklahoma. Therefore, questions concerning the accuracy, operational effectiveness, and overall impact of OTIS must be answered. The operational effectiveness of the system can be determined by responses to questions about the individual subsystems.Specific questions relevant to each subsystem are: Dcrnar21Sj._,LIbs stern- -A re the data collection procedures efficient? Are the data processed effectively? Are the forecasts of manpower demand complete and accurate? Do the data sources and collection procedurgs used by OTIS complement or duplicate Department of Labor (DORsources ane. procedures? Does OTIS data bank duplicate existing DOL data? Supply Subsystem--Are all sources of manpower included? Are data processed efficiently? Aye forecasts accurate and complete? Interface Subsystem- -Are the interface procedures efficient? Are net manpower requirements presented and utilized properly? Are interface tables timely and error free? Follow-Up Subsystem--Does training influence placement or career patterns? Does feedback data result in system changes? Underdeveloped Human Resources SubsystemAre underdeveloped human resources identified? Are supplemental training and assistance provided? Cost Subsystem--Are the cost data useful for resource afloat*. tion? oals And Subsystem- -Are realistic goals and objectives established and performance monitored? Educational Resources Subsystem --Is an inventory of plants and equipment compiled and used for resource allocation? Training Process stem--Are the outputs of other subsystems utilized to develop operating strategies? Questions concerning the overall system and system impact include: Does OTIS provide the information needed for vocational and technical education planning? Are system outputs utilized to adjust the program mix and/ or to maximize the utilization of educational resources? Does the reallocation of resources enhance the placement of vocational and technical graduates?

Does OTIS provide the information needed for manpower plan- _ ning and/or formulation of economic development strategies? Can OTIS be exported or replicated easily and inexpensively? Clearly, many of these questions are interrelated. More importantly, several of the OTIS subsystems are not operational so that some of the questions cannot be answered. Thedemand/supply interface is the nucleus of OTIS. Because of these limiting conditions and the criticality ofthe demand /supply f. interface, our evaluation focuses on the demand and supply subsystems and on the overall impact of the system. The results of our analysesof OTIS in Okla- home and Kentucky are presented subsequently. Findingsabout the subsystems are presented first,followed by our analysis of the total system and its impact. MANPOWER DEMAND SUBSYSTEM This section describes the demand data collection proceduresutilized in Oklahoma and Kentucky and presents our findings regarding the manpowerdemand. component. Specifically, the following aspects of the demandsubsystem are cussed:

-17- EV'

The similarities and differences between the demand data collection methodologies currently utilized in Oklahoma and Kentucky The accuracy of the employer forecasts of manpower demand The relative accuracy of forecast demand by type of forecast, size of firm, and industry The accuracy of the occupational forecasts by region The effect of clustering on the accuracy of the occupational forecast The problems encountered in collecting the demand data in Kentucky The comprehensiveness of the manpower demand data utilized by OT IS

1.` OKLAHOMA AND KENTUCKY UTILIZE DIFFERENT PROCEDURESTO COLLECT MANPOWER DEMAND DATA Employer forecasts provide the basis for projectingdemand in both Okla- homa and Kcniticky.However, the procedures utilized by each state to collect these data differ markedly. ed. Oklahoma collects data from a sample of state employers. Kentucky collects data from a census of employers of a particular type, e.g., all manufacturing employers. Exhibit VI, following this page, strates the manpower demand dataCollection process for each state.Further, Eihibit VII, following Exhibit VI, compares the attributes of the Oklahoma and Kentucky surveymethodologies. Iriboth states, the demand data collection activity dc;pends upon personal contacts between employers and trainedrepresentatives of the OESC or the Kentucky State Department of Vocational Educationand other state agencies. Personal contacts are emphasized to attempt to assurethat employers produce 6.4 JO:

AVALOVE laOil EXEMIIT VI Economic Development Adminlietution

OTIS DEMAND DATA COLLECTION PROCEDURES

OK tAlicALA r" VOT eat One:nets Recruit Gnash...is Train Students As With ES for ES SCRIM ewes Prevaratican Students Data Collectse Data Collection

Identify Is fish. a Companies Which ElateeVear Stave's., .Nre sum Develop Instates- . Have large >o Yct Sample employment Sueses Chanute Cluttiticmnaiee

NIP Are wry In Tor Select Sample Did -ompinl TV.o Quartsics' Companies C °triplets and A Return It, T Tee V 11!.. le Company In Top Antic !voted1 Two Quartiles? Mango' E.' Mail Second Personal Visit t Surrey Vet l No Data Collectle4- Questionnaire

No New Dam Personal Visit tot Did Company Required MCA COHVG.i.014 Complete Data Complete and Returned To ES Return It f a - Yes I No IComplete Data . Returned To IS Pomona! Follow-Up for Data Collection

kFY f S f wiptoymmt Service Vns- I .? 'One-ail:mai }ducat ion Demand 'Cornish& Data VolechVocafisstuil Technical Iducatien Data Are Sent To IS Edits, Inflates.14-- Returned To ES State Volech and Ptiduces Ft at na"

KENTUCKY lanctdState Board of

Emptovms, 1 or Survey

Quettiotatairta Are All Data Ponta! Visits To Completed and Complete and Rettnned To Al' rtnrioycni Correct' Tops flatfish Sem Volech iNOta Called at Data Coded On ELVA Edits NC/ OCR form Magnetic Tape To FDA Fee Data Tape Procesing

Are Data Cosset? /teetotal Tape Returned 1 manna To Kentucky No 1-- fira Svcs:Whore For Ccevectien Conceseted KEY : ES Employment Service FDA Economic Development Admaitetretion r VoeidVocational Education Volech'0(.1 ion.I Technic.? thicatinn Are Data Complete Report Cent to Dela Collectors j Demand Data Selected KCIStUCicy Of Repast Pitutud Cycle Report

Taps Held For Selene: of Data EXHD3IT Vii

ECOZiOntic Development Administration BESTCOPYMAMIE OKLAHOMA AND KENTUCKY DEMAND SURVEY ATTRIBUTES

A - bite Oklaborn-

/. Survey Period (Baseline) Memorial Day, 1971 - Labor Day, June, 1972--No end date established 1971

2. Data Submission Cut-Off September 15, 1971 No cut-off date established; data Date outstanding on Febmcry 28, 1973 t 3. Forecast Target Date June, 1971 May, 1972

4. Target Employers All business and government Manufacturing =eluding the self-employed

S. Employer Identification Employment Security records Directory of Manufacturers, pub- Technique maintained by OESC lisped by Kentucky Department of Commerce, and Employment Ser- vice data

6. Stratification Technique Separate by major industry divi- Stratification not required; census for Sample eon; arrange business by region survey and employment size; determine quartile separation; select sample

7. Sample Ratio Selection By Quartile:

.First 100 percent 100 percent .Second 100 percent 100 percent .Third 20 percent 100 percent .Fourth S percent 100 percent

8. Survey Data Collection Special schedules prepared by Schedules prepared by BLS for the hstruents OESC Occupational Employment Statistics (OES) ?rogram modified by Kentucky

9. Data Collection Personnel Graduate students hired by OESC Personnel fromvariousstate agencies

10. Collection Personnel TrainingOne week, conducted by OESC 1 days, conducted by OTIS, Inc.

11. Survey Response Results 4 .Number contacted 2,636 2,307 .Usable responses 1,865 2,175 Response ratio 71 percent 94 percent Ettablishrnent employment84 percent eb 12. Data modification OESC personnel review all data for Mixing and/or incorrect data are accuracy and modify where reces- returned to Industrial Coordinator saw reliable employment demandforecasts and to establish rapport amongthe public and private sectororganizations involved. And (1) Oklahoma Contacts SampleEmployers By Personal Visits l As noted earlier, duringthe early stages of OTISdevelopment, Oklahoma endeavored to collectdemand data from all employers on a census basis.Vocational educatior n,..rsonnel wereutilized to collect these data.This method was fount; be very expensive and was not utilized. Currently, demand data arecollected through a sample of state employers conducted byOESC. The largest employersin terms of total employment are contactedpersonally. The remainingfirms receive survey questionnairesin the mail and arecontacted personally only if an adequate response for aparticular cell is not received. In Oklahoma, baselinedata are collected everyfour years and are updated annually to include newfirms and significant employmentchanges for listed firms. Asnoted in Exhibit VII, following page18, the 1971 sample survey was stratified asfollows: OESC records were utilized toidentify all businesses that paid a wage to four or moreworkers in each of 20 weeks. The state was divided into11 state planning 'districts. The employers in eachindustry division in each district were ranked in descendingorder by total employment. The employers in each area weredivided into quartiles, based on totalemployment per firm. The sample was selected asfollows: 100 percent of the first twoquartiles 20 percent of the thirdquartile 5 percent of the fourth quartile The selected sample wasadjusted to include noncoveredfirms and major employers nototherwise included in the sample stratification. Overall, the Oklahoma sampleincluded 416 manufacturing and2,217 nonmanufacturing establishmentsemploying approximately 375, OOupeople. The response rate for thesample was approximately 71 percent, or 1,865 reporting units. The most recent demanddata in Oklahoma were collectedby graduate students utilizing surveyquestionnaires developed byOESC. These questionnaires included morethan 280 occupationsclassified by the Dic- tionary of OccupationalTitles (DOT) code. A samplequestionnaire is included in Appendix D.Employers were asked to forecastthe demand for each occupation for two-and four-year periods.

(2) Kentucky Conducts A PersonalInterview Census The initial Kentucky demanddata collection activitiesfocused on conducting a census of themanufacturing employers listed inthe Kentucky Directory of Manufacturers,published by the State Departmentof Com- merce. Thus,Kentucky data collection does notinvolve a stratified sample such as was undertaken inOklahoma.Instead, all manufacturers were surveyed with the followingresults, as of September, 1973. 2,946 employers were in theuniverse. 2,855 employers were contacted. 374 were nonmanufacturers,who were thereby eliminated from the survey population. 275 firms were determined to beinactive, i. e. , out of business. 69 employers refused to respond. 61 employers never werecontacted. Thus, a total of 2,175 usable responses,exclusive of nonmanufacturing- and inactive firms, were received, giving anoverall usable response rate of approximately 94 percent.Employers responding employ approximately 214,000 of the 234, 500 manufacturingemployees. The demand data were collectedby over 500 individuals, including vocational education teachers and avariety of individuals from other agencies.Thirty-four different Bureau of LaborStatistics (BLS) Occupa- tional Employment Survey (OES)questionnaires, including some 1, 400 occupations, were utilized to collectthe data. A sample questionnaire is contained in Appendix D.Employers were asked to forecast manpower demand by occupation for one-and two-year periods.

THE FORECAST OF MANPOWERDEMAND MADE 13Y EMPLOYERS PARTICIPATING EN THE OKLAHOMARESURVEY WAS UNDERESTIMATED The nucleus of the entire OTISsystem is the interface ofprojected manpower demand withestimated manpower supply. Theforecasts of manpower demand are prepared by employers.Therefore, an assessment of the accuracy of these employer forecastsis crucial to an evaluation of the OTISdemand data collection methodology. An assessment of the overallreliability of the OTIS demand data requires an analysis ofall aspects of employment forecastingboth by firm and by regional summaries of occupational data.Relevant questions concerning theability of individual employers to forecastinclude: ,=7,-. a

Are specific, individualemployer forecasts accurate? Does the accuracy varyby type of forecast, e.g.,change or no change, orby size in terms of employment? Are the aggregate forecastsof employers in some industries better than in others? Are forecast errorsassociated with particularindependent variables, e.g., number ofoccupations? More general questionsconcerning the occupationalforecast include: Are the regional summariesof manpower demaud by occupation accurate? Do the forecasts of"All Remaining Employees"adversely affect the overall accuracyof the occupational projections? Does clusteting improvethe accuracy of the regionalsummaries? To answer questionsof this type, a comparisonof the OTIS forecast employ-,, ment figures withactual employment figures is necessary.Therefore, the Proj- ect Monitoring Committeedecided to resurvey 200firms that had participated in the current OTIS surveyin Oklahoma. The resurvey wasundertaken at the end d the two-year forecast period,thereby enabling forecastemployment data to be compared with current actualemployment. The resurveymethodology, including sample selection and surveyprocedures, is described inAppendix E. The results of this comparisonindicated that the forecast of manpowerdemand was underestimated by 1,342 positions.This understatement represents an error of 4 percent whencompared to actual employmentand 96 percent when related to actual change inemployment. During theforecast period, employment in if Oklahoma, excluding agriculture,increased approximately 9 percent.

II Unpublished BLS data Alternative statistics for evaluating the accuracyof the OTIS forecast were being developedconcurrently with data preparation. After extensive analysis, the Project Monitoring Committeedecided to use the following three statistical measures to evaluate the accuracyof the OTISdemand data. U Statistic- -This statistic wasdeveloped by Henri Theill and has been used previously toevaluate the accuracy of employer forecasts of manpowerdemand.V The U Statistic is calculated as follows: r(P. - A.)2 EA2

where P. and A. stand for predicted and actualchanges. Thus, it can be seenthat U= 0 only if all forecasts areperfect and that U = 1 when the prediction error isof the same magnitude as a no-change extrapolation.In other words, "by using the inequality coefficient one measures theseriousness of a prediction error by the quadratic loss criterionin such a way that the zero corresponds with perfectionand the unit with the loss associated with no-change extrapolation.It will be clear that the inequality coefficient has nofinite upper bound, which is tantamount to saying that it is possible todo considerably worse than by extrapolating on ano-change basis. However, as discussed subsequentlyand illustrated by the OTIS data, the U value of an incorrect"no-change" forecast is 1.0 regardless of the magnitude of the error. Statistic I- -This is a relatively simplestatistic that relates the forecast error to total actualemployment. The formula for deriving Statistic I is: Total Forecast - Total Actual Statistic I Total Actual The forecast error is computed using onlytotal employment for the employer or particular aggregation.Thus, errors in individual occupational forecasts,comprising the employers' total forecast, can be offset by otherline items or by the regional summaries.

1. Theil, Henri, Applied lonomic Forecasting, Rand McNally & Company,Chicago, 1966. Manpower Policies, an unpublished 3J Maier, Collette H. , An Evaluation Of Area SIdU Stuvelis As A Basis For doctoral dissertation at the University of Wisconsin, 1971.

-23- Statistic II--This statistic relates the forecast error to the actual change in employment over the forecast period and is computed as follows: Total Forecast - Total Actual Statistic IITotal Actual - Total Original Similar to Statistic I, this indicator allows canceling of internal errors and considers only the total employment of the employer or other aggregation.In the case of an incorrect no-change forecast, Statistic II is always 1.0 because the total forecast equals total original employment. A complete explanation, including the derivation, limits,special cases, and interpretations of each of these statistics, is presented in Appendix F. Our resurvey of employers in Oklahoma indicated that the forecast of total employment statewide was understated. The three statisticsdescribed above were utilized to evaluate the overall accuracy of employer forecasts. The results of these analyses indicated that: Forecast accuracy varied widely by firm. Type of forecast and employment size did not substantially affect accuracy. Forecast accuracy varied by industry. Accuracy of the occupation forecast varied by region. Clustering did not improve accuracy substantially. Forecast figures for "All Remaining Employees" did not adversely affect the accuracy of the total forecast. These analyses and resultant findings are presented in the succeeding sections.

(1) Forecast Accuracy rarieMms The accuracy of the manpower demand forecast made by the 193 employers in Oklahoma participating in the resurvey varied widely.

-24- ti Exhibit VIII, following this page, presents the distribution of employers for each error statistic.The complete distribution by region for each statistic is included in Appendix G. A sample printout for a firm is also included in this appendix. As shown in Exhibit VIII, the range of U varies from 0 to over 2.0000. The distribution of forecasts within this range is as follows: Some 15, or approximately 7.8 percent, of the forecasts were perfect. Given the characteristics of the U Statistic, this means that the individual forecasts for each of the occupations were perfect. A total of 17, or approximately 8.8 percent, of the forecasts had a U value between .0001 - .5999. Some 43, or about 22.2 percent, of the forecasts showed a U of from .6000 - .9999. A total of 72, or about 36.7 percent, of the forecasts had a U value of 1.0000. The large number of employers having a unit value is due to the high incidence of incorrect no-change forecasts. Some 46, or 24.5 percent, of the forecasts had a U value of 1.0001 or over. This analysis of the U Statistic values indicates that over 60 percent of the forecasts did not decrease the inaccuracy which would have occurred had a simple, no-change extrapolation beenused. More importantly, many of these forecasts were more inaccurate.Therefore, a more accurate forecast would have been available if a simple, no-change extrapolation had been utilized. As noted earlier, Statistic I relates the forecast error to total actual employment at the time of resurvey. This tends to narrow the range of the statistic because of the relatively large base. Even so. the Statistic I values range from over -. 500 to over+.500. The distribution of forecasts within this range is described subsequently.

-25- DISTRIBUTION OF ERROR STATISTICS IN OKLAHOMA 2 NumberEnwloyers of U Statistic Value EmployersNumber of Statistic I . Value NumberEmployers of Statistic B Value 15 2 0 1310 - Over ,500 8028 -.801- Over - 1.000 1.000 3 .2000.0001 - .3999.1999 69 -.201-.251 - .250.500 14 9 -.601 - ;800 1312 .40001.6000 - .5999 - .7999 8 -.101-.151 - .200.150 3 -.201-.401 -- .600 .4(X) 416, 7230 .8000 - .9999 1.0000 2318 -.001-.051 - .050.100 26 5 e -.001 - .200 0 1913 1.25011.0001 --1,2500 1.5000 2126 +,001 - .050 0 3 +.201+,001 -- .400 .200 23 1.50011.7501 - 1.7500- 2.0000 12 +.101+,051 - .150.100 21 +.601+.401 - .600.800 9 Over 2.0000 7 +.201+.151 - .200 .250 14 5 ++,801 Over -1.0(X) 1.000 1110 +.251+ Over - .500 .500 Some 26, or approximately 13.5 percent, of theforecasts were perfect. The majorityof these were no-change forecasts. Some 87, or approximately 45.1 percent,of the forecasts were low. The remaining 80 employer forecasts, or 41.4 percent, were high. Ignoring plus and minus signs, the forecast error,when compared to actual employment resulted in: Some 74, or 38. 3 percent, of the forecasts had aStatistic I value from .001 - .100. Some 36, or approximately 18.7 percent, of theforecasts had a Statistic I value of from.101 - .200. Another 36, or approximately 18.7 percent,of the forecasts had a Statistic I value between .201 - . 500. The remaining 21, or 10. 9 percent, of theforecasts had a Statistic I value of over . 500. Thus, nearly 30 percent of the firms madeforecast errors of over 20 percent when compared to total actual employment. Statistic II relates the error in the forecast to theactual change An total employment over the forecast period.Thus, the range of values for Statistic II is usually high when compared toStatistic I because of the much smallerbase. Obviously, the. same number of employers made a perfect forecast in the Statistic II as in Statistic I.Ignoring the signs, the range of forecast values is as follows. Some 14. or approximately 7.4 percent, of theforecasts had a Statistic II value between.001 - .400. Some 26, or 13. 5 percent, of the forecasts had aStatistic II value between .401 - .800.

-26- A total of 85, or 44.1 percent, of the forecasts had a value between .801 - 1.000.Similar to the U Statistic, the high incidence of unit values is due to the fact that the value for this statistic is one, if there is any error in a no-change forecast.Thus, an error of one employee or 400 employees results in a unit value. The remaining 42, or 21.8 percent, of the forecasts had a Statistic II value of over 1.000. Thus, over half of the employers had a forecast error equal to or greater than the actual change in employment.

(2) Type Of Forecast And Employment Size Did Not Substantially Affect Forecast Accuracy. Eighty-two, or approximately 40 percent, of the employers participating in the resurvey had originally made "no-change" forecasts, i.e., they predicted no change in total employment or shifts of employment among occupations. No-change forecasts are difficult to evaluate. For example, they can represent the informed judgment of the employer or, as discussed later, can indicate an inability orunwillingness to forecast. As noted earlier, for a no-change forecast the U Statistic values are either 1.0 or 0. addition, the Statistic II value for an incorrect no-change forecast is 1.0. More importantly, it had been hypothesized that smaller companies cannot forecast accurately and thatthe occupational codes do not fit in small companies when workers often perform alarge number of varied tasks.Consequently, it was decided to evaluate the impact of type of forecast and employment size on forecast accuracy. Exhibit IX, following this page, presents the forecast statisticsby type of forecast and size of employment. Appendix H containsthe summary source data for this array. The UStatistic value for a particular employer is computed using the predicted and actual change for each occupation. The computation of the forecast error for Statistics.I and II EXHIBIT IX SUMMARY OFEconomic OKLAHOMA Development FORECASTSBY TYPE Administration AND EmmoTmENT SIZE Type of Forecast/ Number of ...1 Number of Employees Forecast Actual U Average Statistics I Total Statistics 1. Actual .5-1. meat (1)No Change 0 - 100 62 1,908 0.85484 0.32529, 0.77419 1.00000 -. 05917 - 1.00000 (3)(2) Over101 250 - 250 16 4 2,0642,148 2,0642,1481,908 2,4042,0281,724 1.00000 0.131490.15212 0.875001.00000 1.00000 +. 19722- .10649 - 1.00000 1.00000 2. (1)Clumge 0 - 100 54 1,831 2,130 1,946 1.08490 0.35537 1.67940 1.60000 +.09455 1.60000; (3)(2) Over101 - 250250 2829 221 018 4,689 5,108 222 615 24 012 5,201 1.002281.12751 0.147850.16293 1.410121.49940 0.465000.57440 - .05818.01788 4).46660-0,14164 TotalAverage 193 - 33,658 - 35,973 - 37,315 - 0.99661 . 0,26879 . 1.24175 - 0.36666 - .03196 i - - 0.36697 - 4. uses the totals for an employer, thus allowing some offsetting of errors in the projections for individual occupations.In order to prevent additional offsetting errors, an average value for U, Statistic I, and Statis- tic II was computed using the values for all employers in each of the six categories. The averages for Statistics I and II were computed using absolute values, i.e., ignoring the signs. The average error statistics for all firms are: 0.99661 for the U Statistic 0.26879 for Statistic I 1.24175 for Statistic II As. Exhibit IX shows, the averages of the employers in each of the six categories are similar to the average for all employers, with minor exceptions../ The slightly lower values of U for employers who made "no- change" forecasts reflect the fact that the upper limit of U for a no-changeforecast is 1.0000, regardless of the magnitude of the forecast error.Similarly, Statistic II tends toward the unitfor incorrect no-change forecasts. These data also indicate that the average value for Statistic I for small firms is higher than the overall average regardless of the type of forecast. This is probably due to the factthat forecast errors are related to a smaller base in small firms than in large firms. Exhibit IX also presents the three statistics computed rinsing the number of original, forecast, and actual employees in each category and statewide. This allows additional errors to be offset among the firms in each category. Consequently, the statistics reflect a higher degree. of accuracy at this level of aggregation.

it The results of a multiple regression analysis conducted using the EDA RIMED Stepwise Maniple Regression package indicated that the ability of an employer to forecast accurately was not associated with type J fore-. cast, number of employees, or number of occupations* (3; Some Variation Occurs In Forecast ,Accuracy Hy Industry Exhibit X, following this page, displays the values for U, Statistic I, and Statistic II by industry.Similar to the preceding array by type of forecast and employment size, these values are the averages of all employers in a particular industry.Values of Statistic I and Statistic II were ..omputed using absolute values, i.e., ignoring the signs. The range of values is: U Statistic ranges from 0.85152 to 1.17823. The average for all industries is 0.99661. Statistic I ranges from 0.13288 to 0.91375. The average for all industries is 0.26879. Statistic II ranges from 0.82835 to 1.86111. The average for all industries is 1.24175. In our judgment, the statistics are relatively high, indicating poor forecast accuracy. However, the statistics do Indicate that the Finance, Insur- ance, and Real Estate Industry and the Medical and HealthService Industry forecasts are more accurate than average forecasts of employers in Con- struction and Durable Gouds Manufacturing Industries.

(4) The Accuracy Of The pTIS Occupational Forecast Varied By Region The previous analyses were concerned with the ability of an employer or group of employers to accurately forecast manpowerdemand by occupation and in total. As described in Chapter II, the data produced and utilized by OTIS are regional summaries of net manpower requirements by occupation and/or occupational cluster.Thus, the accuracy of the net requirements forecast by occupation is a function of the accuracy of the demand forecast.Consequently, the accuracy of the forecast for manpower demand by occupation must be determined.

-29- EXHIBIT X

Economic Development Administration BEST COPY1,:,-.;'.1:::;LE SUMMARY Cif. OKLAHOMA FORECASTS

,BY INDUSTRY

T Number of Number of Em .1. ees vera Statistics Industry Em ors Ori :,. al Forecast Actual U

Construction 6 198 231 251 1.040660.809631.86111

Durable Goods 23 6,189 7,167 8,261 1.178230.243891.15328 Manufacturing

Finance, Insurance, 31 2,645 2, 812 2,860 0.851520.132881.19629 and Real Estate

Medical and Health 12 2,804' 2,964 3, 363 0.894810.159460.82835 Services

Mining 15 1, 014 1, 008 915 0.870520.913751.33384

o Nondurable Goods 18 4,544 4, 789 4, 519 1.135080.179681.41440 Manufacturing

Public Utilities 23 8, 369 8,693 8,951 1.138870.144710, 97322

Service 12 401 430 389 0.993360.549911.68518

Trade 53 7,494. 7, 879 7,196 0.948380.155111.28048 Total 193 33,658 35,973 37,315 .. ..

Average - - - - 0.996610.268791.24175 As part of our analyses, theoccupational projections of each employer in a region were totaled toobtain regional summaries of manpower demand byoccupation. A sample summary is includedin Appen- dix H. The accuracy of these summarydata was evaluated using the same th;:ee error statistics used to evaluateemployer forecasts. Exhibit XI, following this page, displays the errorstatistics for the occupational summary in each region. Errors amongemployers for a specific occupation are netted out before anycalculations are made. These statistics vere computed in the same way as the statisticsfor an individual employer. The U Statistic was computed using thepredicted and actual changes for each occupation in the summary. Statistic I was computed using the totals forall of the occupations in the region, thereby allowing someoffsetting errors among occupations to netout.

els Statistic II was also computed using the totalsfor all occupations, thereby allowing some errors tobe offset. As indicated in the exhibit, thestatistics vary in value by region. Excluding regions 1 and 11 because they haverelatively .srall numbers of employees and extremely large errorstatistics, the range and average for each ofthe three statistics are:

The U Statisticranges invalue from 0.6727 to 1.1492. The average U for the remainingnine regions is 0.9393. Statistic I ranges in value from 0.0177 to0.1760. The average, computed using absolute values, is0.0641. Statistic II ranges in value from 0.0848 to 0.6731.The average, computed usingabsolute values, is 0.5288.

The regionalperformance variesby statistic.For the U Statistic, regions 2 and 4 made the best forecast and regions3 and 5 theworst.Based on Statistic I values, regions 3 and 9 madethe best forecast and regions 4 EXHIBIT XI rIc.1,1101. rre-trrl Economic Development Administration

OKLAHOMA OCCUPATIONAL FORECASTS BY REGION

tuber of 1 ees Statistics Region Original Forecast Actual U I II

1 519 568 524 1.1498 .0840 8.8000

2 1,965 2,056 2,210 0.6727 .... 0697 -0.6286

3 574 753 739 1,1492 .0189 0.0848

4 1,204 1,353 1,642 0.8216 .1760 -0.6598

5 738 814 789 1,1109 .0317 0.4902

6 3,435 3,556 3,711 0.8241i -.1027 -0.65S8

7 1,851 1,969 2,212 0.9583 -.1099 .0.6731

8 16,150 17,092 16.715 1.0334 .0226 0.6673

9 526 SS4 564 0.9107 ...0177 -0.2632

10 245 249 256 0.9725 -.0273 -0.6364

11 487 565 519 4,4795 .0886 1.437S

Total 33,658 35,973 37,31S - -. ..

Average - - ... 0.9393 .0641 0.5288 (9 Region) and 7 the worst.In addition, region 8, the largest region in terms of employment, had an excellent Statistic I value of approximately 2 percent.For Statistic II, regions 3 and 9 made the best forecast and regions 7 and 8 the worst.

(5) Combining Occupational Totals Did Not Significantly Improve The Accuracy Of The Test Clusters The preceding section discussed the accuracy of the regional summaries of manpower demand by occupation. The basic finding was that the occupational forecasts were often inaccurate.However, OTIS groups various occupations into vocational education training clusters and pub- lishes summary data by cluster. This procedure combines occupational totals, thereby providing opportunity to offset or average errors in individual occupations.Therefore, to fully understand the entire forecasting problem, an examination of the impact of clustering on forecast accuracy was required. Based on an examination of the OTIS Cycle Five Report, EDA decided to examine six clusters in four regions and on a statewide basis. Data from the resurvey were clustered and analyzed using the three error statistics.Exhibit XII, following this page, presents the results of this analysis.In general, the forecast accuracy was not substantially improved. The principal reason for this lack of major improvement is that many of the forecasts in a particular cluster tended to be either low or high, rather than a combination of some high forecasts with some low forecasts. Con- sequently, errors for individual occupations were averaged as a result of clustering rather than reduced substantially by offsetting errors in the occupational forecasts. EXHIBIT XII .. Economic Development Administration ,r,°,;,11,11.c. BEST Cr; FORECAST ACCURACY BY CLUSTER IN SELECTED REGIONS IN OKLAHOMA

Number of Employees Statistics Region Cluster Ori alForecast Actual U I II

Carpentry 4 16 20 19 0.332 0.053 0.333 6 SO 55 SS 0.686 0.0000.000 7 35 38 36 1.141 0.056 2.000 8 143 16S 14S 1.952 Ot 13810.000 State 342 379 333 1.4390.074 2.364

* Practical (Vocational) M using 4 -- -- N -- N -- (Nat a Cluster) 6 * . -,. -...... -- , 6 9 7 2.0000.285 2.000 * 8 -- -. -- -- ..- ....

State 1 218 252 244 0.531 0.0330.308

Electricity /Electronic Technology/Electra- 4 1 2 4e& 0.978.0.9560.978 mechanical Teciviclogy 6 5 8 1 1.74¢ 7, 000 1.750 7 4 4 4 0.0000.0000.000 8 12 12 . 12 0.000 41 000 0 000 State 55 60 87 0.872-0.310-0.844

Machtue Shop/ Machine Tool Operation 4 8 12 10 1.0000.200 1.000 6 189 216 234 1.135-0.077 0.400 7 12 14 14 0.000i 0.000 0.000 8 110 12S 156 0 6034, 198-0.674 State 373 426 470 0.848-0. 094-0.454

Accounting and Computing 4 SO 53 57 1.118-0.070-0. 571 6 551 570 645 0.907.0. 116-0.798 7 199 211 181 1.2650.166 1.667 8 808 865 876 0.757-Oz 013-0.162 State 1,876 1,974 2,079 0.812.0.081-0.517

Transportation/Truck Driver 4 72 88 104 0.826-0.154-0. 500 6 696 695 819 0.907-0.151-1.008 7 45 45 47 0.906-0.043-1.000 8 37 46 65 0, 680-0 292-0t 679

State 1, 882 2,029 2,188 0.572-0.073-0.520

Resurvey data not available 4

(6) The Forecast For The "AU Remaining Employees" Category Did Not Affect The Overall Forecast Adversely The survey instruments used to collect the OTIS demand data contain a "catch-all" occupational category called "All Remaining Employees." The use of this category simplified data collection and preparation.Since over 30 percent of total manpower demand is reported in this category, an analysis of the impact of the "All Remaining Employees" category on forecast accuracy is required to complete the forecast picture. Exhibit XIII, following this page, presents the results of our analysis of the effect on forecast accuracy of the "All Remaining Employees" category.This analysis is based on a comparison of Statistic I and II for total employment and adjusted total employment. The adjusted total was computed by subtracting "All Remaining Employees" from total employment. As indicated in the exhibit, the adjusted total was: More accurate in three regions Less accurate in six regions Approximately the same in two regions The statewide totals were also approximately equal. Thus, the inclusion of the "catch-all" category did not affect forecast accuracy adversely.

3, THE FORECAST OF MANPOWER DEMAND MADE BY EMPLOYERS PARTICIPATING IN THE KENTUCKY RESURVEY WAS OVERESTIMATED This section presents our analysis of the accuracy of the employer and occupational forecasts in Kentucky. The analyses, error statistics, and calculations are similar to the procedures utilized to assess the OTIS data in Okla- homa.In contrast to the resurvey of about 200 employers in Oklahoma at the end of an approximate two-year forecast period, the resurvey in Kentucky was undertaken at the end of a one-year forecast period; and included 301 firms. The resurvey methodology, including sample selection and survey procedures, is described in Appendix E. The results of a comparisonof forecast and actual

-32- rra EXHIBIT X1U

MARIE. Economic Development Administration fc.S1CT/ ALL REMAINING EMPLOYMENT FOR OKLAHOMA

Number Of Erardovset Statistics Restion catestory Original Forecast Actual { 1 AU Remaining 144 166 140 0.1857 .6.5000 Total 519 568 524 0.0840 8.8000 Adjusted Total 375 402 384 0.0469 2.0000

2 All Remaining 594 642 718 -0.1058 -0.6129 Total 1,965 2,056 2,210 -0.0697 .0.6286 Adjusted Total 1,371 1,414 1,492 -0.0523 -0.6446

3 AU Remaining 122 134 137 -0.0219 -0.2000 Total 574 753 739 0.0189 0.0843 Adjusted Total 452 619 602 00283 0.1133

4 All Remaining 276 304 301 0.0100 0.1200 Total 1,204 1,353 1,642 -0.1760 -0.6598 Adjusted Total 928 1,049 1,341 -0.2177 -0.7070

S All Remaining 229 260 246 0.0569 0.8235 Total 738 814 789 0.0317 0.4902 Adjusted Total 509 554 S43 0.0202 0.3235

6' All Remaining 3,435 3,556 3,711 -0.0418 -0.5616 Total 9.399 10,000 11,145 -0.10V .0.655e Adjusted Total 5,964 6,444 7,434 -0.1332 -0.6735

7 All Remaining 373 389 380 -0.4941 - 0.9596 Total 1,851 1,969 2,212 -0.1099 -0.6731 Adjusted Total 1,478 1,580 1,822 -0.1375 -0.7119

8 All Remaining 5,896 6,074 5,863 0.0360 -6.3939 Total 16,150 17,092 16,715 0.0226 0.6673 Adjusted Total 10,254 11,018 10,852 0.0153 0.2776

9 AU Remaining 140 152 169 -0.1006 -0.5862 Total S26 554 564 -0.0177 .0.2632 Adjusted Total 386 402 395 -0.0177 0.7778

10 All Remaining 3S 36 70 -0.4857 -0.9714 Total 245 249 256 -0.0273 -0.6364

Adjusted Total * 210 213 186 0.1452 -1.1250

11 All Remaining o 51 51 49 0.0408 -1.0000 Total 487 565 519 0.0886 1.4375 Adjusted Total 436 514 470 0.0936 1.2941

Total All Remaining 11,295 11,764 12,173 -0.0336. .0.4658 All Regions Total 33,658 3S, 973 37,315 -0.03601 -0.3670 Adjusted Total 22,363 24,209 25,142 -0.0371 -0.3357 _employment indicated that the forecast of demand was overestimated by 1,100 positions.This overstatement represents an error of 2 percent when compared to actual employment and 98 percent when related to actual change in employment. During the forecast period, manufacturing employment in Kentucky increased by approximately 6. 5percent.-W Our analysis of the Kentucky resurvey data indicated that manpowerdemand was overstated.Specifically, the results of our analysis indicate that: Forecast accuracy also varied widely among employers. Forecast accuracy was not influenced substantially by type of forecast or employment size. Significant variations in accuracy by industry did not occur. Accuracy of the occupational forecast varied by region. Clustering did not improve forecast accuracy significantly. These findings are discussed in the following sections.

(1) Forecast Accuracy In Kentucky Also Varied Substantiall Among Employers Forecast accuracy among the 301 manufacturing employers in Ken- tucky participating in our resurvey also varied widely. Exhibit XIV,following this page, presents the distribution of employers foreach of the three error statistics.The complete distribution by region for each statistic is included in Appendix G. A sample printout for amanufacturing firm in Kentucky also is included in this appendix. As shown in Exhibit XIV, the range of the U Statistic variesfrom 0, or a perfect forecast, to over2.0000. The distribution of forecasts within this range is:

Insured employment figures from Kentucky Department of Labor.

-33-

O EXHIBIT XPI DISTRIBUTIONEconomic OF Development ERROR STATISTICS Administration U Statistic Statistic I Statistic U IN KENTUCKY NumberEmp:oyers of 33 Value 0 EmployersNumber of 7 - Over .500 Value EmployersNumber of 60 - Over 1.000 Value -- 2 .2000.0001 - .1999.3999 19 8 -.201-.251 --.500 .250 127 7 -.801 - 1.000 93 .. .4000 - .5999. 15 4 -.101-.151 - .150.200 12 9 -.401-.601 - .600.800 137 25 .8000.6000 - .9999.7999 1.0000 2636 -.001-.051 - .050.100 42 5 -.001-.201 - .200.400 0 4915 1.25011,0001 - 1.50001.2500 2742 +.001 - .050 0 3 1 +.201+.001 - .200.400 47 1.50011.7501 - 1,7500- 2.0000 2032 ' +.051 +.101 - .150 ... 100 5 +.401"4601 - .600 - .800 17 Over 2.0000 11 7 +.201+.151 --.250 .200 24 1 ++.801 Over - 1.000 1.000 A 3116 ++.251 Over -.500.500 I Some 33. or approximately 11 percent, of theforecasts were perfect forecasts. As noted earlier, a perfect forecastrequires the forecast for each individual occupation to be correct. A further 39, or approximately 13 percent, of theforecasts had a U value between .0001 - .9999. Some 137, or 45.5 percent, of the forecasts had a Uvalue of 1.0000. The high incidence of firms with a U value of1.0000 is caused by the large number of incorrect no-changeforecasts. As noted earlier. an error in a no-change forecastresults in a unit value for the statistic,regardless of magnitude. A total of 92, or slightly over 30 percent, of theforecasts had a U value of1.0001 or over. These results indicate that, in over 75 percentof the forecasts, the error was either equal to or greater thanthe error associated with a simple no- change forecast.In other words, in the majority of the cases, a simple no-change extrapolation of current employment atthe time the survey was taken would haveyielded results equal to or better than those obtained from the forecast. The Statistic I values range from over -.500 to over+.500. The forecasts are distributed fairly evenly throughout this range.Specifically, of the 301 forecasts: Some 42, or 14 percent, of the total were perfectforecasts. The majority of these were no-change forecasts.A higher number of perfect forecasts occur in this statisticthan in the U. Statistic because only the totals and notall of the occupational forecasts must be accurate. A group of 115, or approximately 38.2 percent,forecast manpower demand below actual. The remaining 144 employers, or 48 percent,forecast a higher\ demand than was observed. Using the absolute values. the range offorecast errors compared to actual employment is:

-34- a

Some 121, or over 40 percent, of the forecasts had a Statis- tic I value of from .001 to .100. Some 50, or approximately 16.6 percent, of the total had a Statistic I value of from .101 to .200. A further 65, or approximately 21.6 percent, of the total had a Statistic I value of from .201 to.500. The remaining 23, or approximately 7.6 percent, of the forecasts had a Statistic I value of over .500. Thus, over 29 percent of the employers made forecast errors of over 20 percent when compared to total actual employment. The range of Statistic II varies from over -1.000 to over +1.000. The range of this statistic is greater than the range forStatistic I because the same forecast error is applied to actual change in employment rather than total actual employment. Fourteen percent, or 42 employers,made perfect forecasts.Ignoring the signs, the range of forecast values is: Some 18, or 6 percent, of the forecasts had a Statistic II value of from .001 to .400. Some 29, or about 9.5 percent, of the total had a Statistic II value of from .401 to .800. A total of 128, or nearly 43 percent, of the forecasts had a Statistic II value between .801 and 1.000. The large number of employers in this range is caused by.the high incidence of incorrect no-change forecasts, which result in a unit value. The final 84, or nearly 28 percent, of the forecasts had a Statistic II value of over 1.000. These data indicate that over half of the firms made aforecast error equal to or greater than the actual change in employment overthe forecast period. (2) Fo.-e.cast Accuracy Was Not Affected Substantially B_ y Type Of Forecast Or EmploymentEm to ment Size Nearly one-half of all of the forecasts in the initial OTIS survey were no-change forecasts. As noted earlier, it is difficult to identify the reason for a no-change forecast. More importantly, as discussed later, many no-change forecasts in Kentucky were actually no forecasts. Exhibit XV, following this page, presents the forecast statistics by type of forecast and employment size. Appendix H contains the summary source data for this array. The U value for.a particular employer is computed using both the predicted and actual changes for each occupation. The computation of the forecast error for Statistics I and H tides the totals for an employer, thus allowing offsetting of errors among the occupations.In order.to prevent additional offsets, an average value for U, Statistic I, and Statistic II was computed using the values for all,employers in each of the six categories. The averages for Statistics I and II were computed using absolute values. The average error statistics for all employers is: 1.08004 for the U Statistic It 0.30372 for Statistic I 1.88408 for Statistic II A comparison of the statistics in each category with the averages indicates substantial departures.In our judgment, however, these departures reflect the characteristics of the error statistics and do not indicate differing abilities to forecast among the groups.liFor example, the much lower values of U for no-change forecasts reflect the fact that the upper limit of U for a .no- change forecast is 1.0.Thus, with an infinite upper limit possible in change forecasts, the average is higher.Additionally,

1/ The results of a multiple regression analysis conducted using the DA BIOMED Stepwise Multiple Regressiou package indicated that the ability of an employer to forecast accurately was not associated with type of forecast, number of employees, or number of occupations,

-36-

4 p EXHIBIT XV SUMMARYEconomic OF KENTUCKY Development FORECASTSBY AdministnotiOn TYPE AND EMPLOYMENT SIZE Actual EmplarznentType of Forecast/ lumber of ,,,.1. en Original Number of ),., Forecast us Actual Ay-, , e Statisti ,, I U U Total Statistics 1 II 1. (1)No Change 0 - 100 97 2,962 2,962 2,957 0.72164 0.2761 0.69070 1.00000 .00169 - 1. r rni (3)(2) O101 v e -r 250 323 0 20,673 3, 763 20,678 3,763 20,853 3,846 0.956350.95652 0,0,1077: -,.=,. 0.933300.91304 1.000001,00000 -.00839-.02158 . $16 6 6 0 1 6 6 1 2. (1)Change 0 - 100 80 2,809 3,379 2,753 1.60102 0.5685 3,61390 10.00000 .22739 - 11.1785 (3)(2) Over101 250 - 250 3833 24,175 5,080 24.183 5,727 253104,873 11.27257 237309 0,11272.47, 2.630301,97880 0.115004,1200 .00502.17525 0.1114,125 , '$ AverageTotal 301 - .....59,467 - ....-...61,692 - 60,592 - 1.08004 - 0.303 - 1, 88408 - 0, - .01815 .. 0,97778 - I Cr .: VNlWW4ryl

the statistics do not indicate that accuracyvaries by size of employment. However, they do indicate thatStatistics I and II are higher for small firms.In our judgment, this is due toapplying a forecast 7ror to a smaller base in both cases, e.g. , theactual employment and probably the actual changes are smaller in smallfirms than in large firms. Exhibit XV also presents the three statisticscomputed using the number of original, forecast, and actualemployees In each category and statewide. This allows additional errors tobe offset among the firms in each category. However, with theexception of Statistic I, the statistics are still relatively high.

(3) Forecast Statistics Do Not Indicate MajorIndustry Differences Exhibit XV I, following this page,displays the average value for U, Statistic I, and Statistic II of theemployers in each industryclassification to the values for Statistics Iand II computed using theabsolute values for each employer. The range of the averagevalues is: U Statistic ranges in value from0.061953 to 2.72777. The average for all industries is1.08004. Statistic I ranges in value from 0.00870 to0.65428. The aver- \ age for all industries is0.30372. Statistic II ranges in value from 0.80000 to7.50000. The average for all industriesis 1.88408. Most of the statistics arerelatively high, indicating inaccurateforecasts by industry. Me:importantly, these averages do notindicate a clearcut example of one industry producingbetter forecasts than others for all three statistics.For example, the best forecast inU, Statistic I, and Statistic 11, respectively, are theLeather and Leather ProductIndustries, Petroleum Refining and CoalProduct, and the Musical InstrumentsIndus- tries. The worst industries werethe Nonferrous Rolling andDrawing/ Miscellaneous Primary MetalProducts, Tobacco, and StoneProducts

-.37- EXHIBIT XVI

BEST COPY AVAILABLE Economic Development Administration

SUMMARY OF KENTUCKY FORECAST'S BY MANUFACTUR INC. INDUSTRY

Number of Number of Duplaes Average Statistics Manufacuning indratry Ernplovera Original ForecastJ Actual U t U

Metal Workieg 97 21,455 22,763 21,881 1.250950.318332.01965 Meat Pocking 6 1,093 1,099 1,123 1.004250.323231.25000 Dairy Products 9 416 434 493 0.622410.116231.56771 Canned and Preserved Fruits, Vegetables, and Sea food 3 536 635 473 1.717990.214184.77419 Grain M111 Products, Sugar, Confectionary, and Mite. 9 564 572 570 1.177020.100061.22222 Foods

Bakery Products 1 200 200 203 1.000000.014781.00000 Beverage Products 26 3,275 3,254 3,241 1.066170.213572.11035 Tobacco Industry 6 3,709 1,852 3,634 0.847790.654280.92394 ,. Textile Mill Products 3 1,654 1,770 1,829 0.843510.103150.98093 Apparel and Other Textile Products Industries 18 5,476 5,590 5,939 0.954030. 137592.16494 Lumber and Wood Products 21 1,338 1,450 1,231 0.980210.092311.93487 Furniture and FintIleS Industry 13 1,722 1,734 1,615 0.881410.320030.65384 Converted Paper and Paperboard Products 8 1,466 1,512 1,526 1.162380.064422.75173 Printing and Publishing Industries 26 2,851 2,898 2,731 0.1337290.47891 1.05083

Industrial Chemicals Industry 2 290 298 323 1.121580.168722.96875 Hegira and Synthetic Fibers Industry 6 3,536 3,566 3,635 0.989700.132920.91216 Soaps, Cleaners, and Toilet Preparations 2 18 20 17 1.000000.369035.50000 Paints and Allied Products, Agricultural Chemicals, and 7 215 227 224 1.035460.080600.73809 Miscellaneous Chemical Products

Petroleum Refining and Coal Product Industries 1 116 116 115 1.000000.008701.00000 Rubber Tins and Tubes and Fabricated Rubber Products 2 93 93 93 1.000000.071981.00000 Miseellasseorst Plastic Products Inchistry 10 1,042 1,173 1.040 1.798460.151753.65036

1 Leather and leather Products Industries 2 117 123 124 0.619510.050005.00000 Footwear Cut Stock and Footwear Except Rubber 1 199 199 278 1.0000010.284171.00000

Flat Cass, Glassware, and Giant .%ducts 1 91 91 95 1.000000.062111.00000 Cement, Structural Clay, Amery, Piaster 11 257 284 284 0.679870.214700.88636

Steno Products and Miscellaneous Mineral Products 1 48 61 46 1.50000O. 326097.50000 Industries Shut Furnaces and Basic Steel Products Industries 4 6,076 5,993 6,043 2.218380,076703.12570 sos Iron and Steel and Nonferrous Foundries 3 493 610 1.009310.178830.72222 Nossfertous Rolling and Drewhig/MisecUanecnis Primary S 984 1,016 1,020 2.727770.145413.60000 Metal Products \ :/ Musical Instruments, Toys, Sporting Goods, and Other ...k.5 137 141 154 0.803060.080440.60000 Selected Products Total 301 59,467 61,692 60,592

Average - 1.080040.303722.88408 - -- ... and Miscellaneous Mineral Products Industries. More importantly, the large industries group with the most employers and employees had rela- \tively high values for all three statistics, including Metal Working, Blast Furnaces and Basic Steel Products, and Apparel and Other TextileProducts.

(4) The Accuracy Of The Kentucky OTIS Forecasts Varied By Region The previous analyses considered the ability of an employer or various groups of employers to forecast manpower demand accurately. However, OTIS actually produces and uses regional summaries of net manpower demand by occupational cluster. Manpower requirements are then determined by interfacing manpower supply and demand data by occupation.Consequently, to understand the actual utilization of the subsystem, an assessment of the accuracy of the forecasts of manpower demandby occupation is necessary. Regional summaries of manpower demand by occupation were compiled from our resurvey data. A sample summary is included in Appendix H. The accuracy of the regional summaries was assessed using the same three statistics used to evaluate employer forecasts. The regional data were computed in the same manner as the data for anindividual employer.

. The U Statistic was computed using predicted and actual change for each occupation.

. Statistic I and Statistic II were computed using the totals for all occupations. Thus, U Statistic allows only net outs among firms for a specific occupation.However, Statistic I and Statistic II allow net outs among employers and among occupations before any calculations are made. Exhibit XV H, following this page, presents the error statistics for each regional summary. The range and average of each statistic are:

-38- EXHIBIT XVII

Economic Development Administration

031°71 1ki171- KENTUCKY OCCUPATIONAL. FORECASTS BY REGION o

Number of Emp loes Statistics Region Original Forecast Actual 1 II

1 387 395 361 1.00542 0.09418 - 1.30769

2 2,383 2, 877 2, 633 0.99588 0.09267 -1.22000 .1.64179 4 3 5,995 6,129 5,794 1.22450 0.05696

4 5,498 5,924 6,064 0.94112 -0.02309 -0.24735

5 417 455 498 0.94502 - 0.08635 0.53086

6 21,362 21,950 21,495 0.932222 0.02117 3.42105

7 3,445 3,743 3,643 1.09780 0.02745 0.50505

8 1,560 1,681 1,684 0.94828 - 0.00178 .0. 02419

9 307 307 311 1.00000 -0.01286 .1.00000

10 4, 862 4, 893 4, 793 1.09996 0.02086 -1.44928

14 603 652 740 1.03912 -0.11892 .0.64234

15 12,198 12,691 12,576 0. 95930 0.00914 0.30423

Total 59,467 C1,692 60,592 - - .. Average . - - 1.01572 0.04712 1.02449 The U Statistic ranges from a lowof 0.93222 in region6 to a high of 1.22450 in region 3. The averageof all, the regions is 1.01572. Statistic I ranges from a lowof 0.00178 in region8 to a high of 0.11892 in region 14.The average value for allregions is 0.04712. Statistic II ranges from a lowof 0.02419 in region 8 to ahigh of 3.42105 in region 6.The average value is 1.02449. The regional performancevaries by statistic.The U Statistic was high in all regions. However,considering Statistics I and II, regions8, 4. and 15 had relatively goodforecasts.ConVersely, regions 3, 14, and 1 had relatively poor forecastsbased on Statistics I and II values.

(5) Combining The Occupation Data IntoClusters Did Not Significantly Improve Forecast Accuracy As discussed in our earlieranalysis of the Oklahomademand data, the Project MonitoringCommittee decided to test theeffect on forecast accuracy of combiningoccupational data into clusters.Six clusters were selected by EDA for testing infour regions and also statewide.The Com- mittee also wanted to test thepotential impact of clusteringin Kentucky. Since the Kentucky OTISdata are not published andthe clusters not available, the Oklahoma clusteringscheme was utilized to testthe Kentucky data. The occupationalclusters drawn up in Kentucky werederived from an analysis of theOklahoma OTIS Cycle Five Reportand the Office of Edu- cation instructional programclassifications. Once the appropriateDOT codes were selected, thecorresponding BLS codes werematched and the clusters developed. These data were analyzedusing the three basic errorstatistics. Exhibit XVIII,following this page, presents the results of thisanalysis. In general, forecast accuracy wasnot improvedsubstantially. As was apparent in the case of theOklahoma clustering data, the reasonfor this

-39- EXHIBIT XVIII

Economic Development Administration

EFFECT OF CLUSTERING IN KENTUCKY

Number of Era es Statistics Cluster Region Original Forecast Actual V I II

Machine Shop/Machine Tool 3 111 114 95 1.039000,200001.18750 Opera dm 4 595 628 604 0.762230.397002.66600 6 427 437 517 0.95810-.O. 15470- 0.88800 15 56 224 144 0.658700. 555000.90900 State 1,890 2,244 2,160 0.927500.388900.31111

Accranting and Computing 3 116 117 108 1.009900.,083301.12500 4 175 177 188 1.02460- 0.05850-0.84615 6 396 402 392 1.545960.025502.50000 15 71 72 81 0.94921- 0.11110- 0.90000. State 981 1,073 981 4.354000.0937810.00000 .4...... 4 lim.....mum

Transportation/Truck Driver 3 104 107 97 1.131370.103091.42850 4 410 429 321 1.180000.336401.21348 6 621 662 632 2.128000.047402.72700 15 60 62 71 1124490,..0.12670- 0.81818 State 1;614 1, 695 1,585 2.320000.069403.79310

Carpentry 3 75 81 59 1.178900.375001.37500 4 24 24 20 1.000000.200001.00000 6 506 517 483 0.017660.070391.47826 15 203 212 138 1.004980.536201.13846 State 965 1,007 876' 1.152710.14954- 1.47191

Sheet Metal 3 45 47 52 0.82764.0. 09615.0.71428 4 76 72 83 1.86547.0.13253- 1.57142 6 331 343 388 0.95289.0.11597- 0.78947 1S 357 406 411 0.00587.0. 01216 4.0.09259 State 1,305 1,464 1,420 0.331000.031000.38261

Composition, Makeup, and Type- 3* 8 8 8 0.000000.000000.00000 setting/ Printing 4** ...... 6 287 295 286 9.000000.031469.00000 15 SS 55 48 1.000000.145801.00000 State 363 371 355 2.000000.04507- 2.00000

Notes: * Inadequate data ** Resurvey data notavailable lack of improvement is that many of the occupational forecasts for manpower demand in a particular cluster were either high or low. Conse- quently, errors for, individual occupations Were lveraged rather than reduced by offsetting errors.

4. THE INITIAL OTIS DEMAND DATA COLLECTION EFFORT IN KENTUCKY WAS A LARGE UNDERTAKING The demand data collection effort in Kentucky involved the collection of employer forecasts through a census of all state manufacturing establishments. As already noted, data have been collected recently from 2,175 employers.In addition, ongoing data collection is presently being directed toward more than 8,500 nonmanufacturing employers. The primary reasons for conducting a statewide census through personal interviews are to: Foster a rapport between industry and educational personnel, thereby:

Improving response rates Acquiring assistance in the development of accurate occupational forecasts Facilitating counseling, placement, and curriculum modification efforts Enhancing placement prospects for vocational/technical graduates Stimulate interagency support and interest by cooperating to collect data Consequently, an evaluation of this approach must focus on the advantages of increased response rates and employer rapport over the disadvantages of increased costs and management problems. (1) Increased Rapport EmployersWas Established In Many Instances Two expected results of the personal interview approach utilizedby OTIS in Kentucky were the establishment of rapport with employersand improved data collection. The most recent personal interViews involved over 500 vocational education teachers and state agencypersonnel, and 1,551 man-days of effort.Thus, approximately one-half of a day, including travel time, was allowed for each interview and information wascol- lected about 2,175 firms.Clearly, this amount of time is adequate to complete a survey instrument and establish communications. Hence, rapport and .abaSis for future cooperation were probably established with many manufacturing employers. Nonetheless, the results of our resurvey indicate that the anticipated level of rapport was notrealized because the data collection effort was not carried out properly in some instances andbecause some employers did notagreewith the survey aims. For example, of the employers resurveyed, approximately25 percent did not remember completing the survey. After discussing the surveywith the resurvey interviewer: Fifteen percent of the employers stated that the personalvisits involved only presenting the survey document and collecting it later on. Twenty-five percent of the employers said the survey instrument was received in the mail with a self-addressedenvelop and no instructions. Fifteen percent of the employers said the survey instrument was presented and collected byvocational education students, but that no instructions were received. In addition, some employers statedthat they (1) had prepared too many forms and that OTIS forms duplicated the ESdocuments, (2) believed the training of vocational education graduates wasinadequate, and (3) could not hire vocational education graduates due to lowturnover or

-41- restrictions on outside hiring.Therefore, in many instances, the survey did not result in theexpected employer rapport. More importantly, the lack of both personal contact andinstructions to accompany the questionnaires hampered the data collection ieffort inseveral instances.

(2) Many Kentucky No- Change Forecasts Exhibit XIX, following this page, presents a summaryof the forecasts received in Kentucky. Over 50 percentof the employers participating in the survey made no-changeforecasts. The percentage of no-change forecasts ranged from 45 percent in regions 12and 15 to 73 percent in region 9.Similar percentages were noted in our resurvey. Clearly, many of these forecasts reflectaccurate employer judgments of expansion demand, e.g., mostof the correct forecasts%fere no-change forecasts. However, our employerinterviews indicate several other important reasons for the highincidence of no-change forecasts. For example, some employers did not: Receive instructions to make aforecast Understand they were forecasting Make any forecast at all Consequently, we conclude that manyof the forecasts reported as no-change were in fact non-forecasts.Several of the employers interviewed were sure that they hadmade no forecasts, but the printoutsindi- cated no-change forecasts. Moreimportantly, some employers kept copies of their original documents,verifying that, in fact, no forecasts were made.In our judgment, the high incidence of no-changeforecasts is attributable not only tomisunderstanding of instructionsand/or an unwillingness to forecast, but also to counting noforecasts as no-change forecasts. Economic Development Administration EXHIBIT XIX Respondents Projecting SUMMARY OF EMPLOYER FORECAST RESPONSES: KENTUCKY p Area of SurveysNumber Total NumberRespondents Projecting No Change * Percent NumberRespondents Projecting Increase Percent NumberIncrease in First Yearand No Change in Second Year Percent NumberRespondents Projecting and IncreaseSecond Year in Find Percent GreenPennyr)lePurchase River 179123131 115 6270 645350 6461 473650 402637 281533 213824 211817 BarrenLincoln River Trail 164 89 4889 54 4175 46 2820 2217 2147 242918 JeffersonNorthern Kentucky 147570 26 380 8317 676556 190 64 9 443533 2986 6 232015 fb4 35 3 2412 FivcoGatewayBuffalo Trace 6022 3116 7352 29 6 2748 16 1 27 5 13 5 n22 BigKentucky Sandy River 31 1415 i.? 4548 1617 5255 87 2623 32 9 29 BluegrasslakeCumberland Cumberland Valley 294134110 131 7053 454852 163 6457 485552 692530 2322 9434 252932 Note: * Declines are included in No Change; very few were reported. Total 2,111 57 917 43 428 20 489 23 (3) Delays In Employer Submissions Shortened ForecastPeriods Originally, OTIS demand data were to be gathered inJune, 1972. Employers were asked to forecast their manpower needsfor May 15, 1973, and May 15, 1974, so that one-year and two-yearforecasts would be available.Unfortunately, a substantial amount of the data were not submitted promptly. Exhibit XX, following this page, presents adistribution of the months during which data were submitted andreceived by OTIS. This exhibit shows most of the data being collected inSeptember, October, and November, 1972, but also that data were stillbeing collected in April, 1973. Consequently, the forecasts wereprobably submitted shortly after preparation and, of the 2,214 respondents: 247 employers, or slightly over 11 percent,made a forecast for 10 to 12 months. 1,856 employers, or nearly 84 percent, made aforecast for seven to nine months. 111 employers, or 5 percent, made forecasts for sixmonths or less. This distribution of responses can bedescribed as follows: The mean forecast period was 8.53 months. The modal forecast period was nine months. The median forecast period was ninemonths.

5. OTIS DEMAND DATA AS PRESENTLYCALCULATED DO NOT REPRESENT TOTAL MANPOWER DEMAND Our analysis indicates that the demanddata compiled in both states do not represent total manpowerdemand. The principal causes of incompletedata involved the current status of the system inKentucky and the instruments for collecting demand data in Oklahoma. ECODOTElie Development Admin istration EXHIBIT XX Monthly Survey Responses Received DEMAND DATA SUBMISSION DATES BY REGION: KENTUCKY Reids 06/72 07172 08/72 09/72, 11_2/21 11/72 12/72 01/73 02/73 00001 03/73 04J73 Total Purchase 0010 0011 1010 106 0. 18 ..... IWO 2 .... 126 Permyrile 1 30 ...... 01 .00 M. IMO010 101.01. 001001. 1..0 130 Green River .... 1010- 186 ..... 15 2 110. 203 Barren River wow-- IMO MO 1 146 MO -- 23 ... -.. OM.... MAO 1 171 Lincoln Trail . 77 9 I 010.0 11Nr. .... 87 ..... 10.1 10. 1 W...... Jefferson 00. 11.11 50 219 1.10 269 31 ON, MHO 569 Northern Kentucky 40011 IOWM. 126 21 MI. 0110014 9 170 Buffalo Trace 1110-- 22 2 5 .... 3 NMI NO.-- 3217 4, Gateway 17 2 37 9 72 7 4 UMW 68 Five° 1001 0.0 11010 UM, MOM .00 IMO WPM 0000 :74*.7..J Big Sandy 1010 30 OM* -001001 100. 30 .101 Kentucky River UMW 22 1110. 5 1011 4 31 103 Cumberland Valley MOP94 IMO 11.M 122 16 1110 11010 IOW 11010 falle 110136 BluegrassLake Cumberland 0010 2 59 .100 3814 00110 1 3 7 OW. 334 Total 224 1111M111M .00 2 3 1171011=-111,M 344 441 38 saes 21 17 17 26 9 2, _214 (1) OTIS Kentucky Was Unable To Incorporate Replacement De andm Data Into The Total Demand Data System Since Attrition Rates Were Unavailable Total demand for manpower equals the total number of job openings that are available or are expected to be available over the course of the forecast period.In other words, total demand is comprised both of expansion demand and of replacement demand. Expansion Demand refers to changes in the number of jobs due to expansion or reduction in manpoiver needs. Thus, expansion demand is usually positive in expanding industries, zero in static industries, and negative in declining industries. Replacement Demand refers to openings created by normal attrition in the work force due to deaths and retirement. The OTIS survey instruments utilized in Kentucky collected projections of the total number of pbsitions expected to be available as of the forecast date. These forecast totals are compared to existing employment levels to obtain the net change in positions available.Thus, OTIS Kentucky collects only expansi& demand data. Kentucky was not able to incorporate replacement demand into total demand in the original system design because attrition data were not available.11However, combined death and retirement rates by occupation have been calculated recently from the 1970 decennial census data. The weighted average rates for the .total work force in Kentucky are approxi7 mately 4 percent.It is reasonable to assume that the calculated death and retirement rates are representative of the manufacturing work force in Kentucky. Hence, these rates can be applied to the number of employees included in the OTIS survey in Kentucky to determine approximate replacement demand. The following illustrates this calculation:

These data are now .vallable and are being incorporated into OTIS.

\\ The total work force included in the OTIS Kentucky survey was 225,293. The survey provided an overall projection of 5,200 new positions. Based on the calculated death and retirement rates shown above, the total replacement demand is about 9,000 positions. Since total demand is computed by adding OTIS expansion and replacement demand figures, the total demand is about 14,255 in Kentucky for manufacturing employers. Finally, the result of a comparison of the expansion demand reported by OTIS and the probable total demand shows that OTIS Kentucky reports only about 37 percent of the total.

(2) The OTIS Demand Survey.In Oklahoma Captures Data For Vocational Education Training Occupations Oklahoma utilizes a series of 11 forms in their demand survey. These forms were developed for particular industries and contain approximately 280 DOT codes. The occupations selected represent the areas in which vocational training is conducted, i.e., low-skill or menial occupations plus highly trained professional classifications were excluded pur- posely.Unfortunately, these limitations preclude collection of total expansion demand. The Oklahoma survey included 2,633 employers of nearly 375,000 people. Survey responses were received from 1,865 employers representing approximately 312,000 employees. An analysis of these responses indicates that over 30 percent of the work force of the respondents were employed in occupations not included on the form. As noted earlier, our resurvey substantiates this percentage. Thus, demand data cannot be assigned to these occupations and they are, therefore, exclr. from OTIS totals and the cycle reports. 0

MANPOWER SUPPLY SPBSYSTEM

This section presents ourfindings regarding the manpower supply subsystem.Specifically, the following characteristics of the subsystem are discussed: The sources of information concerning manpower supply, including the student accounting system Theinformation provided by the student accounting system

The comprehensiveness of manpower supply data

1. THE SOURCES OF MANPOWER SUPPLY INFORMATION IN OKLAHOMA AND KENTUCKY ARE SIMILAR Oklahoma collects manpow 'r supply information from five sources. Kentucky plans toobtain supply data from similar sources.In both states, the student accounting system is the primary source of data on individuals with vocational training.However, the information provided by this system differs substantially in ea ch state.

(1) Implementation Of The Manpower Supply Subsystem Has Been Approached Similarly In Each State In Oklahoma, information concerning statewide manpower supply is obtained from five sources: The Oklahoma State Department of Vocational and Technical Education is the principal source of supply data, including data concerning: High school vocational program graduates Post high' school vocational program graduates Adult vocational program graduates Manpower Development and Training Act (MDT4) graduates The Governor's Comprehensive ManpowerPlanning Staff provides supply data on the number ofgraduates of federal training programs notassociated with the State Department of Vocational and Technical Education. Bureau of Indian Affairs Concentrated Employment Program (CEP) Department of Corrections Migrant Education Work Incentive Program (WIN) OpportunitiesIridustrialization Center (OIC) Rehabilitative Services Tulsa Urban League The State Accrediting Agency providesinformation on the number of private school graduates based onlicense applica- tions required annually by state law. OESC provides information on thenumber of qualified registrants available to fill jobs. The Oklahoma Board of NurseRegistration and Nurse Educa- tion, working jointly with the StateDepartment of Vocational and Technical Education, providessupply information on the number of nurses available for employment. Information for vocational educationand MDTA program graduates is reported automatically inOklahoma through,the student accounting system, as described below. Supplydata for the last four categories are reported by special arrangementwith the responsible agency.Additionally, special studies are conducted to assessthe manpower supply for eachof the appropriate areas covered inthe OTIS cycle reports. Currently, the Kentucky systemcollects supply information primarily from state-fundedvocationalitechnical programs, from DOL apprenticeship programs, and from ESregistrants. However, future planscall for the inclusion of additional sourcesof manpower supply similar to those utilized by OTIS Oklahoma. (2) The Information Provided By The Student Accounting Systems In Oklahoma And Kentucky Differs Substantially As discussed in Chapter II, the student accounting system is the primary mechanism for collecting and processing manpower supplydata in both Oklahoma and Kentucky. Exhibit XXI, following this page, provides a comparative overview of the Oklahoma and Kentucky systems. In addition, the Oklahoma portion of the exhibit shows the interface between student accounting data and other supply sources discussed in the previous section. The student accounting system collects the following data: Enrollment data by student and course to establish a record Dropout/transfer data to update existing records Termination data to modify the existing data baFe. These data are utilized in the supply subsystem and are being utilized in the student follow-up subsystem. The scope of information required for each state system differs substantially.In addition to the collection of data for tracking enrollees, transfers, and graduates, the Kentucky student accounting system will collect a broader scope of data than the subsystem in Oklahoma. The major differences in data collection are outlined below: Oklahoma collects the following four categories of disadvantaged characteristics data: Not disadvantaged Member of a family in which the yearly income is less than $3, 000 Academically disadvantaged and needs special programs or services Both economically and academically disadvantaged EMBUS XXI BEST COPY fr.7'Lli3LE Economic Development Admienelnition

OTIS STUDENT ACCOUNTING SYST1 DATA COLLECTION PROCEDURES

Students CompleteI Postlate 01 ON LA 1.40sIA Emil mem hover , e--- tostructow rEnTollmesatlow i State Board Conaplem Mailed To Pftwarres Forma Caw Ratter Remainder of 1-40 I-- IP State Board end Genetetes , larollment Form Class Roster

tratructlat bete Resistants Vositiso Avocet Complete Enrollment form

Is Rooter Ammeter

State Board State Board M.D. Corrections Made No I Yes \---01 ROOM** Completica Cards Completion Data To Itocrotto

Addiating Scroll., Data Oatrylata Car& BIA sad Rotuma To Wpm* Madam,..110- sues Pond . WIN, CEP . old

(Private Vow:lama sod Tedsoloel Solna* Ratatiop Sunk Data Sew

Itortoployol SittlInd Wakes

Ragiatsrod Mow Licoon4 Patti* Ntaven

Enrollment Form KENTUCKY CompletedBy Any Osage aa Students sad "At' StufsStatue Tandems State loud YetI No Mt ite Out Terminetkm Fano

Data Tromso itted hninfitIotal $Eppff Data To State Dept. Contrive of Vocational Clomp Foam Appestalme Education kSRerrintems

Data Received and Reviewed By State . Dept. ofVoostiossal4 Education

Pt eta Scbool Coot itoomed ay Sanaa rapt. at Vocational Vocation

aerate Coastated Stow NV. of Vacritkoad Meadow gr

In Kentucky, 10 categories ofdisadvantaged data are collected: Not disadvantaged MO: Academic Socioeconomic Aged Correctional institute inmate Cultural or linguistic isolation Dropout potential Ole Correctional institute parolee Juvenile delinquent Migrant IND Other Oklahoma collects informationconcerning a student's aspira- tions in the occupational training area.In contrast, the Ken- tucky system' collects data about astudent's educational goals in the following categories: Obtain GET? Complete high school Continue education in an on-the-jobtraining program Enter military service Continue study in vocational, technical,trade, or business school Attend community college(two-year program) Complete college for bachelor'sdegree Study beyond bachelor'sdegree Other The Oklahoma termination cardreflects only the following information, supplied by theteacher: Complete the training programand (1) graduate, or (2) leave school withoutgraduating Complete the training programand remain in the school in other course work -49- , Complete this year in the training program and plan to continue in the program Drop out of the training program but continue inschool Transfer to another school Dropped out of school and is employed full time in the field for which he/she was training Dropout or transfer and status is not known, or is deceased Was not in program this year The Kentucky system also tabulates terminationsbut, in addition, collects a substantial degree of evaluativedata in three areas: Degree of accomplishment rated by theteachers as: Excellent Good Average, . Fair . Poor Type of termination Reason for dropping out, according tothe teacher Additionally, Kentucky plans to develop a moreextensive linkage between the student accounting and follow-upsystems, e.g., to relate evaluative data to subsequent success.TI,is capability will provide for the eventual development of processand impact evaluation strategies. OTIS OKLAHOMA SUPPLY TOTALS ARECOMPRISED PRIMARILY OF GRADUATES FROM FORMAL TRAININGPROGRAMS AND EMPLOYMENT SERVICE REGISTRANTS In any discussion of manpowersupply, job market entrants usually include individuals from the following sources:.

-50- Graduates of formal training programs such as vocational education programs and federal work/training programs Individuals formerly outside the labor force: Housewives desiring to reenter the labor force Handicapped workers Returning veterans People entering the labor force through immigrat:on or net in-migration Individuals formerly working in other occupations, i.e., entering the labor pool through nonvocational education and upward mobility An analysis of the operational supply subsystem inOTLS Oklahoma indicates that OTIS manpower supply totals are comprised principallyof graduates of formal vocational training programs and registrants withOESC. Exhibit XXII, following this page, presents the composition of manpower supplyreported in the OTIS Oklahoma Cycle Five Report by source and category.Of the 25,769 individuals, 11, 561, or nearly 45 percent, were OESC registrantsand 10,592, or over 41 percent, were graduatesof vocational education programs. The remaining 14 percent were graduates of federal training programsand private schools.

(1) Three Major Sources Of Job Market Entrants Are NotIncluded In OTIS Supply Totals A comparison of the Oklahoma sources of supplydata indicates that, with the exception of company-trained individuals, personsreceiving foe."-,. mal vocational training appear to be adequatelycovered. However, three categories of labor supply entering the work force are notconsidered directly: 1/

It This problem has been recognIze4 and procedures are being developed to collect additional supply data.

-51-

O EXHIBIT 'ow Economic DevelopmentMI5 Achnbristratite COMPOSITIONOKLAHOMA SUPPLY AND SOURCE DATA OF Percentage of Oklahoma Stateand Department Technical ofEducation Vocational Source HighPost School High SchoolVocational Vocational Program Program Graduates Graduates Supply Category 1,6015, 707 Total Supply 22.1 6.2 Total Supply AdultMDTA Vocational Graduates Program Graduates Total Voc-Ed Supply ^.. , 1,1583, 285 10, 593 12.7 4.5 41.1 StateGoverror's Accrediting Comprehensive Agency Manpower Staff OtherPrivate Federal School Programs, Graduates e. g. Total Federal Program Supply , CEP, WIN, OIC 151 2,3061,309 0.6 8.95.1 Oklahoma Employment Setiurity Coramis.,, TOTAL SUPPLY mai=25 69 561 100.0 44.9 Outside the labor force Other occupations inonvocational training Immigration Quantifying the extent of the omissions, i. e., the number of individuals not included in supply, is extremely difficult.However, the results of numerous studies indicate that the number of people entering the job market from nonvocational schools and other occupations, i.e., interoccupa- tional mobility, isconsiderable.-V For example,many workers in Okla- homa entered the labor force from institutions that are not included in the sources of supply for the current OTIS report, e. g. High school Junior high school Companies Armed forces Correspondence schools In addition, the OTIS supply component does not consider individuals entering the labor force with one to three years of college.However, recent studies in Oklahoma indicate that many of these individuals enter the job market in the area in which they received vocational training. Undoubtedly, some of the above persons omitted currently from OTIS supply data are included in the OESC registrants, which comprise approximately 50 percent of manpower supply. However, despite the relatively high Employment Service penetration rates in Oklahoma, it is likely that many workers are not included in the OESC totals.Thus, the published OTIS supply figures understate supply. The manpower supply subsystem in OTIS Kentucky is presently under development by the state professional staff. To date, little formal documentation of the supply component hasbeen developed. Some MDTA

It U.S. Department of Labor, Manpower Administration, Office of Manpower, Automation, and Training, Formal Occupational Training Adult Workers, Manpower /Automation Research Monograph No. 2, Washington, D. c.; \ U.S. Government Printing Office, 1964; and Young, Robert C. , Manpower Information For Vopatipticklilucatioli Flan:tits, The Center For Vocational And Technical Education, Ohio State University, Columbus, Ohio, November, 1969; p. 102. -52- and apprenticeship program data have been collected. Generally, subsystem design appears to be similar to that of the OTIS Oklahoma subsystem. For example, the Kentucky system designers plan to utilize a supply component based on enrollments and graduations of: Public vocational schools Technical institutes Community and junior colleges Private vocational and business schools Federal training programs In addition to these data sources, the staff plans to collect in-migration data from Employment Security offices throughout the state. The methodology for collecting, integrating, and utilizing the collected data has not yet been established for review and analysis.

(2) Manpower Supply Forecasts Are Comprised Of A Mix Of Estimated And Current Data 0 The supply forecasts contained in the OTIS Oklahoma reports are developed through the utilization of a variety of methods. Adjustments are made based on formulae developed from the student accounting system and follow -up, data. The adjustments are detailed in Appendix I. Private school graduate data are not adjusted. The previous year's graduation level is used as the forecast for the current year, i.e., no forecast is made. Federal program graduate data are not adjusted. An graduates are assumed to be job-ready. Employment Service registrants are not forecasted. Avail- ability and eligibility determinations are made by OESC. personnel based on registrant files. Thus, the OT:S supply forecast is comprised of a combination of OESC current zegistrant data and other adjusted and unadjusted supply estimates.

-53- INTERFACE, STUDENTFOLLOW-UP, COST, AND EDUCATIONAL RESOURCESSUBSYSTEMS This section presents theresults of analyses of theinterface, student follow-up, cost, andeducational resourcessubsystems, Specifically, the topics discussedinclude the: Development and use of theinterface statistic Differences in the scope of thestudent follow-up systemsin Oklahoma and Kentucky Uses and potential ofthe cost and educational resourcessubsystems WITH CAUTION 1. THE INTERFACESTATISTIC SHOULD BE USED The interface statisticis the principal outputproduced by OTIS. Exhibit Oklahoma III, following page6, displays a sample pageof output from the OTIS Cycle Five Report.The interface statistic isdetermined by comparing man- educational pro- power supplyfor each occupation(DOT code) wilain a selected gram clusterwith forecasts of manpowerdemand fox-the same occupations. for a Thus, the interfacestatistic represents the netmanpower requirements specific vocation training programcluster in a given planning areaand on a cumulative basis statewide.1/ As noted earlier, someof the OTIS forecastsof manpower demand are inaccurate and additionalsources of manpowersupply are required. However, data corn- even if manpower.demand data were accurateand manpower supply with caution. Some of the reasons .plete, the interfacestatistic should be used why this statisticcould be misleading aredescribed below.?

jj See above, pp. 6-7. The OTIS Cycle Ilve Report also notesthat the manpower supply and demandtables should be viewed dynam- ically, especially at the regional levelswhew changes in the plans of a single firm couldsignificantly influence demand in a given occupation.

'410 Manpower requirements data are aggregatedstatewide and by region. However, commutation patterns canaffect the results. For example, an excess of supply(people) for a particular occupation statewide or in a particularregion may be offset by workers commuting to another state orregion, e.g., nearly 36,000 more Kentucky residents commuteto Ohio than Ohio residents commute toKentucky.11 In- and out-migration may alter the net manpowerrequire- ment projections over the courseof the forecast period by decreasing or increasing manpower supply. The jobs included in demand are notstrictly comparable with the occupations listed in supply.For example, it is unlikely that vocational education enrollees inpolice science will be applying upon graduation for jobs as sheriffs. Not all of the graduates of manpowertraining programs are competent to perform the jobs forwhich they are trained due to improper trainingand/or attitudinal problems. Conse- quently, supply for a given occupation maybe overstated. Supply of manpower may not beavailable concurrently with demand. Therefore; Imbalancesbetween--demand and supply may occur over the courseof the forecast period. OTIS supply counts all personsavailable for the labor market. However, in Oklahoma, OTISdemand includes only demand for selected occupations. Therefore, netdemand is understated because many people in supply canqualify for jobs not included in the survey. Many trained individuals canand do qualify for jobs outside the score of their training. Forexample, many workers have multiple skills and demonstrate ahigh degree of occu- pationalmobility.' Thus, supply for a particular occupation can change with personnelshifts.

Comm. jultjX1)1__tternsof KentiCounties Kentucky Department of Commerce .?" Young, Robert C., ,MMower natimFbrional Education Pia= The Center For Vocational And Technical Education, Ohio State University,Columbus, 'Ohio, November, 1969. a FOLLOW-UP 2. THE OPERATION ANDUTILIZATION OF THE STUDENT SUBSYSTEM DIFFERSSUBSTANTIALLY IN EACHSTATE The purpose. of thestudent follow-up systemin both states is totrack pro- linkages. gram participants,thereby establishingindividual training and career im- However, the methodof conducting the follow-upsurveys varies, and, more portantly, the use of the surveysin Kentucky and Oklahomadiffers substantially. Kentucky uses a directstudent follow-up technique aspart of the student accounting system discussedpreviously. This approachconsists of a periodic former students. Theform is completed by the ter- \mailing to a percentage,of minee and returned tothe state foranalysis. Experience to dateindicates: Of the 10,806 personscontacted in the pilot study,48 percent or 5,189 hadresponded after threemailings. A follow-up of 50 studentswas made todetermine non-, respondent informationand/or bias. The collection data wereutilized to prepare reportsof the afollowing tyneq Currerwork-related activity by programarea Job-related training programby program area Frequency of the useof learned skill by program area Wages earned byvocational program service area

In contrast,Cklahoma follows upstudents by contacting theclassroom program. instructor. Questionnairesare mailed tothe former instructor of a The instructor is given afollow-up card for eachstudent. Through his orher The local knowledge, andcontacts, the teachercompletes the follow-up card. card contains a listof 10 possiblepost-training activities. Thedata produced Cycle ThreeReport.Y were used inthf development ofSupplement V to the OTIS Individual data .,".re notcollected for MDTA programstudents or special school of Vocational, and Technical students.In addition, theOklahoma State Department

V, a follovoup study ofOklahoma vocational and technical cation OTIS Cycle III Report, Supplement on, graduates and drIpouts; 1968 -1969 and 1969 - 1970, Divisionof Research, Planning, and Eva { Oklahoma State Department ofVocational and Technical Education.

-56- Education conducts a parallelfollow-up survey of students, stratW -'dby occu-- pational objectives, to validatethe teachers' follow-up information. Thus, follow-up informationobtained in Kentucky is much more comprehensive than that collected inOklahoma. More importantly, Kentuckyplans to develop a more extensive linkagebetween the student accounting and follaw-up subsystems. The purposes of thislinkage are:

t To develop indices for evaluatingthe training process in terms of the success pr failure of programgraduates To collect data to ensure therelevancy of training to the working environment encounteredby graduates ! Evaliation of the educational ?rocess requiresassessment of program impact.t. accordingly, thedata collection methodology of thestudent follow-up subsystem can be utilized directlyto formulate a comprehensiveevaluation strategy that analyzes theinherent relationships among: Student graduate success and programresources requirements Student graduate success and programcontent Student graduate success anddisadvantaged program considerations The resulting evaluationwould provide key societaldata for cost-benefit analysis and resource allocation decisions.The data provided by thesesubsystems are also necessary to support(1) placement and jobdevelopment, (2) employability development and vocationalcounseling, and (3) planning anddevelopment of training program curricula.

3. THE OBJECTIVESOF THEKENTUCKY COST AND EDUCATIONAL RESOURCES SUBSYSTEMSAPPEAR TO BE SIMILAR OTIS Kentucky conceptualmodel contains both a costsubsystem and an educational resources subsystem.Our analysis of these twocomponents indicates that theseobjectives are similar. In concept, the cost subsystem wasdesigned to produce two outputs: An analysis of program costs vs thebenefits accrued to society and students.This analysis is to be based, primarily, ondata collected by the follow-up system andfinancial data collected by the State Department of Vocationaland Technical Education. A cost-benefit report presented to the statedirector, state board, local boards, executive secretaryfor CAMPS, the Governor, and the state legislature.The basis of this report is the cost-benefit analysis. According to the OTIS Final Report,the cost subsystem was to be formulated "to help determine program mix, i. e.,the proportionate distribution of resources amongoccupational program service areas."if An educational resource inventorydata file has been started in Kentucky. Principal data include: Buildings, including schools,classrooms, shops, laboratories, and other structures for educationalinstruction and administrative purposes Equipment, including instructionalequipment (i.e., desks, machines, typewriters, car engines,shop tools, etc. ) and maintenance equipment Personnel, consisting of an inventoryof teachers, administrators, maintenancepersonnel, and support staff In addition, plans arebeing formulated to add communitycharacteristics, industrial and economic analysis,land-use, and community servicesdata.These elements are not part of theOTIS system developed forKentucky. The objective of the educational resourcessubsystem is to provide data to program plannersinvolved in making resourcereallocation decisions that reflect factual considerationsof currently available facilities,equipment, and personnel. The system willrelate requisite resources tospecific programs to be undertaken.Further, these resources canbe compared to the specific

Braden, Paul V. , et al Occupational TrainingInformation System (OTIS) Final Repeat, ResearchFoundation, Oklahoma State University, Stillwater,Oklahoma, 1970; p. 444.

-58- Subsequently, theanalyses sup- goals andobjectives of theplanned program. evaluations of programimpact ported by thissubsystem may becombined with extensive planningand evaluation. based on studentfollow-up data tofacilitate more subsystems appear tobe identical. This originalobjective and usesof these and the costsubsystems in Thus, in ourjudgment, theeducational resources thereby providing Kentucky could becombined into onecomprehensive system, that resourceallocations can be users withboth resourcesand cost data so made effectively. OVERALL SYSTEMAND SYSTEMIMPACT and its impactand replica- Our findingsregarding theoverall OTIS system following topics arediscussed. bility are presentedin this section.Specifically, the and the needs The similaritiesbetween OTISinformation needs of a typicallabor marketinformation (LMI) system state and local The extent towhich OTIS outputis utilized for planning OTIS developmentand operating costs The cost ofreplicating OTISin other states LMI SYSTEM 1. OTIS IS ASPECIAL-PURPOSE self-contained systemproviding com- As originallyconceived, OTIS was a education and manpowerplanning. prehensive informationfor state vocational and reported byOTIS However, muchof the informationgathered, processed, Employment Service.Exhibit is similar tothe informationprocessed by the needs that shouldbe met XXIII, followingthis page, lists28 broad information and comparesthese data with by an LMI systemin a largemetropolitan area incorporates onlyone-half information processedby OTIS. TheOTIS system Consequently, OTIS canbe viewed of the needsspecified for theLMI system. part of a muchlarger manpower as aspecial-purpose LMIsystem, i.e., as system.

-59- I Economic Development Administration COMPARISON OF LMI/OTI.SINFORMATION NEEDS =err =ugly EMI Information Needs Yes OTIS In Kentucky only Output Confidentiality maintained in Clidahous ?iote privileged in 2. . EstablishmentlocsiticasalEstablishment and Identifying Employment Womanize By hutustey By For Survey ollecteccLedYes Yes No KentuckycycleCould repasts be leoduced from data collected, but not peeseutly in' ' . Establisfuneszt Employment By Occupation Yes reportsNot printed in cycle schools.madeIn Kentucky, available completed at area voc-ed forms are Current status not included in cycle report 4. SpecialEstablishment Winker Potential Groups By for Occupancy Employment of Yes No notSpecial formally needs produced data collected but Critical data am available after editing 6.5. EstablishmentTotal Employment labor Denunasi Distribution By Occupation ity Yes NoYes One-year New job Projecticas Na printedreplacementOTIS provides in cycle demand. forecast reports. 'Totalof expansion employment and, an per Ok selalaoina, is not 7. OccupationCharacteristics of the "Typical job" By No No No Output of View System in Oklahoma and Kentucky 9.8. ClussacteekeistsWiest= of Labor of Supply "Worker By Customarily Occupation Hired" NoYes YesNo Projected supply No TheseToincluded be interfaceddata are in OTIS,available with e.g.demand from , View various System sources in Oklahoma but are not 10. /in the Occupation shiptIndicators ay Occupation of labor Demand/Supply Relation. No Yes occupationsNet surplus, by backatty, of Produced by interface system 11. 11 AnticipatedOccupation Short,- Tem Total Labor Demand andk-ctedTwo-year twowyeen, in Clidaborna; roles:tiro in itaznacky coloat- Yes Annualcycle projetnisas repast included in Two-year projection is halved in Cklahoma for one-year data. 44. COMM nitit2) MI irdeernation Needs Labor Demand Leantteras forecast of four Collected Procesed Yes S Only short-tern annual projection Output Oidabonia collects two- and four-leas data Kentucky Note collects 12. ByAnticipatedPsobable Occupation ChangesLonger-Tenn in CEoracteristics Total of "Typical years; Oklahoma No No included in cycle report No one-vvrtem.At andthis two-yeartime, job data.analysis is not pan of either operating OTIS 13,14. job" ByProbable Occupation *maw In Chanacteristics of 'Worker No No No Included in View System in Oidahoma and Kentu.-lo, 15. CusransarilySpecifiedCone AreaHired" in the Occupation aal Esthnates of the Population By No No No SomeSystem skniographic data collected through Student Acocunattng 16. torteCurrent By Annual Specified and Area Quarterly Extirnates of labor Partial Mental projection only Supplybasis estimates on an annual estimatesSupply orestimates, projections. however, am not based on total labor fume is conclved to pro- 17. SpecifiedIndicators Areaof Manpower Service Needs By Partial , in..Through System Student P. count. DisadvantagedZotatty, students area, and data state ptojections duceUnderdeveloped someOTIS ofOklahoma the Human indicators has Re-sources six-year Included Srierr record, in this component. 18.19. CommunityCommunity Employment Develops/am:A Treads and Economic Partial Yes toEconomic update Oklahoma trend data pao-used Yes Demand projection adjustments Economic analysis not part of Kentucky projections Outlook Yes jections See Item 12 above; long-term projectiorts are available from 20. IndustryLonger-Tenn (incitutiA Employment Assumptions) Projections By No No Wage rates are not included in providedView DOI./System by ES the utilizes Employment wage data in Service Kentucky and Oklahoma --, 21.22. CcnomaxiityTrakaing Opportunities Wage Rates By Occupation No ,, No present OTIS cycle reports. Yes as partPlans of are KITES; being includeddeveloped .-in in Kentucky View Systern in Oklahoma to prepare training list 23, Apprenticeship Opportnnities No No Included in View data in Oklahoma and Kentucky ar 24. Licenses, Credcatials, Certificates No No No Included to View data in Oklahoma and Kentucky OMIT =P) 25, Faropioymexcit-Related Supportive Services 1/41 laforniation Needs Collected No Proceared No OTIS . No Note 26. link= No No NoNo 27.28. tattoo}WogCsnranunity FacilitiesClarneb Commuting Patterns and Transpor No No No

Source: Thal -Lary, Margaret, 1Ftsndremaan and Design of a Labor Market information Sintezr, for a Lance Metropolitan Area, liniveity of California, Berkeley, November, 1972. 2. OTIS DATA ARE UTILIZED IN THE PLANNING PROCESSBY BOTH OKLAHOMA AND KENTUCKY The principal objective of OTIS is to providethe regional and state planning and operating information necessary toformulate educational, manpower, and economic development plans andpolicies. Our analysis indicates that OTIS data are utilized in the planning processby both Oklahoma and Kentucky.

(1) OTIS Output Influences Directly The OklahomaState Vocational Education Plan Prior to the development of OTIS, stateofficials believed that adequate planning data were notavailable and/or that existing data were:

Aggregated only at the nationaland state levels Not easily disaggregated Limited to certain uses because ofconfidentiality constraints More importantly, local level occupationaldata were not expected to be available soon. Thus, OTIS was developed toproduce regional aggregations of demand and supply data byoccupation and to incorporate these data into a comprehensive planning process.Therefore, OTIS has been widely accepted because it gatheredlocal data and produced local regional aggregations of supply and demand oy occupation.Additionally, in the judgment of users,OTIS produces astraightforward set of data. The data produced by OTIS areutilized in the formulation of the Oklahoma Vocational Education Plan.For example, provisions of the plan require that: "In allocating funds among local educationalagencies, the State Board shall give due consideration toinformation regarding current and projected manpowerneeds and job opportunities on the local, state, and nationallevels. "If "The State Board shall give particular considerationto those local educational agencies whoseproposed vocational education programs are best designed to:

Oklahoma State Plan Fulfill current or projected manpower needs inexisting occupations at the local level by preparingstudents, for current or projected job opportunities insuch occupations, or 4 Fulfill new and emerging manpowerneeds at the local, state, and national levels by preparingstudents for new and emerging job opportunities at suchlevels. "1-/ More imp tantly, theDivision of Vocational and Technical Education has been directed to establish only those new programsthat are able to demonstrate a high level ofemployment demand. Consequently, the Division has developed a planning processincorporating this mandate. Exhibit X. following this page, presents an overviewof the Okla- homa vocational education planning process.A formula has been developed to weight the various criteriaconsidered in funding new programs. The criteria and weights comprisingthe formula are: Manpower needs--weight 5 Vocational education needs--weight5 Relative ability to pay--weight 3 Excess costsweight 2 The weights are based on dataobtained from the student accounting system. However, the emphasis on manpowerdemand is clear and these data are obtained from OTIS.

(2) KentuckIs Incor oratin OTIS DataInto The Plannin Process Kentucky also plans to integrateOTIS data into the state and local planning process. To date, OTIShas had limited impact on the stateedu- cational planning system becausethe system is new and interface output has not been produced.However, the results of our interviewsindicate that Kentucky plans to usedata produced by OTIS forapplications similar to those in Oklahoma.Currently, plans are beingdeveloped to build on

Oklahoma State Plan

-61- EXHIBIT XXIV OKLAHOMA VOC-ED PLANNINGEcceornic Development PROCESS Administration individual Nee& Needs Being Met Goals, Objectives, Alternatives, Recommended Awareness Manpower Needs -=.116 and Priorities A Master Plan -10 1 I Master Plan DecisionMaker1 Evaluation implementation the experience of the first year of operations and to implement a series of steps to refine the OTIS data collection methodology. These steps include: Conduct a census survey of three seginents of nonmanufacturing employment: Mining Contract construction Transportation Q Analyze and adjust the student data system from the standpoint of feasibility.Determine system adjustments and identify users and user needs. Define sources of supply data and develop procedures for obtaining supply information. Review the former student follow-up system to identify procedural changes.In addition, identify user needs and develop rep-. cling formats. Prepare and initiate procedures for updating manpower supply and demand components of the system. Analyze pilot study results to determine further steps in the implementation process. . . Develop comprehensive lists of private and public training programs within the state. 1 3. OTIS OKLAHOMA AND OTIS KENTUCKY EACH HAVEUNIQUE COST CHARACTERISTICS Exhibit XXV, following thir,, page, presents the separatecosts for developing and operating OTIS in Oklahoma and Kentucky: Development costs include expenditures for systemdesign, development, programming, and test activities. Operating costs include the normal expenses incurred in collecting supply and demand data, producing anddistributing the interface statistics, and utilizing the other subsystems for planning and resource allocation. ti xxv

Econo Deus pleat Administration BLSI CO ?Y OTIS DEVELOPMENT AND OPERATING COSTS IN OKLAHOMA AND KENTUCKY

Oklahoma Kentucky

I.DEVELOPMENT COSTS

(1) Demand Subsystem $ 101,997 014* (2) Supply Subsystem $ 203,500 (3) Interface Subsystem PS MI (4) Fallow -Up Subsystem 22,694 (5) Educational Resources Subsystem %SO WOO

Total Iievelorsaaent Costs 203 500 Cleamiga

2.ANNUAL OPERATING COSTS $ 110,356 (1) Demand Subsystem $18,938 (2) Supply Subsystem 9,764 51,890 (3) Interface Subsystem 2,500 (4) Follow-Up Subsystem MOP 180 (5) Educational Resources Subsystem 111MMI.P1 + Total Operating Costs 31 2022Q2 luau Et

if Costs of the Follow-Up Subsystem areincluded in the Supply Subsystem.

Source: The source and derivation of the Oklahoma andKentucky costs are contained in Appendices 8 and C, respectively. As expected in a pioneering effort, OTIS development costs are higher in Okla- homa than in Kentucky.Conversely, annual operating costs are higher in Ken- tucky because of the demand data collection methodology utilized.The composition of these costs in both states is detailed below.

(1) The Costs Of Developing And Operating OTIS Oklahoma Are Approximately $200, 000 and $31, 000, Respectively The costs of developing and operating OTIS in Oklahoma are presented in Exhibit XXV. The development costs were incurred over several years; the operating costs are based on Fiscal Years (FY's) 1972-1975. The estimated development costs of approximately $200, 000 are comprised of: A grant of $153, 500 d-1 the Oklahoma State University Research Foundation from var4,us federal and state agencies for system design and development Computer time and other related expenses of approximately $50, 000 The annual cost of operating OTIS in Oklahoma is approximately $31, 000. These operating costs include: $18,938 for the .demand subsystem $ 9,764 for the supply and follow-up subsystems $ 2, 500 for the interface subsystem The relatively low annual operating costs reflect the fact that in Oklahoma benchmark demand data are collected through a sample every four years. The calculation of the annual operating cost of the demand subsystem is detailed in Appendix B. Essentially, this cost was derived asfollows: The total costs of the OESC contract with the Oklahoma Depart- ment of Vocational Education and Office of Community Affairs and Planning (OCAP) were $120, 000. The total costs of the OESC baseline survey data collection effort were approximately $102, 500. The OTIS share of this expenditure wasapproximately $58.300. The remaining costs areattributed to OCAP. The update costs in FY 1973,1974, and 1975. are estimated to be approximately $7, 450,$4, 980, and $4, 980, respectively. The total baseline and update costsof approximately $75, 750 are divided byfour to obtain annual operating ,costsof $18. 938. The allocation of the annual costsof the supply and follow-up subsystems is also detailed inAppendix B. Specifically, these costs are aportion of the total annual costs ofoperating the student accountingsystem.

(2) Develoyment And OperatingCosts of OTIS Kentucky Are Approximately $125, 000 And$185, 000, Respectivel An accurate determinationof the true costs of the OTISmodel in Kentucky is difficult becauseall costs are recorded inline-item budgets. Thus, the allocationof expenditures to particularactivities requires reconstruction of work loads,assignments, and supportive expenses.The reconstruction and allocationsutilized herein were developed inconjunc- tion with key officials inKentucky.All of the assumptions andcalculations are detailed inAppendix C. As displayed in Exhibit XXV,following page 62, it has costapproximately $125, 000 todexielop the OTIS model in Kentucky.This figure cis comprised of approximately$102, 000 for the demand systemand $22, 700 for the follow-up system.The' demand subsystem developmentcosts include: $68, 697 for contractual andconsulting services for system design and development An estimated $14, 500 for computerprogram development S An estimated $18, 800 inEDA technical assistance support The costs for developingthe follow-up subsystemrepresent only Kentucky's share of an EDA contractwith Ohio State Universityfor the development of a system for Kentuckyand West Virginia.

-64- The current operating costs for OTIS. Kentucky are approximately $185, 500 and include: $110, 356 fur the demand subsystem $ 51,890 for the supply subsystem $ 23,180 for the education resources subsystem The demand costs are comprised principallyof personnel costs and travel expenses. The costs of the fieldstaff alone were over$70, 000, including over $50, 000 for thesalaries of field surveyors. The salaries of field surveyors were includedbecause, in our judgment, they represent a valid demand data collection expenditure.Obviously, if these expenditures were not included, total operating, costswould be substantially less. With the exception of approximately $10,000 inmaterials for the supply subsystem, personnel expenditures comprise the costsof the supply and education resources subsystems. Determining annual operating costs inKentucky is difficult because much of OTIS still is not fullyoperational.In our judgment, the annual costs of collecting demand datawill remain at current levels because of the census approach to demanddata collection or increase due to thelarger number of nonmanufacturing employersto be surveyed.This assumption is reasonable and is inaccordance with the assumptions of keypersonnel subsystems are implemented fully, involved with OTIS. When the remaining 0' operatitig costs will probably equal orexceed current expenditures. Con- sequently, the annual operating costsof OTIS in Kentucky are estimated to be approximately $185, 000.

4. THE COSTS OF11.EPOCA1ANIG THE KENTUCKY DEMAND DATA COLLECTION SYSTEM ARE EXCESSIVE The fundamental distinctionbetween the Oklahoma and Kentuckydemand data collection methods isthat Kentucky conducts a census and0 lahoma, a >.. sample.In addition, Kentucky utilizespersonal interviews excl stvely and Oklahoma contacts large employerspersonally and small employers by mail and telephone.In result: -65- ,..tiketz,e

Oklahoma attempted to obtain data from a sample of 2,636 employers from all sectors of the state's economy at a field data collection cost of $16, 000, or approximately $6 per employer. Kentucky attempted to obtain data from a census of 2,894 employers in the mamifacturing_sector at a field_data-collec- don cost of approximately $80, 000, or approximately $28 per employer. Exhibit XXVI, following this page, lists the number of employers and total reported employment in Kentucky by major industry group.This exhibit also presents the estimated cost of conducting a censusof Kentucky employers. The total field cost of demand datacollection is $1.2 million, assuming a cost per firm of $25.00.Retail trade employers comprise 33 percent of the number of employers and only 20 percent of totalemployment.If these smaller firms were excluded or surveyedby mail, demand data collection costs for the remaining firms still would be over $800, 000.Conversely, a 10 percent sample similar to that used in Oklahoma would costfrom approximately $29, 000 to $120, 000, depending on the extent of the personal interviews.Thus, thepadditional costs of conducting a personal interview censuscould be more than $1 million. Exhibit XXVII, following Exhibit XXVI, presents theestimated cost of replicating the Kentucky and Oklahoma demanddata collection methodologies in other states. The cost of utilizing a census ranges from slightly over$200, 000 in Nevada to over $8 million in New York. The cost of using a 10 percent sample similar to the sample used in Oklahoma ranges from $4, 890 to nearly$200, 000. The additional cost of conducting apersonal interview census ranges from nearly $200, 000 in Nevada to over$7.8 million in New York. Thus, in our judgment, the cost of utilizing the KentuckyOTIS demand data collection approach is excessive.

7'r

CJ -66-

he EXHIBIT XXVI

BEST COPY AV(,:i_171..E Economic Development Administration

OTIS KENTUCKY DEMAND DATA COLLECTION COST ESTIMATES

Estimated Data Collection Costs Total- Reported --Number of ---Cen Siirrar -KW-Salm le IMI- Major Industry Group Employment Reporting Units $25/Unit *25(Unit $6/ Unit

Agriculture 2,613 342 $ 8, 550 $ 850 $ 204

Contract Construction 39,716 4,460 111,500 11,150 2,676

Mining 29,472 1,092 27,300 2,730 654

Manufacturing 234,508 2,214 55,350 5, 535 I, 326

Transportation 40,354 2,108 52,700 5,270 1,266

Wholesale Trade 46,699 3,751 93,775 9,378 2,250

Retail Trade 138,944 16,305 407,625 40,763 9,786

Financial, Insurance, and 35,641 3,930 98,250 9,825 2,358 Real Estate

Services 123,466 13,142 328,550 32,855 7,884

Unclassified 6, 763 1,,,465 3,6 5253,663 882 1111 ....._....._. ----- 698 176 48 809 $1,220,225 29 286 Total niimmlom -.am. Waal

Total State Employment 928,000

Percentage Reported 75.28 EICHIBIT XXV71 OTISEconomic REPLICATION Development COST Administration ESTIMATES t Data Collection Method Florida Pennsylvania Tennessee State Nevac':. New York Kentucky -v Census NumberField Dataof Reporting Collection Units Costs p $25.00 /unit $ 3,235,350 129,414 $ 4,239,650 169, 586 $ 1,608,175 64, 327 $ 203,650 8,146 $ 8,155,075 326,203 Oklahoma -- Sample Number of Reporting Units p 10% 12,941 16,959 6,433 815 32,620 Field Data Collection Costs p $6.0f.1 Cost Difference /Unit $$ 3,157,704 77,646 $$ 4,137,896 101,754 $ 1,569,577 38,598 $$ 198,760 4,890 $$ 7,959, 355 195,720 .t rrc-r fr7AVAR.P.111

This chapter has presented our findingsregarding the operational effec- tivE.4ess end overall impact of OTIS.In general, the systems in both Oklahoma and Kentucky have widespread support and areproviding data that are utilized in the planning processes. However,several major problems persist.In our --judgment,- the employerforecasts-of-manpower-demand-are-unreliable. If in addition, the personal interview censusutilized in Kentucky to collect current and forecast demand is not cost effective.Equally important, OTIS manpower supply does not include all informallytrained individuals. Our recommendations for overcoming these problems arepresented in the next chapter.

J These remits are neither new nor surprising. Many experts in thefield believe employer-based forecasts are not reliable.In fact, as a result of numerous studies, DOL has dropped a similar program. See:(1) John Fletcher Wellimyer Associates, An Appraisal Of Area Skill Surveys In BattleCreek, Michigan. And Trenton, New _Tersex., November, 1965, Washington, D.C.; (2) Chernick, Jack,Manpower Forecasting...Through The Occupational Needs Survey, Institute of Management and Labor Relations, Rutgers--The State University; (3) Hartle, Douglas C., Canadian ErnploYer Forecast Survey,Canadian Studies In &comics, University of Toronto Press, 1962; (4) Mater, Collette EL , An Evaluation Of AreaSkill Surveys As A Basis For Manyower Policies, an unpublished doctoral dime:tat:1cm at the University of WISCOZSin1971; (5) Labor Market Infer. irLativitgmaz...... _AuldThe Federal -State ment Service System, Report by the Advisory Committee on Research to the U.S Employment Service, U.S. Departmentof Labor, Washington, D.C.; and (6) Kidder. David F., Review , . is Research On over *me- : For VocationalEducation The Center For Voca- tional and Technical Education, The Ohio State University, Columbus, Ohio,February, 1972.

-67- IV. RECOMMENDATIONS

-3.- The purpose of this study was to analyzeOTIS in Kentucky, determine its strengths and weaknesses, and recommend systemimprovements. Since OTIS Kentucky is still under development, the ongoingOTIS system in Oklahoma also was examined.System strengths and weaknesses werePdescribed inthe succeeding chapter. Our recommendationsfor improving OTIS are presented below.

1. THE DEVELOPMENT AND TESTING OFOTIS KENTUCKY SHOULD BE CONTINUED The basic purpose of OTIS is to provide manpowerand vocational education planners with state and regionaldata. OTIS Kentucky is still underdevelopment. To date, the system hasgained widespread support and preliminary data are being utilized in the planning process.In our judgment, the type of valid need and we believe the develop- ,. information provided by OTIS meets a mental effort should be continued. Kentucky plans to continue OTIS operationsand is willing to commit state funds for further system development.In addition, the state has requested further funds from EDA. Werecommend that these additional monies begranted. We also recommend that technicalassistance be provided by BLS.Collectively, these federal ana state resourcesshould be utilized to make the followingimprove- ments. Implement a new employment datacollection methodology by expanding the Occupational EmploymentStatistics (OES) program. Implement and test a new approach toforecasting manpower demand, using the industry-occupationalmatrix technique developed by BLS.

-68- . -1;;.c..s?.--. -

Improve the estimates of manpower supplyby obtaining additional long-term information concerningoccupational and geographic mobility. Increase the rapport between employersand vocational education personnel by discussing with eachemployer his particular needs. Each of these recommendations is discussedin detail below.

2. THE OES SURVEY SHOULD BE UTILIZEDTO COLLECT CURRENT EMPLOYMENT DATA In 1970, BLS, the ManpowerAdministration (MA), and a number of state employment agencies joined to initiatethe OES survey.(The development, objectives, and current status of this surveyand related BLS/MA programs are described in Appendix J.) Kentucky was not oneof the initial.states participating in this program. However, theOES survey is now being implementedin Ken- tueky.In our judgment, this programspould be used to collect currentemploy- ment data for use in OTIS. The OES survey was selected for thefollowing reasons. Collecting manpower data is an integralpart of the ES function. ES personnel are expert in thiRfield. Manpower data are the responsibilityof DOL and affilirf.ted state agencies under the 1968amendments to theVocational Education Act. of 1963. The survey utilizes a statistical samplerather than a ?ensus. The adoption of the OES survey willeliminate redundant OTIS data collection activities. The survey is compatible with thesystems in use in other states and with the BLSindustry-occupational matrix. The OES survey is one of threecomponents of the OES programand is designed to produce estimatesof occupational employment byindustry for the

-69- nonagriculture wage and salary work force.Specifically, the OES survey has the following objectives. Develop occupational employment estimates by industry to provide-the-basisfor producing a-time series of estimates of employment by occupation Provide data about the occupational composition of industries, including: Differences in occupational composition within an industry, e. g. , by size of establishment, process Extent of inter-establishment variability Changes in occupational composition in response to such external factors as changing technology Provide occupational information in a form that permits the development and improvement of industry-occupational matrices at the state and area levels and, ultimately, at the nationallevel In our judgment, the OES survey can meet theinformation needs of OTIS users in Kentucky if certainrefinements are made. Our recommendations for specific refinements in the survey parameters aredescribed below.

(1) Jointly Develop A Sampling ProcedureThat Will Permit Compilation Of Reliable, Local-Level Data As noted above, OTIS data are utilized in theeducational planning process in both Oklahomaand Kentucky.In addition, information from OTIS has helped to attract new industry in both states.Clearly, job pros-,. pects and trends data are essentialfor: School and vocational /technical education administrators Manpower program administrators Economic development officials ES and vocational counselors Equally important, disaggregated data arerequired.However, sample size, data collection problems, and costs increasewith the number of substate areas involved. Thus, the increased cost and complexityof an expended sample must be balanced against the need for improveddata. Currently, OTIS collects data from a census of employers,but this method is not cost effective.It is our understanding that the ES in Kentucky plans to select a sample to permit data compilation for four principal regions. This decision should be reviewed carefully with OTIS users, especially the Bureau ofVocational and Technical Education.In our judgment, alternative sampledesigns and concomitant data reliability should be developed. For example, the reliability and costs ofcollecting four regional data samples should be compared with thereliability and cost involved in collecting data from the15 regions utilized in the OTIS survey.In this respect, the existing OTIS data bank, which contains1972 employment by occupation from the census of manufacturingemployers, might be useful in developing sampling alternatives.This information then can be utilized to make a valid judgmentregarding the most appropriate sampling methodology.Perhaps combining certain OTIS regions to limit the level of data aggregation andreduce cost will be warranted.

(2) Complete An Entire OES Survey As Soon AsPossible The standard OES survey consists of three :separateannual surveys. Manufacturing employers Nonmanufacturing employers Trade employers Each group is surveyed every three years,using the standard forms developed by BLS for the industries in each group.Existing plans provide for the conduct of the manufacturing survey inthe summer of 1974. Thus, nonmanufacturers and trade employers will becontacted in 1975 and 1976, respectively. We recommend that this schedule beshortened. Given the immedi- acy of informationneeds and the necessity of testing and validating astate

-71- industry-occupational matrix, we believe all three major categoriesof employers should be surveyed this year. More importantly, we recommend that firms from all three groups be surveyedannually for the initial three-year cycle.This will provide current data and an immediate time series to develop the BLS "shift" matrix described below. At theend of the three-year cycle, data and forecasts should be available toevaluate properly the feasibility and impact of switching to the regularthree-year survey cycle. MA estimates that the first round of any portion of the OES survey in Kentucky, such as manufacturing, would costapproximately $48, OWL This includes the costs of developing a current samplingframe stratified by SIC code and size of establishment and selecting astratified random sample. Thus, in our judgment, the cost of conducting themodified OES program recommended abovestill will be less than the existing cost of collecting OTIS data. More importantly, in the long run,the OES survey will be even more cost effective. For example, it isprobable that once the program is fully operational the regular three-yearcycle can be utilized. In addition, smaller samples will be utilizedin future surveys, since industries which do not exhibit significant changes intheir occupational structure can be sampled lessfrequently and those which have similar occupational structures throughout the nation need besampled in only one or two labor market areas.

(3) Utilize The Survey Data To DevelopIndustry-Occupational Matrices For The State The OES survey data should be utilizedimmediately to develop regional industry-occupational matrices forKentucky. The process that has been used to prepare occupational employmentestimates from the sample data involves four basic steps.

-72- The reported occupational employmentfigure for an establishment is multiplied by theestablishment's sample weight. . The weighted occupational employmentfigure for the establishment is summed with similar data forall other sample establishments in the same industry size class.(cell). The sum of the weighted occupationalemployment data is' divided by the sum of the weighted totalemployment for all reporting establishments in. the cell.This produced the occupational ratio for the cell. The ratio is multiplied by the totalindustry employment figure for the cell, as determined from ES 202data, to yield the estimated number of workers in each occupation inthe cell. This process can be expressed by theformula below.

(E Pi M Ie. w. 1 1 where: Pc Estimate for occupation P in an industrysize cell

Pi Reported employment for occupationP by number of establishments in an industry sizecell The ES 202 benchmark employmentfor the industry size cell

e. = Reported total employment in the ithestablishment in an industry size cell

w. The reciprocal of the sample weight orthe sampling 1 ratio or the ith establishment in anindustry size cell The occupational estimatesproduced for each cell can beaggregated at various levels to producedata on the occupational compositionof the wage and salary workforce at different levelsof industry detail, e.g., three-digit or two -digit SIC codeindustries.Subsequently, these regional industry - occupational matrices canbe developed and utilized in conjunction

-73- demand by with the BLS forecastingmethodology to project manpower occupation. SHOULD UTILIZETHE MANPOWERDEMAND 3. OTIS KENTUCKY BY BLS FORECASTINGMETHODOLOGY DEVELOPED The results of ouranalysis of employerforecasts and of the regional occu- N forecasts were inac- pational summariesin Kentuckyindicate that many of the curate. Moreimportantly, the annual costof conducting apersonal interview conducting a sample. Con- census isinordinately high comparedto the cost of methodology should sequently, in ourjudgment, the BLSdemand data forecasting be substituted forthe employer forecastfor the following reasons. The BLS program isthe result of yearsof research conducted by acknowledged expertsin manpower demandforecasting. The BLS methodologyconsists of a rigorousanalytical framework. Thus, exogenousvariables affecting manpowerdemand can beaccommodated, e.g., thepresent energy eriplis, mone- tary and fiscalconstraints, and wageand price controls. Forecasting manpowerdemand is an integral partof the overall function of ES. inaugurated The National/State Industry-OccupationMatrix system was with the MA by BLS in 1972.The overall system wasdeveloped in conjunction information at and state employmentsecurity agencies todevelop occupational the state and regionallevels. And Industry- (1) Utilize The KentuckyInterirLo'ections Occupation Matrix ToDevelop ImmediateForecasts projections for Ken- BLS has producedinterim industry-occupation and tucky and a 1970 matrix.This matrix is similarin format, concept, data base to the BLSnational matrix and isbased on a specialtabulation as of industry byoccupation by class ofworker from the 1970 census, provide: well as otherindustry data. The purposeof this system is to a occupations in over A state employmentmatrix for over 400 200 industries that canbe utilized to estimateoccupational requirements with greateraccuracy thannational patterns alone A flexible, multipurposecomputer system toallow for matrix updating, e.g., tosupplement the matrixwith data from OES survey and preparesubstate estimates rates This system alsoincludes occupation-specificdeath and retirement demand. for Kentucky that canbe utilized to estimatetotal occupational the first year, Since the OESmatrix-based projectionswill not be available we believe this interimmatrix should beutilized to develop manpower demand projections. Annual Updates Of (2) Utilize The StateMatrices To Produce Manpower Demand As noted earlier, two yearsof OES data are necessaryto determine second year can shifts in industry-occupationratios.Projections for the In be made using thematrices developedspecifically for substate areas. obtained addition, the actualmanufacturing employmentfigures for 1972 matrices by OTIS might beuseful for matrixdevelopment.In turn, these projections to can be utilizedin conjunction withindustry employment develop forecasts of manpowerdemand by occupation.The state ES should. data determine appropriateforecast periods inconsultation with major For example, users and preparetotal employmentprojections by industry. 3-, 5-, and10-year projectionswith annual updatesmight be feasible. Subsequently, the accuracyof these state andregional forecastsshould be evaluated. IMPROVED 4. THE ESTIMATESOF MANPOWERSUPPLY SHOULD BE because The OTIS estimatesof manpower supply arepresently understated from nonvocatIonai a substantialnumber of individualsentering the job market can schools are notincluded.In addition, migrationand commutation patterns

-75- have a significant impact on supply in anyparticular area. Consequently, additional research on occupational andgeographic mobility should be undertaken.

(1) Adjust The Supply Figures Informally Trained Individuals The principal deficiency in the OTISsupply totals is that informally trained individ..zals are not included, e.g., personsfrom high schools or individuals who have become proficient throughinformal training or experience on the job. The number ofinformally trained individuals is extremely difficult to quantify particularly at theoccupation level.However, because net manpower requirements canbe Substantially affected by an understatement of supply, some effort mustbe devoted to estimating the extentof the understatement. There is a paucity of data in this area.BLS, in an attempt to alleviate this problem, iscurrently conducting two studies in this area. An analysis of the sources of trainingbased on 1970 census data An examination of trainingprovided by employers based on the 1970 census of manufacturers Both of these studies may yieldvaluable information concerning theinci- dence of informally trainedindividuals in the work force. Someof the data also might be relevant atthe state level or suggest similarresearch using state data.In addition, the expanded scopeof the follow-up subsystem discussed below willprovide additional information onwhich to base expansion of the OTIS supplyfigures.

(2) Identify The Extent Of GeographicMobility As noted earlier, labormobility can have a significant impact on manpower supply in a particular area and, hence, onnet manpower requirements.In the judgment of knowledgeableofficials, current net migration in Kentucky involvesapproximately 1,500 individuals per year.Thus, the impact probably is notsignificant.However, intra- and interstate commutation does involve asubstantial proportion of the workforce' inKentucky.lf It probably is not feasible toquantify commutingpatterns4 occupation. However, two approaches thatcould be taken to provideadditional information to be used in conjunctionwith the regional OTIS netrequirements data are: Intrastate mutin --The county commutationpatterns could be utilized to develop approximatecommuting patterns for the regions selected for OTIS.These figures thencould be utilized in conjunction with the net manpowerdata to make planning decisions. Interstate Commuting - -Demand and supply estimatesfor the adjacent states should be compiled sothat judgments can be made utilizing data 'L'or theentire interstate area.

In addition, theexpanded follow-up systemdiscussed below will yield data on commuting patterns.

5. ADDITIONAL FOLLOW-UP STUDIES SHOULD BECONDUCTED Currently, follow-up inKentucky isbased on information from the student accounting system. All termineesfrom state vocational-technicalschools and area vocationaleducation centers are sentquestionnaires shortly after graduation. The follow -upincludes information on thepost-schooling experience of _ terminees by occupationand relationship to training programs,in- .me, geo- graphit mobility, andjob satisfaction.In addition, a sample ofterminees is 'interviewed personally todetermine if non-responsebias exists. These follow- upProcedures have two principalshortcomings. 4 The period shortly aftergraduation tdnd's tobeta 'transitional period and may not accuratelyrepresent long-range trends. Only terminees of occupationaltraining programs are included.

I/ Commuting Patterns Of KentuckyCounties, Kentucky Departmentof Commerce, Frankfort, Kentucky. In our judgment, additionalfollow-up studies should be conducted two tofive years after graduation.Further, longitudinal data could be gatheredby following the same terminees over one or more years.More importantly, terminees from ncn-occupational training programsshould be included, e. g. , from high sch s and communitycolleges.The results of'hese studies should provide which to assess the long-term impactof vocational training and to determine the occupational and geographic mobility of graduates fromall types oi schools in Kentucky. 6. THE QUALITATIVE ASPECTS OFVOCATIONAL EDUCATION SHOULD BE EMPHASIZED TO INCREASEEMPLOYER RAPPORT One of the important intangiblebenefits of OTIS is the rapportObtained with employers. However, asnoted earlier, rapport was not establishedwith some employers for avariety of reasons.In our judgment, the level of rapport can be increasedby separating data collection from otheractivities and by emphasizing the qualitative aspects ofvocational education. For example, under current procedures, rapport issought by asking the employer for currentand forecast employment by occupationand by emphasizing the benefits ofOTIS. This is done within the tight time nr_nstiaints necessary to meet datacollection requirements. Consequently, schedulingand conducting interviews are difficultfor both vocational education personnel andemployers. It Using the OES/BLS demanddata collection and forecastingmethodology, the industrial coordinatorswill receive the OTIS forecasts fortheir region. Sub- sequently, as part of their normalfunctions, the coordinators can discussthese More impor- occupational projections and relatedinformation with employers.. tantly, they can take the time to discuss thequalitative aspects of vocational education, e. g. , the characteristicsof a typical job and/or worker. Job characteristics includeduties, working conditions, hours, wages, shifts, fringebenefits, and opportunities foradvance- ment Employee characteristics includeskills, attitudes, physical requirements, educational andtraining requirements, and work experience -78- In addition, current vacanciesfor placement purposes, potentialtechnological changes, such as the introductionof new machines or products, and theemployer's recent experience with graduatesshould be discussed.In our judgment, bringing a forecast to the employer,obtaining his ideas on its validity,and discussing how vocational education can meethis particular needs, will result in ahigher level of rapport than exists currently.

-79- APPENDIX A

CONCEPTUAL DIAGRAMOF MANAGEMENTINFORMATION SYSTEM FORVOCATIONAL EDUCATION

ti APPENDIX A(1) CONCEPTUAL DIAGRAM OF MANAGEMENT SYSTEM FOR VOCATIONALEconomic Development EDUCATION Administration INFORMATION DEMAND I Target Population ( input) Training Pr-)Tams SUPPL Y (or 3Ce3S) ManpowerEmployable (output)Trained Jobs 1 II ConsiderationsPublic Policy gliew...ewxemiallpo Management a el11111111111111111111111111IIIMMIIIIIIIIM1111111111111111111.2 MANAGEMENT 11K FORMATION SYSTEM 11111111111111111111111111111111111111111111111111111111111111111111t1 PLANNING Dots Collection EVALUATION Net Manpower Requirements Data Analysis Resource ManpowerMaroovw Otrilind . ConProgram AnAysts Ettecttreoen All. 00111..NM Allocation 1 CAstsPtoarn ;Es SiurtentStudent ProgramChore teratm Prererern mym.o. MN....Imo Strategies UoetrdtRetoutv-. A.o..1141.Plar Resources It wtoey Pon Se1001-11 Ptinkty i =..ome r. =vs4WDam iII11MMI1111111.1111111111111 Special Reports 111111111111111111111111111111115 M111111111111111111111111111111111111111111111111111111111111Mla StaitWA!Erivloysn Aietots Districts ReportsAnnual 4 NI11111101110.1111110116 0E1 Go Roma 411.4 I 1/41 ' SOURCE: Ohio State Uitivenity, Cohen 4. I *). " The Center for Vocational and Technical Education, APPENDIX B

SOURCES AND DERIVATION OF OKLAHOMA OTIS COST ESTIMATES APPENDIX )3(1)

SOURCES AND DERIVATION OFOKLAHOMA OTIS COSTESTIMATES

This appendix presents theestimated cost of developing andoperating OTIS in Oklahoma. The sources,assumptions, and derivation ofthese costs are presented in thefollowing five exhibits: OTIS Oklahoma Development andOperating Costs Cost of OESC Demand DataCollection Calculation of OTIS Demand DataCollection Costs for Fiscal Year 1972 Calculation of the Supply andFollow-Up Subsystem Costs Calculation of OTIS AnnualOperating Costs EXHIBIT B-I

Economic DevelopmentAdMinictration

OTIS OKLAHOMA DEVELOPMENT AND OPERATING COSTS

1. DEVELOPMENT COSTS

(1) Demand Subsystem (2) Supply Subsystem (3) Interface Subsystem (4) Follow -Up Subsystem

Total Develuprnent Costs

2. OPERATING COSTS

( 1) Demand Subsystem $ 58,326" (2) Supply Subsystem 9,7643/ (3) Interface Subsystem 2,500 3/ (4) Follow-Up Subsystem

Total Operating Costs c5=290

Oklahoma State University Research Notes: 11 This figure was obtained from Dr. Warren Edmiscar, Direttor, Foundation. The amount is comprised of a grant toOklahoma State University for $153, 500 from various federal and state agencies and $50,000 for computertime and other related expenses. Funds were provided by:

DOL--$99, 390 Ozarks Regional Commission--$40, 000 Oklahoma Industrial Development and Park Department-434,863

The remainder of the $203, SOO was comprised of staffrelease time from the State Department of Vocational and Technical Education, OESC:, andPrivate Schools Association.

2/ This figure is the OTIS share of the total OESC demanddata collection costs of $102, 574. The OESC data collection costs are detailed in Exhibit Tice derivation of the OTIS share is explained In Exhibit B-III.The calculation of annnai oterating costs is explained inExhibit B-V.

3/ Student Accounting System costs are $30,112, as detailedin Exhibit B-N. Supply subsystem and follow-up subsystem costs are estimated to total $9,764. " , rr,..,`:::,:.-r-:,1,.-r, t .1c.r..- :4.1.,:.':-%:::',1:',.::".-.--.,,A.:,. ,..., , ..... ;;.1*

EXHIBIT R-II

Economic Development Administration

COST OF OESC DEMAND DATA COLLECTION BEST COPY AVAILABLE

Fiscal Year 1972 1973 1974 1975

1/ Persta3a1 Services $ 63,867.02 $ 5, 125. 84 $ 11

$ 1,391.39 ! ExPenses: Benefits $17,261.49 Supplies 975.29 86.81 Communications 601.26 121.84 Trawl 10,253.47 21.01 1/ 1/ Equipment 5, 542. 76 " 151.83 Premises 3,605.41 452.93 Services, Other 467. 78 105.78

Subtotal Expenses $38,707.46 $ 2, 331. 59

Total Expenditures g 102, 574. 48 c 9p4, 4.984,19.1/

Cumulative Expenditures 10274.48 14S4lial $ 120,000,00,

Sources: State Department of Vocational $40,000. 00 $ 1111. Mb and Technical Education -e Office of Community Affairsk\_ 20, 000. 00 ...... and Planning V

Oklahoma Employment Security 42, 574. 48 7,457.13 4,984.20 4,984.19 Commission

Total , 102, 574.48 $ 7,457. 11 ,$ 4.984.p $ 4,984,19

Source: Mr. Wayne Edge, Comptroller, Oklahoma State Employment Security Commission

Notes: IfUnexpended balance for coming fiscal yeas 2/For a description of this project, see Exhibit Note

1 EXHIBIT B**III(1)

Economic Development AdrninistTation

CALCULATION OF OTIS DEMAND DATA COLLECTION COSTS FOR FISCAL YEAR 1972,

I. Total FY '72 Expenditures: $102,574.48

II. Total Employers: Units Surveyed Personal Visits Responses

OTIS 2,636 1,087 1,865 OCAP* 408 4 111 2 495 2 962 ash= nrinegsat gehapt=

Percentage of OTIS /OCAP: 64/36 44/56 63/37

IV, Estimate of OTIS/OCAP Field Data Collect-J(7n:

1. Field Staff Costs

11)Personal Services . 11 GraduateSNdents $520/month x 12 weeks $ 17,160.00 . 2 ES ProfessionalStaff p 1 and 3 months p 2,850.00 $600 and $750, respectively . Benefits 020f:, ES staff only 570.00 . ClericalSupport (estimate) 500.00 Estimated Total Personlit-Setvices $ 25,080.00

(2)Other Thar Personal Services . Travel and Per DiemExpenses $ 10,253.47 . Mailing(estimated 2800 pieces 0 244 each) 672.00 . Communications 601.26 Estimated Total Other Than Personal Services .===.,m11,526.73 Total Estimated Expenditures for Field Data Collection $ 36,606.73

2.Estimate of Field Cost Distribution OTIS (44%) $ 16,107 OCAP (5694 20, 500

V. EstirOve teac and ect Mana emem Costs: I.Total Expenditures S 102,574.48 2.Total Field Cost Estimate 36,607. ea

Tots I $65,967.486

.111.111111.MIN of Oklahoma. This survey was conducted simultaneously Note:* OCAP: Office of Community Affairs and Planning with the OTIS survey, and the entire effort was treated as a single datacollection project. However, the OCAP project will not be repeated. Thus, OTIS expenses, in termsof future costs, should be separated. EXHIBIT 3-III(2) BEST COPT :".t.T!!ARI.E

VI.Estimated Distribution of Overhead and Project ManagementCosts: I.Estimated Overhead and Project Management Costs $ 65,967.48

2.Estimated Distribution OTIS (6490 $ 42,219 OCAP (3690 23,748

VII.Oklahoma OTIS Demand Cost Estirnate:

I.Field Data Collection Cost Estimate $ 16,107 2.Overhead and Project Management Cost 421219

Estimate .M.1..111111.1WR

Total OTIS Demand Cost 1.510.2.1

Source: Demand Cost Estimates are oased on data and assumptionsdeveloped in conjunction with Mr. William Hunter and Mr. Wayne Edge of the Oklahoma EmploymentSecurity Commission.

r EXHIBIT 13.8N

Economic Development Administrsti*

CALCUIATION OF THE SUPPLY AN& FOLLOW.,UP SUBSYSTEM COSTS

1. Annual aulent Accotmtim; System Cosi.)

(1)Administrative Personnel $ 6,000

(2) Teacher Time--2 bows: 30 minutes (Enrollment Completion 9,000 and Follow-Up): 1800 teachers 0 $5.00/hour

(3) SAS Personnel ("Full-Time) 8,000 .Additional Help 390 .Travel 1,716

(4) Computer Time--26 hours 0 $35/hour 910

(S) Keypmch--100,000 cards @ 44 /card 4,000

(6)interrupt Card (completion of Follow-Up)--24 hams p$4.00/hr. 96

Total 132d1 22

2. Allocation of Supply and Follow -Up System Costs

(1) Supply Subsystem 9,764 (2) Follow-Up Subsystem! EXHIBIT Baq

BEST C3r1 Economic Development Administration

CALCULATION OF OTIS ANNUAL OPERATING COSTS

et,

1. Demand Subsystem Costs

(1) FY 1972 Baseline $ S8,326

(2) FY 1973 Update 7, 457

(3) FY 1974 Update 4,984

(4) FY 197S Update 4. 984 $ 75,751 Divided by 4 years

Demand Subtotal $18,938

2. Supply and FollowUp Subsystem Costs 9,764

3. Interface Subsystem Costs 2.500

Total Annual Operating Costs 446,at ir

APPENDIX C lz

SOURCES AND DERIVATION OF KENTUCKYOTIS COST ESTIMATES

. gio Not APPENDIX C(1)

SOURCES AND DERIVATION OF KENTUCKY OTIS COST ESTIMATES

This appendix presents the estimated costs of developing and operating OTIS in Kentucky. The sources, assumptions, and derivation of these costs are presented in the following two exhibits: Derivation of Kentucky OTIS Cost Estimates Letter from Dr. Janie L. Jones

N. EXHIBIT 04

Economic Development AdminislintICIP

DERIVATION OF KENTUCKY OTIS COST ESTIMATES

I. DEMAND SUBSYSTEM 1. Development Costs

(1) Contractual and Consulting Services $ 68,6971/ (2) Computer Program Development 14,500-V (3) EDA Personnel Support 800-2 /

Subtotal. $ 101,997

2.OpemtiniS2sts

(1) Management Personnel $ 27,7683/ 70,088 (2) Field Personnel 1/ (3) Travel 9 (4) Supplies I,0044/ if (5) Computer Support 22 500 Subtotal 110.356 Total Demand Subsystem $ 212,353

II. SUPPLY AND FOLLOW -UP SUBSYSTEMS 1. Development Costs 5/ Student Accounting Mal (1) 6/ (2) Follow -Up 22 r694 Subtotal $22,694 2. W2a tiri&gaits (1) Personnel $41,6904/ (2) Supplies 10,2004/ Subtotal 111.51.890 Total Supply and Follow»Up Subsystems 74,584

III. EDUCATIONAL RESOURCES SUBSYSTEM 23.1801/

TOTAL COSTS .3.1Pk117 .Development Costs $ 124,691 .Operating Costs $ 185,426

Source: 1/Exhibit C-II, copy of a letter from Dr. Janie L. Jones 2/Estimated by MSI based an two man -days per week on-site 3/Letter from Or Jana L. Jones includes $51,426 for costs of fieldsurveyors to conduct only the census of manufacturing employers, i.e. , subsequentemployer survey costs are not included 4/atimated by Mr. Dave Portch of EDA Information Systems andServices Division 5/No developmental costs for the Student Accounting System are included because the system was developed previously. 6/Includes only Kentucky's share of the cost of an EDA contract with Ohio State University;totals costs are approximately $100,= EXHIBIT 0.11(1)

Economic Development Administration .

BEST C9F'','"v:1.1,711 LETTER FROM DR. JANIE L. JONES

COMMONWEALTH OF KENTUCKY Depart:limit of Mutation BUREAU OF VOCATIONAL EDUCATION FRANKFCIT 40601

August 3, 1973

Dr. William J. Tobin Economic Development Administration Program Analysis Division 14th Street and Constitution Avenue N.W. Washington, D. C. 20230

Dear Dr. Tobin:

I have gone through the cost estimatesfor Kentucky again, and I also asked Mr. Hume, Director of Fiscal Control andFinancial Accounting, to look at it.

In addition, I am sending you a copy of the notesI made on the Oklahoma costs. I realize this has been revised and that someof these things may have been considered in the revision.

As I told you on tne phone, here in Kentucky wefeel that the benefits to be derived from the system aremuch greater than just the obvious ones.

I hope this will help in getting thedisagreements worked out. If I can be of further help, pleasecall.

Sincerely,

Janie L. Jones, Director Occupational Information Unit

JW /ew

Enclosure EXHMIT C-11(2)

KENTUCKY'S INFORMATION FOR TRAINING AND EDUCATION SYSTEM COST ESTIMATES August 1, 1973

I. Demand Component

(1) CoordiLator (full time) 15,288 Secretarial 4,740 Director (k time) 4,014 24,042 Benefits @ 15.5% 3,726 27,768

(2) Field Staff Industrial Coordinators 10,827 Secretarial 5 31i 16,156 Benefits @ 15.5% 18,662 TOTAL PERSONAL SERVICES 46,430

(3) Travel Industrial Coordinators 4,500 Field Surveyors 4,500* 9,000

(4) Supplies 1,000 TOTAL OPERATING COSTS (Demand) 56,430

*If salaries of field surveyors are included, an additional $51,426 should be added(55,926- 4,500) making a total of $107,856.

(5) Project Developmental Costs Computer pram deveiopmeni; 14,500 Contractual and Consulting Services68,697 TOTAL DEVELOPMENTAL COSTS (Demand) 83,197,

TOTAL COSTS (DEMAND) 139,627

II. Student Accounting and Follow-Up System

(1) Coordinator (full time) 11,412 Secretarial 4,512 Director (k time) 4,014 19,938 Benefits @ 15.5% 3,090 23,028

(2) Industrial Coordinators 10,827 Secretaria l -5,331 16,158 BenefitsCa.15.5Z 2 50/ 18,662

p. EXHIBIT C-I1(3)

(3) Supplies 10200

TOTAL COST (SUPPLY & FOLLOW-UP) 51,890*

*The $22,69` charged to Ohio State was not an . expenditure_ for Kentucky. This mmount-watreceived from Ohio State in support of the follow-up.

III. Educational Resources

(1) Coordinator (full time) 16,056 Director time) 4,014 20,070 Benefits @ 15.5% 3,110

TOTAL COST (EDUCATIONAL RESOURCES) 23 180

TOTAL OPERATING COSTS 131,500 TOTAL DEVELOPMENT COSTS 83,197 214 697

Total Costs Annualized for a 5 year period:

Operating Costs (131,500/year for 5 years) 657,500

Developmental Costs 83,197

Total 5 year operating and developmental costs 740 697

Costs Annualized for 5 years 148,139

Prepared by:

Janie L. Jones, Director Occupational Information Unit APPENDIX D

SAMPLE OKLAHOMA AND KENTUCKYQUESTIONNAIRES KENTUCKY QUESTIONNAIRE LL'vey Number: 27 Report on Occupational Employment PRINTING and PUBLISHING Industry, 1972 BEST COPY AVAILABLE KENTUCKY MANPOWER PLANNING COUNCIL --BUREAU OfVOCATIONALEDUCATION OCCUPATIONAL EMPLOYMENT SURVEY Report No.

Address Label THIS INFORMATION WILL BE MADE to be placed here AVAILABLE TO VOCATIONAL EDU- CATION AND OTHER AGENCIES FOR MANPOWER PLANNING, CAREER GUIDANCE. AND CURRICULUM CHANGE PURPOSES. Change address if incorrect

General Instructions

REPORTING UNIT: Complete this report for your company operation (Reporting Unit)identified on the above mailing 13brI ' help multi-unit employers correctly identify the Reporting Unit, its physical location has beenprinted in the lower left portion of the label and our estimate of its total employment appears as a six digit number in thelower right hand corner of the label. ( Multi- unit employers may receive' queStionnaire forms for more than one Reporting Unit). ,

REPORTING PERIOD: Report information for a payroll period that includes May 12, 1972. If, because ofunusual operational problems (e.g., work stoppages, temporary shut - downs), the May 12 period is not a typical period, please reportfor a pet nnr- . earMay 12 in which operations most closely approximate the normal.

PART I. GENERAL INFORMATION

I. TOTAL EMPLOYMENT (9999) Enter the total number of persons on the payroll covered by this report who workedfull- or part-time or received pay *or any part of the period reported. Include salaried officers of corporations and executives and theirstaffs, but exclude proprietors. members of unincorporated firms, and unpaid family workers. Include persons on vacations and sickleave for which they received pay directly from your firm for the period reported but exclude persons on leave without company paythe entire period and petvioncrs and members of the Armed Forces carried on the rolls but not working during the period reported.

2. NATURE OF BUSINESS

(a) Describe the principal activity and the major product or service of the Reporting Unit(e.g.. Manufacturingwomen's shoes; Warehousingsteel products; Researchradio and T.V. receiver; and Retail tradeshoe store).

(b)Is the Reporting Unit primarily engaged in performing services for other unitsof your company? Yes

If "yes" please check the one block below that best describes the service beingperfomred.

(1) r_10 Central administrative office (3)a Storage (Warehouse) (2) Ll Research, development, or testing (4) 0 Other (Specify, e.g., powerplant) 3. STATUS OF ACTIVITY . Payroll period that includes May 12, 1972.

If the reporting unit did not operate under your management duringthe May 12 period, please check the appropriate block below:

This unit has been sold or merged. New name and address is*

This unit is out of business.

Other (Describe)

4. If questions arise concerning your report, whom should we contact?

Mr. Mrs. Was Name Title City State Area Code I ch.. No. LE PART $1 REPORTING INSTRUCTIONS BEST COPY NANA

The occupations listed in Part 11 of this form are trouped into broad functional categories and include most of the joba that are likely to be found in the Reporting Unit's type of operation. Some of the listedoccupations will not he found in !Lea establishment of this type.

Please review the broad categories for an overall view of the kinds ut occupations we are studying. Definition. ot specific occupations within a group are provided in the enclosed "Booklet of Definitions " (Definitions are presented in "Definition Number" order. Use the Definition Number that appears in column t ! ) to easily locate the definition for any listedoccupation.) Subtitles, shown in parenthesis, may follow some occupational titles. these represent mans but not all of the different titles employers sometimes use for these occupations and are shown for itiustrative purpose: only .

Report each of your employees in one or the other of the listed occupations or ALL 0 Utille categories Do nut report a worker in more than one neupation.

F.mployees of the Reporting Unit should be counted in the occupations hated in Part :lot the. I. J "Working As" basis as of the date of the report. regardless of the occupational field in which they receivedthen training. rot evinip, an employee trained as an engineer but working as a mathematician ut the dateit rue report should he counted 41 a nuthemanciart. An employee whose job duties cover more than one ot the listed 0,cupaiikkii, should tit kook., in the occupation that, in your opinion, requires the !Attest level of skill.

Fmployees whowere absent .withoutpay during the period None tcoorted ti.e.. short term absnces resulting from illness, aa7viather. temporaryayeayojury duty) should be included in the occupationaltinints.

- Working Foremen (those spending20',7, or more of their time at tasks similar to this None. pertormed by %sinkers under their supervision) should be counted in the occupation most closely 'elated to that work duo.. . a %%inking foreman in an inspection department should be counted as an "Inspector "1.

- Part-time Workers, Learners, and Apprenticesshould hr tointed in the ot oration its which the online:Iv perform their work.

- Clerical Supervisors should not he counted in anyitt the %pectin of L tipation. Instc,,d, thee should he . owned io the most appropriate ALL 011111 category J -Payroll Supervisor- ,ttogiti be oolic41 inob. Ai 1 in k mike Clerical Worker Category).

- Owners and partners of unincorporattod firms andunpaid family worker, should bys. Rent bbl \lei +'rt

Include in this report. company personnel who pertorm their oars, ditties At F "out of- the reporting unit 1t! %%hi% for administrative reasons, are earned on payrolis of other units tit soor color," 'r irjve.ine salesmen service personnel who work "out ofthe reporting unit should by in lotted in soot prt a ern thous+ these personnel may he carried on the rolls of another unit of your company.

Reasonable estimates of the number of workers you employ in an occapatton are act eptatitc.

Description of Column Headings:

Column (3): This column is headed "TOTAL" and appears for r'iery occupation. Filter rn this column thy total number of workers you employ in the listed occupationsItthis is COtisutvted a i:riti. at shortage occupation, please circle the definition number

Column (4): This column is headed "Apprentice" and appears only with the "Maintenance. Construction. Repair and Powerplant"..,and selected "Production (Plant t- occupations Niter in this voltam) the number of apprentices, already reported in columni). that you emplov rn any of these oce.tpations (Set. definition of apprentice on page 2 of the "Booklet Prtioitions.")

Column (5) This column is headed "R & D" (Research :net 1).selonmem a ;mil alrouAr, only w silt sctcrrrtfic and technical occupations. Enter in this column the number t %soars.... ArVatiV 1,vrted fir eulurtltr 13i, who are working primarily inresearch andam.i.cs,Ikiikmbi !irk. de. mimic ttf r :tich and, development on page 2 of the -Booklet of Definitions.";

Columns (6) and (7) Enter your estimate of the total number of your workers In reporting unit you spe, t to have in June. 1973, and in June. 1974. Assume. 41) a `: nati.oral Unemployment rate. (2) your current plans for expansion or modernization will materialise according irk s, tredolc. t 31 scientific and technological changes affecting production methods, manpou requirements. and riosemptron patterns will continue according to industry expectations, and 14 a the normal hourly' workweek it sour establishment will continue through the forecast period.

The sum of the column (3) counts should be approximately eqoal to the "It)! ," oVAII. NT" reported in Part t of this form. ( These may differ slightly because column (3) data may include employees et1 .hurt :C111) absence without I. iy you do not employ anyone in a partn.ular occupation leave column 13) lilac's BEST SAMPLE ENTRIES COPYAMIABLE

Appeen- Occupations basal ties III 121 (3) f41 Mallittitasee. Construesn. Royale and Power Plato OrvapatIona 5 carpenters are employedTull the 5 is an "apprentice." Carpenter 5 1 3 electricians are employed. None areapprentices. I it% tit. ...ft J

S nonworking lot enter' :ere employed m"Production WNW) occupations. Produetkiis (Plante :retmattotts t of the 5 is the inspection departmentforeman. 10 inspectors arc ent1,0 "learner.- Ci.ltann 0Ft mformatorrt rotl1.01 nonworktite 1 is a -working.' foreman and I is a on Apprentices is notrequited for these occupations. 11.1

tte, Total 1i & (2i 131 1%) : of these mnage enpmecnng lortourrs # ;# il.litterv, mut officers .ere .niceties an.l I is ih managrr of thy .ante's chemical analysisdepartment. The remainingare nontechniral i(sontroller. SalesMee , ct..1 and are reported In the ALL 01111 N category. 1 of the engsnectstneaged At Ittl ill it rom:so isI.,. research and development letivittes

Prolwoloaal t lceupathen. Seleolltl rciirct 7 of Nli fINIOlf .5" 2.. /4 engineer, air employed Inotcounting Managers and Officers). t.I.:r1C;II MO or, ri.11 these work tn specifically listedengineering occupations. !e remaining englat engineer is a "Mining Engineer" anoccupation not specifically listed. therm( engineer arid is reported in the ALLOTHER category. 2 of the 5 mechanical en At Iri 1111 P gtneers are engaged Inresearch and development tectonics

Expected Expected Defini- EmploymentEmployment toot) in in number Occupation.: Tof..41 June June (1l (2/ (3) 1973 1974

TECHNICAL OCCUPATIONS 10 draftsman were employed as of May 1972. 13 are anticipated for EngineeringTechnicians June of 1973 (a net increase of 3) /7 and 17 by June of 1974 (a net in- Draftsman crease of 7) 1>Vir;a121.31 PART U EMPLOYMENT BY OCCUPATIONS PRINTING & PUBLISHING INDUSTRY PcSi COPY

Definition Expected Expected number Occupation Total Employment Employment (1) In Ione. 1973 In June.19/4 (2) (3) (6) 11)

1.000 EMPLOYMENT IN ALL OCCUPATIONS, TOTAL___ 2.000 MANAGERS AND OFFICIALS. TOTAL

3.000 PROFESSIONAL AND TECHNICAL WORKERS. TOTAL s 3.010 Accountants and auditors

3.020 Commercial artist (illustrator, editorial artist, art layout man)

3.030 Commercial designer (bank note, book cover, form designer)

3.040 Engineer (electrical, industrial, mechanical, chemical engineer)

3.050 Estimator T... 111 0011.. deMillo ORM*

3.060 Mathematical scientist (mathematician, statistician, actuary)_ i 3.070 Natural sicentist (chemist, physcist, biological scientist, geologist)

3.080 Personnel and labor relations specialists (employment interviewer, job analyst) 3.090 Photographer_

3.100 Purchasing agent . a

3.110 Social scientist (economist, political scientist. pyscho- logist: sociologist)

3.120 Systems analyst, electronic data processing

3.130 Writers. editors, and reporters (copy writer, rewrite matt, deskman, advertising editor)

3.140 All other professional workers (physician, lawyer, archi- tect, professional nurse, reference librarian)

3.150 Technicians

3.151 Computer programer

3.152 Draftsman

3.153 Electrical and electronic technicians

3.154 All other technicians

4.000 SALESMEN (ADVERTISING, CIRCULATION. BOOK, PRINTING SERVICES SALESMEN)

5 000 CLERICAL WORKERS, TOTAL

5.010 Accounting clerk

ti 4 BEST COPY AVAILABLE BY OCCUPATIONS PRINTING & PUBLISHING INDUSTRY PART H EMPLOYMENT .M.10, I p.tv.1 Definition ispitted Total Irnpli,ment I nThl..%,,u,n, number Occupation In June WI In him- i.1-4 (1) ( 2) (31 161 1't

5.020 Bookkeeper

5.030 Cashier

5.040 Classified -ad clerk. newspaper

.5.050 Correspondence clerk

5.060 Customer Service representative (customer liaison man)

5.070 invoice - control clerk

5.080 Ottice machine operators. total

5.081 Billing andlor bookkeeping-machine operator

5.082 Digital-computer operator (computer console operator) ,

5.083 Keypunch operator All other office machine operators. (calculating, 5.084 ... ,duplicating. tabulating machine operators I --

5.090 Payroll clerk

5.100 Personnel clerk

5.110 Procurement clerk (purchasing clerk ) _ _ ......

5.120 Telephone ad-taker. newspaper

5.130 Telephone operator (switchboard operator. PBX operator)

5.140 Secretaries. stenographers. and typists. total

-5.141 Secretary

5.142 Stenographer

5.143 Typist (clerk-typist) 5.150 All other clerical workers

6.006 PRODUCTION, MAINTENANCE. MATERIALMOVEMENT, AND POWERPLANT WORKERS, TOTAL ___ ...... _

APPRENTICES (Exclude learner; and helpers 1

6.010 Apprentice bookbinder fcompositor. 6.020 Apprentice compositors and typesetters (hand Linecasting-machine operator, phototypesetter operator) . . 6.030 Apprentice electrotyper

6.040 Apprentice etchers and engravers. except photoengravers 6.050 Apprentice lithographic preparation worker (cameraman, platemaker, stripper) _

S PART II EMPLOYMENT BY OCCUPATIONS -- PRINTING & PUBLISHING; INDUSTRY I COt"IIIVAIMILE Daffi Itton t eiptutril t ikee.ted 0.Lup.ition Implmerneeett I periovment matthcr lotal in luneeM It.teem.icrt4 II1 t 2) t 3 ) MI e le

tellitl(1 AMIN't)the,pholocogrx, , t -....&

f (rt) 11%ttlitt.Cins1/.01(1%-tt'Spt, %%1110P:ION litt: i`h. ..distil 01 tVt'itthiWr.tpht. pro...mall, _

6.0h0 %pprontlit: trot per

6040 Apprehtt,:e.. all other prtnttnp. tr;tdc. I nuilor..L.roon prtntrt

6.10(4 Approlti.o.ill otltor I olc-tri. ion. m.1,111111.1. .tatioit.ir ilttm,cs I

COMPOSING RC)OM ()CUT %1 IONS

elII II ( 1%p% 1iltt,rl- tip% Ittlit I __ ---______. -- ._ .._

6.120, Hind Lompo,./),,r II

t% I:4) II1910%.1., !III III.IkcIII' Iltot 4 l", kop w.o). %tom !Lola )

ft. I 40 I inecdst mgmaltmo oper.itt+r I {mot% pe op..) at))) . ollot

t II' ep..t.itor 1 _

(1I c 0 I in,..1.1tritr !tut hinke1.1%,tatil opordts+r t telet pcstct pertoratfw c)pentrot I

b.! 60 I inet..tstin..!Mat {UM'tender t tolot peet tint: m,schino monitor!

6. I '0 I it,110%-t11.1.11111,'or,tato). I 11,1,,1% pe iperatort

I. I tai(! Mark-up man

1 6 I 90 Nri I ) ilt liIle -L...istittg. ma,. limo op, tator I rmInt,matt, ,-,1.4,:i I

E 6 .100 .tottot pokoN hoard opvratotinictehthV: ii.e,%1.'wad opotatort

6..1 10 Past cup mon

i 6 .1_0 l'hotolottot me ma.I1,114:illsOr.lh'r

6.230 l'hoto4 posottittp-maatne ke% homilI tilt*/ AI it (ph(,to- graphtL ki-% hoard opc t.i tor t _ _

6 240 IPhotot) powttmg-01.1.11111,.11).$11,t, kr I :thotoirrapInt unit operator)

6...150 l'hototN pc sot tor ()pont t t+r4ho1ogrJp1111.N1stem operator) 6.260 Pritolreader. compoNmg room_ -- 6 270 Strike-on machine I yorator

LITHOGRAPHIC" PHOTOM ECH A NI(' L PREPARATION OCCUPATIONS

6.180 Arttst 1 rototth.lter. dot etcher, !o(t:ner-color cli.1),:r) _

ts BEST COPY AVAILABLE PART II EMPLOYMENT BY OCCUPATIONS -- PRINTING & PUBLISHING INDUSTRY

Dainition 1.1%,,t,, i .r.,,t,..1 Total I ,,;t,ftirtit nurnhcr Occupation Inhuh I', I t-, 1.1. 1-4 I I{ ( 2 ) t a t ,.,

6.240 ( anteram.m i 1/4-opN, .0141..011.m. photo lithographer. halt- ',wk.. cameram..

6.300 Developer 1.irk room man) MIIIIIN

6.310 Platemaker t press platemaker. transterrert ...._ ....., ...._

6.320 Strippor

0 OTHER PLATLMAKING OCCUPATIONS

6.330 I' It's Irtst 4 per t electrotv pe c.aster. elec: trot v'pe mOlder )

. 6,340 1.4 has and ertinat.ers. evc.ept photoet4;raters ( panto- grapher. siderographcr. steel the cagraver. letter engraver)._

6.350 rholocitgrater 14:ameraman. printer.tc.-Iter. router. bloc:ker. tinisher)

6.360 Stereop. per

PRESSROOM OCCUPATIONS

6.3'0 I k ko.!ropht. pressman 6.380 ( srattth pt,-,sman. rotogravure and litli led

6.390 I Mel' pressman. sheet led and roll fed t %eh ted)

6.400 Letterset prssmon (dr,. ottset. indirect relict). sheet fed and roll led (wb led)

6.410 Ottset lithographic pressman. sheet fed and roll led t wt-h led) h...,....amlM.14

6.4 : (.) Steel die pressman (die stamping. steet die embotving press, ni,ont -1111NE..140.11.7114

6.430 Press .tsststants and (ceders _

SCRE- EN PROCESS OCCUPATIONS

6.440 Screen cutter (stewl cutter. film cutter) ---- -

6 4S() Sreenmakcr. photographic ;mite. 6.460 Screen printer '-

BINDING. MAILING. and SHIPPING OCCUPATIONS

6.470 Bindery machine ic tup man_...... '______

7 PART H EMPLOYMENT BY OCCUPATIONS PRINTING & PUBLISHING INDUSTRY 601COPYIG..1315.

Definition Expected Expected number Occupation Total Eatpicarneat Ent) lamest la June, 1973 la tune,1974 ( 1 ) ( 2) (3) (6) (7) V 6.480 Bindery worker (bindery hand, bindery man or woman, table worker) _ __ _ 6.490 Bookbinder

6.500 Delivery man

6.510 Mailer

6.520 Routeman, newspaper (route supervisor)

6.530 Shipping and/or receiving clerk

6.540 Truckdriver, light or heavy

CONSTRUCTION. MAINTENANCE. REPAIR and POWERPLANT OCCUPATIONS

6.550 Carpenter 10..11.11 6.560 Electrician

6.570 Machinist (composing room machinist, press room machinist)

6.580 Mechanic, automotive

6.590 Mechanic. general ...... _ 6.600 Mechanic. maintenance

6.610 Mechanics and repairmen, other

6.620 Plumber and/or pipefitter

6.630 Stationary engineer

ALL OTHER PRODUCTION, MAINTENANCE. MATERIAL MOVEMENT, and POWERPLANT OCCUPATIONS ,

6.640 Foreman, nonworking

6.650 All other skilled craftsmen and kindred workers 6.660 All other operatives and semiskilled workers (power truckdriver, inkman, paper conditioner, paper cutter, proof press operator)

6.670 All other laborers and unskilled workers (warehousemen, , press cleaner, paper handler, fly boy, groudkeeper, etc.)

7.000 SERVICE WORKERS. TOTAL 7.010 Guards, watchmen and doorkeepers

7.020 Janitors, porters and cleane.s 7.030 All other service workers 1

8 OKLAHOMA QUESTIONNAIRE BEST COPY AVAILABLE

iDa Not Us.This knee* SERVICE SCHEDULE Editor Dots 1 OKLAHOMA OCCUPATIONAL NEEDS SURVEY QUESTIONNAIRE

Return to Oklahoma State Employment Service Research and Planning Divil310f1 Will Rogers Memorial Office Building Oklahoma City, Oklahoma 73105 L 521-3735 Area Code405 NOTE: Before Entering the Requested Information, Please Read the Instructions Attached to This Schedule.

EMPLOYMENT. t xPECTED NUMBER OFotORRINSNUMBER OF EXCLUDING COMPL VACANCIES EMPLOYMENT IN-PLANTTRAINING T OCCUPATION TRAINEES IN TRAINING UNFILLED CODECODE , PROGRMS fay 30 DAYS s .IDTjAiNeMMALE JUN 973vJUNE1975-JUNE 1473JUNE 197S OR MORE (V (2) el [4) (5) (6) (7. (8) Cif')

PROFESSIONAL TECHNICAL AND MANAGERIAL OCCUPATIONS

01.081.1, 111 HI EEC' T on1.281.0 DRAFTSMAN. ARCHITECTURAL

001.081.0 1.. IX Cl RIC AL ENGINEER

1101_081.1 I EECTRONIC ENGINEER 001.I81.0 1 LECTRICAL TECHNICIAN

00.1.181.1 ELECTRONIC TECHNICIAN 003.28E0 DRAFTSMAN. ELECTRICAL OR ELECTRONIC 4 '00.081.0 CIVIL ENGINELR 00.110.0 CIVIL ENGINEERING TECHNICIAN

Of ).281. 0 DRAFTSMAN, CIVIL no-.0)41 0 411(.11ANICL ENG.NEER n0-.181.0 vEcHANICAL ENGINEERING TI,LHNIc.IAN O'.:fi 1.a DRAFTSMAN, MECHANICAL

010.081.0 PETROLEUM ENGINEER 010.281.0 DRAFTSMAN. GEOLOGICAL

012.768 .1 SYSTEMS ANALYST, HISS. ELEC. DATA PAM 012. USTRIAL ENGINEER MEM TRIAL ENGINEERING TECHNICIAN _1.111111 ALL N. MAP 090.228. IDa, TOR, RCS, DATA PRoCFSSING Abb. 091.1 tR.0ISM ESs A.M. EMMIilli.._ 0a1.228.0 MEL RING AND 41.11.1111111. 041.228.1 TE ACHE R AIMI01.111111 Ada 091.228.2MIZSMIELL aljaMMIIIIIMMINIMEMIN Mill 042'22"EZE OOL 111111111.1111.1.1 MI= "4.22"ISZEIMInaillillinillill NM 044'22"ila 1111.11=111.1.111= 100.168.0112=111.1111111111111111.1111111111111111111111.111111111111111 "2."8.°UngialliMall11111.111111MilliMINENI (CONTINUED ON REVERSE SIDE) PAGE -,-

pr(1 COPY ABLE

SERVICE SCHEDULE (Cent.)

EMPLOYMENT. EXPECTED NUMBER OF wORKL RSNUMBER OF EXCLUDING COMPLETING VACANCIES TRAINEES EMPLOYMENT INPLANT TRAINING D. 0, T. OCCUPATION IN UNFILLED CODE JUNE 1971 PROGRAMS BY: 30 DAYS TOTAL FEMALE JUNE 1973JUNE 1973 JUNE 1973JUNE 1973 OR MORE (1) (21 (3) (4) (s) (6) (8) (7) 19L. PROFESSIONAL, TECHNICAL AND MANAGERIAL OCCUPATIONS (Coot./

132.288.0 COPY READER OR SCRIPT READER

159.288.0 WRITER, TECHNICAL PUBLICATIONS

141.051.0 ILLUSTRATOR. COMMERCIAL ARTIST

141.081.1 LAYOUT MAN (PRINTING AND PUBLISHING)

143.062.0 PHOTOGRAPHER OR CAMERA MAN

160.188.0 ACCOUNTANT'

160.2E8.0 COST ESTIMATOR

163.068.0 PUBLIC RELATIONS MAN

166.111.0 PERSONNEL DIRECTOR

161.168.0 CREDIT MANAGER

189.168,0 MANAGEMENT TRAINEE

CLERICAL AND SALRI OCCUPATIONS

201.168.0 SECRETARY 202.388.0 STENOGRAPHER

205.36110 PERSONNELCLERK

209.568.0 CLERKTYPIST 210.385.0 BOOKKEEF'ER, HAND 211.568.0 CASHIER 213.382.6 CARD-TAPE CONVERTER OPERATOR

213.182.1 DIGITAL COMPUTER OPERATOR 21).182.0 KEY-i-er4C21 OPERATOR

21 782.0 TABULATING MACHINEOPERATOR

215.388.0 BOOKKEEPING MACHINE OPERATOR 215.488.0 PAYROLL CLERK

219.388.0 CLERK. GENERAL OFFICE

219.318.2 PROGRAMMER.DETAIL 219.481.0 ACCOUNTING CLERK

235. ELF PHONE OPERATOR (P. B. X.) CLERK a AN. SERVICES 0.1 DRIVER (ROLTTEMAN) 132.271.0 Aillil ERVFCE OCCUPATIONS 559.978.0 Allah. ANT OR COFFEE S 4Millii 372,568.0 AIIIIIIII AI MIIIIIIIMI 3E2.884.0 AINIIIIIIIIMII AMIIMIIIIIMUNINIII. 407.884.0 AIMMIM1111101111111111111M111111NNIIIIIII 461.885.0 AIIMIIIMMIIIINI 466.587.0 LIVESTOCK CARE. R

PROCESSING OCCUPATIONS

521.782.0 GRIMM R ft ATM tt-RA1N AND FED WEI / 526,78E0 HAE101 1111111111 (CONTINUED ON PAGE 3

PAGE BEST COPY AVAILABLt

Do Not Use This Space

Zdi to Dam fa SERVICE SCHEDULE (Cant.) 1Mum

Return to: Oklahoma State Employment Service Research and Plonr.ing Divis,ort Will Rogers Mem,ricil Oftice Building Oklahoma City, Oklahoma 71105 321.31,75 A -'a Code 405 L NOTE: Before Entering the Requested Information,Please Read the Instructions Attached to This Schedule. ___ IA L YMENT. NUMBER E RSNUMBER OF -- EXPECTED COMPS STING VACANCIES EXCLUDING EMPLIN MENT TRAINEES tNPL ANT RAINING UNFILLED T OCCUPATION :* ,M '30 DAYS MA E JUNE 1973JU E 1975JLINE 1973JUNE 197S 01K MORE ,s, a9, 451 '61 171 (1) 2

MACHINE TRADESOCCUPATIONS

620.281.0 AUTOMOBILE: MECHANIC 620.281.) AIR CONDITIONING MECHANIC, AUTO. 621.281.0 OUTBOARD MOTOR MECHANIC 624.281.0 FARM EQUIPMENT MECHANIC 624.281. DIESEL Mi.C.HANIC 621.2t41.I SMALL ENGINE REPAIRMAN 6I;.281.0 OFFICE MACHINE NERVICEMAN 01,'$2, I OFFSET PRESSMAN DENCHWORK OCCUPATIONS

"20.281.0 RADIO ANTI TFt FVISION REPAIRMAN 721.281,0 ELECTRIC MOTOR REPAIRMAN

123.884.0 APPLIANCE REPAIRMAN, HOUSEHOLD

T61,.S81.0 FURNITURE. FINISHER

M. NJi .0 FURNITURE UPHOLSTERER I 78s. ISLA SEAMSTRESS STRUCTURAL. WORK OCCUPATIONS

88 7. IMMOBILE BODYREPAIRMAN ER, ARC 11111.11111111.111111r"1111111111 GAS 111"111111111111111111, 0.000 nffilizgailiMmlir NIIIIIiillir

DATE SIGNATURE epreterttetree Colopletott Floe Report

PHONE TITLE

PAGE 3 APPENDIX E

RESURVEY METHODOLOGY APPENDIX E(1)

RESURVEY METHODOLOGY

The nucleus of the entire OTIS systemis the interface of projected manpower demand withestimated manpower supply. The forecastsof manpower demand are employer-based. Therefore, anassessment of the accuracy of these employer forecasts is crucial to anevaluation of the OTIS demand data collection methodology. Consequently,the Project Monitoring Committee decided to survey 500 employers whoparticipated in the Oklahoma and Kentucky OTIS surveys. This appendix describes the resurveydata collection methodology, including: Sample selection Interviewer training Data collection Data processing

SAMPLE SELECTION The OTIS surveys in Oklahomaand Kentucky were conducted to develop manpower demandforecasts for the entire state.Oklahoma utilized a sample of employers and Kentuckyattempted a personal interview census.The purpose of the resurvey wasto test the accuracy of theemployer forecast and OTIS regional forecast by occupation.Therefore, our efforts focused onselect- ing a uniform resurvey sample.Sample selection in each state wasconducted as follows.

(1) Kentucky Sample Selection Kentucky attempted a censusof manufacturing employeeslisted in the 1972 Directory of ManufactUrers,published by the Kentucky Depart- ment of Commerce. As c,fSeptember, 1973, the results of the survey were: 2,946 employers were in the universe 2,855 were contacted APPENDIX E(2)

374 were nonmanufacturers, who were thereby eliminated from the survey population 275 firms were determined to be inactive, i.e., out of business 69 employers refused to respond 61 employers were not contacted Thus, a total of 2,175 usable responses exclusive of nonmanufacturing and inactive firms was received, giving an overall usable response rate of approximately 94 percent. Employers responding employ approximately 214,000 of the 234,500 manufacturing employees. r The resurvey sample also was selected from the 1972 Directory of Manufacturers as follows: Employers were classified into four groups, based on employment size The total number of employees in each group was estimated The sample (number ofestal5iishments) for eachgroup was set proportional to the number of employeesro eachgroup Using these procedures, a total of 600 establishments was selected. The number of establishments selected in each group is detailed in the following table:

Number of Number , Manufacturing Employers Establishments Selected First group (100+ employees) 520 388 Second group (50-99 281 79 employees) Third group (20-49 employ- 526 65 ees) Fourth group (under 20 1,353 68 employees) .11 IMOINNIRMID Total 2,680 600 APPENDIX E(3)

This sample was verified using the 1973 edition of the Directory of Manufacturers. Subsequently, the following categories of employers were eliminated: Inactive Noncooperative in first survey Nonmanufacturing and included in directory Failed to submit schedule Discontinued operations, mergers, acquisitions, and other reasons reduced the sample to 486. Subsequently, each company was identified by county and grouped by region. Each region was treated separately to assess the number of sites within each.Exhibit E-I, following this page, identifies

4. the individual towns within a region and the number of employers per town. The regions were then grouped to identify the counties with the largest number of companies in each county.Exhibit E-II, following Exhibit E - I, identifies the seven counties having the largest number of companies per county. The final step involved the selection of actual companies to be resurveyed.Identification of a county cutoff point was related to the cost effective administration of the resurvey. This process resulted in the selection of the 338 companies that are identified on the Scheduling Grid presented in Exhibit E-III,following Exhibit E-II. The 38 additional companies wereincluded to provide for unforeseen problems in the field.

(2) Oklahoma Sample Selection The original OTIS survey in Oklahoma was a sample of employers. The largest employers in terms of employment werecontacted personally. The rPmainillg firms receive survey questionnaires and are contactedonly if an adequate response to the particular cell wasnot received. Encrirr E-I

Economic Development Administration Cid PiMAill\311. BEST NUMBER OF EMPLOYERS BY REGION AND TOWN

Region 91--29 Resion #4--43 Rettionif7--32 Be o02::1.2

Benton 2 Russelville S Covington 10 Corbin 4 Calvcry City 2 Bowling Green 10 Ludlow 2 Middlesboro 6 Paducah 11 Glasgow 7 Newport 3 Williamsburg 2 Wickliffe Munfordville Walton London 4 Mayfield 4 Horse Cave Florence 9 Harlan 2 Murray 3 Tompkinsville 2 Dry Ridge Brats 1 Clinton 2 Franklin S Falmouth 1 Farmington Bonnieville 1 Earlanger 2 Region .14-25 Iton 1 Game liel 1 Carrollton 2 Arington Adairville 2 Bur ler Burnside 2 Lynn Grime 1 Cave City Albany Auburn 2 Region_#8-44 Somerset S Region .2'32 Morgantown 2 Sterns 1 Scottsville 3 Mayesville 4 Summitville 1 Central City 3 Flemingsburg 1 Monticello Madisonville 3 Region #5-2I Vance Burg Pine Knot 1 Marion Liberty 3 Princeton 4 Springfield 2 Region .9.6 Greensburg 2 Hopkinsville 9 Barastown 3 Dunnville 1 Dawson Springs 2 Hodgenville 1 Morehead 3 Ferguson Guthrie 2 Lebanon 3 West Liberty Campbellsville S Kuttawa Leitchfield 2 Mt. Sterling 2 Colombia 1 Grand Rivers Loretto 1 Greenville 1 Hardinsburg 1 Region 910-16 Rsgion 915 43 Elkton 2 Elizabethtown 6 Cadiz 3 Brandenburg Ashland 8 Winchester 6 Baton Hitching Richmond 6 Regioni3-32 Catlettsburg 2 Nicholasville Region 16..134 South Shore Paris 2 Maco 1 Olive Hill 3 Lancaster Henderson 11 Shelbyville 4 Lawton 1 Lawrenceburg 3 Lewisport Simpsonville Lexington 26 Beaver Darn 2 Louisville 113 Resion #11-5 Pranfort 8 Hawesville 1 Shepherdsville 1 Danville 6 Hartford Eminence 2 Paintsville Georgetown 5 Morganlold Mayfield 3 Whitesburg 1 Berea 3 Fordsville Marion I Prestonburg Carlisle Ownesbaro Mt. Washington Lackey Burgin --Providence La Grange 2 Auxier 1 Harrodsburg 2 Sense 1 Jeffersontown 2 Irvine 1 Clermont Region 112 3 Stanford 1 Buckner 1 Stanton 2 Bagdad 1 Hazard 1 Versailles 3 New Castle 1 Jackson 1 Cynthia= S Beattyville 1 EXHIBIT Emil

Economic Development Administration

KENTUCKY SCHEDULING GRID frit.ASIE BESTCOPY

Region Companies it CountyiLargest) Total

# 1 11 4 3 2 2 2 1 25

# 2 9 4 3 3 3 2 2 26

# 3 II 11 2 1 1 1 1 28

# 4 10 7 S S 3 2 2 34 s # 5 6 3 3 2 2 I 1 18

# 6 113 4 3 2 2 2 1 127

# 7 10 9 3 2 2 2 1 29

# 8 4 I I - - - - 6

i # 9 3 2 1 - - - 6

#10 8 3 2 I 1 1 - 16

.. #11 1 1 1 1 1 - 5

#12 I 1 I - - - 3

#13 6 4 4 2 2 1 19

#14 5 5 3 2 2 1 I 19

#15 26 _A 6 b 6 5 5 62

Subtotals: 224 67 41 29 27 20 15 423*

This number is less than 486 because of counties falling outside the chart, EXHIBIT EIII

Economic Deis lomat Adminis tration a %rl.Tavi. 101 KENTUCKY EMPLOYER DISTRIBUTION

Region # 1 11 4 #2 9 4 3 3 3

# 3 11 11

# 4 10 7 5 5 3

# 5 6

# 6 113 4 3 2 2

# 7 10 9 3 2 2

#8 4

# 9 3

#10 8

#11

#12

#13 6

#14 S S

#15 26 8 6 6 6 S s

Scheduled Companies- -338 APPENDIX E(4)

The 1971 sample was selected as. follows: OESC records were utilized toidentify all businesses that paid a wage to four or more workers ineach of 20 weeks. The state was divided into 11 state-planningdistricts. The employers in each industry division in eachdistrict were ranked in descending order by totalemployment. The sample was selected as follows: 100 percent of the first two quartiles 20 percent of the third quartile 5 percent of the fourth quartile The selected sample was adjusted tc includenoncovered firms and major employers not otherwiseincluded in the sample stratification. Overall, the Oklahoma sample included 416manufacturing and 2,217 nonmanufacturing establishments employingapproximately 375,000 people. The response rate for the sample wasapproximately 70 percent, or 1,865 reporting units. Due to confidentialityconstkaints, the names of the employers participating in the Oklahoma survey alid the numberof employees in each firm were not available to MSI. Therefore, theOklahoma sample was selected from a universe of all establishments in theoriginal sample, whether they participated in the OTIS survey or not.Establishments were selected as follows: Government establishments were eliminated. Counties were grouped into seven geographic regions. Within each region, the counties were lasted in descending . order by number of establishments. All counties with 40 or more establishments and a systematic sample (every nth company) of other counties withineach region were selected for sampling. Within each selected county, a systematic sample (every nth company) was drawn. APPENDIX E(5)

Using the above procedures, a total of 530 establishments was selected. The regional and county breakdown is discussed in Exhibit E-IV, following this page.Subsequently, at the request of the Project Monitoring Com- mittee, an oversample of 200 firms was selected using similar techniques. This brought the total number of firms to 730.

2. TRAINING INTERVIEWERS The resurvey interviewers were all college graduates with prior consulting experience or trained professional su...veyors.All interviewers were trained prior to field work. The training included: An explanation of OTIS A description of manpower forecasting concepts An evaluation of the survey documents A description of the various occupations Practice interview procedures

3. DATA COLLECTION The actual resurvey was conducted differently in Oklahoma and Kentucky. Each of these resurveys is discussed subsequently.

(1) Oklahoma Resurvey Due to confidentiality constraints, the first step in the Oklahoma resurvey was to obtain permission from participatingemployers to utilize the survey questionnaires completed as part of the OTIS survey conducted by OESC. Waivers were asked from all employers in the OESC sample universe. The waiver requests and form are included in Exhibits V and V I, following Exhibit IV. After a waiver was received. OESC released the original survey document for, use in the resurvey. Many companies in the resurvey sample were eliminated because they did not participate in the original survey, would.not cooperate in the resurvey, orbecause interviews could not be scheduled during the survey time period. The resurvey included approximately 250 firms. Each interviewer was 'given a resurvey document prepared from the original survey doctiment, and an introductory guide.The guide and a sample resurvey document areincluded in Exhibits VII and VIII, following Exhibit VI., Using the revised OTIS form, theinterviewer: EXHIBIT E-IV

Economic Development Administration BEST COPYAVAILPBLE NUMBER OF EMPLOYERS BY RESURVEY REGION, COUNTY, AND TOWN

Region CotLMn Number Of '...%-tablishment Sampled

I Garfield 18 Kay 16 Kingfisher 10. Texas 10 Woods 10 Grant 5 Cherokee 5 5

II Comanche 10 Custer 12 Grady 12 Kiowa 7 Tillman S Harmon 5

III Carter 12 Pohawatomie 10 Garvin Payne 12 Bryan 12 Murray 9 Coal 5 Oiduskee Pototoc 5

IV Muskogee 16 Washington 16 Ohawa 10 Okmilgea 10 Craig 15 Sequoyah 10 McIntosh 6 Now ata 5 Delaware 5

V Oklahoma 65 Cleveland 16 Logan 10 Canadian 10

VI Tulsa 55 Creek 10 Cline 170 10

VII Pittsburg 15 LeFlore 10 p Haskell Pushmatama 5 EXHIBIT E-V

Economic Development Administration

WAIVER LETTER

U.S. DEPARTMENT OF COMMERCE Economic Development Administration

March 29, 1973

Dear Sir: The .Economic Development Administration hasemployed MACRO SYSTEnS, INC., to conduct an evaluationof the Occupational Training Information System (OTIS) currentlyoperating in Oklahoma. The purpose of this evaluation is todetermine the strengths and weaknesses of OTIS and torecommend improvements, where necessary. The results of this study should benefitall employers and employees in Oklahoma.

In order to conduct this research,the contractor requires assistance from all employers selected toparticipate in the original OTIS survey conducted bythe Oklahoma Employment Security Commission (OESC) in the summerof 1971. Specifically, we need: questionnaire which . Your permission to use the survey was completed by your company orbranch operating in Oklahoma. OESC will only release the surveydocument 3.f you will complete and torward theattached confidentiality release to MACRO SYSTEMS, INC.A stamped, self-addressed envelope is included for yourconvenience.

. Your present employment levelby individual occupation 'code. After a review of your questionnaire,the contractor will contact you toascertain your employment level. This will be done by a trained personnel spe- Cialist and should not require more than onehalf hour cif your time. The inforMation is criticalfor the success of the evaluation, and all current data will be subject tothe same confidentiality constraints as the original,data.

Due to`, the short period oftime available, we ask that you complete the release and return it as soon aspossible. We believe the results of this studywill be mutually beneficial. ConsequeatlyA we sincerely hope you willassist us in the effort.

Sine rgrly,

He S. Becker Director, Office of Administration and Program Arltlysis

Enclosure EXHIBIT E-VI

Economic Development Administration

WAIVER FORM

I hereby authorize :MACRO SYSTEMS, INC.. to examine our 1971OTIS qut:stionnaire on file with the Oklahoma Employment Security Commission. I understand the data will remain confidential.

Signature

NAME:

ADDRI7SS:

TrI, pii()AE :.:1MBER AND AREA CODE: EXHIBIT E-VII

BEST COPY P.MIRLE '..,onomic Development AcIndnistration

OKLAHOMA OTIS SURVEY DESCRIPTION

I am representing MacroSystems, inc., under contract with the U. S. Department of Commerce,Economic Development Admini- stration to study manpowerforecasting techniques used by the Occupational Training InformationSystem (OTIS) operating in Oklahoma. OTIS is a computer basedinformation system designed to effectively supply the required trained manpowerto the Oklahoma labor pool.The system is operated by the StateDepartment of Vocational and Technical Educationin Stillwater. This survey is essential tocomplete the study and should not require much time. The purposeof the study is to determine the current level of employment byspecific occupation codes and is identical to the survey conducted duringthe summer of 1971. Any questions about our authority toconduct the survey should be referred to: M. Will Bowman Chief of Research and Planning Oklahoma Employment SecurityCommission Oklahoma City, Oklahoma Telephone: Ac405/521-3735 EXHIBIT E-VIII(I)

Economic Development Administration

BE51 cCRI"'"" SAMPLE RESURVEY FORM

43S7 WI* Amovut FpLrost 7i i/73 0. No. Use ?h..Spec. FINANCE It 1,IPANCE-REAL ESTATE SCHEDULE OKLAtiamA OCCUFATION4L NEEDS e 4, ft, -Doe SURYEY QUESTiOtittAtRE

Reven Niel,eftss Sreste Emplorrelt Servo.* Research and Flo-nm; O.v 60m to,11 Rove* Mernor,nr OkInhatra Cr,.t O4tatO,K3 73105 .523 3735 Area Co'e 405

COTE. Before InecrIttg the f2,:q.+csfe f Inor-ation, Pase. Readthe lttgructtons ttacked to This Schedule.

....._ . .7ev,,,Torecistect 0 ' :JOE OCCJPATIOS CoITC At Emplorneut Comments Empler.,c-it roe llitle .73 (Deserilw Alt rossible Variationtt . , . --,r 3 t41 l'4) Pk?. fIP:SNAL reCNNICAL AND aPANAG,:t IA. OCCUP4f4t4$ it P.C. Pte0C. t92 1,4 . 1ItS cctRI. CIrt.!. rIt14 --,. e. ,, 4 rs 1, t 7 :tft ,,1,4I c !41.4 fstocr ....,, or.t....,0 . .f, s'1 P. t.ca%t" c5, 4r 1.I It CI 022.It ,st 4: "Ott' w. a... Ia..,( *. tcC ---....-- -1- ----t otr,.s.,:t 1 tr I 4:1 CI.) ,r(...--(..tits, RtCTME1 -.-..---..... (5t. Ne14(F. %PR sr.tt4 41PI 01..tt TIOVS t ! iar ci ..1 ' t h %Pt. tPe R.tt 41.4 *fee UT

i.., 'lit 1t: t,RK -. I,.. ...A I' t bat 411..0, tt AN --.... ------,... ----_----i . 1 $. , VI ,...C1,I I?tett. C Ittp ,:.,.t s , 1it AL lc.1 s. I' .ts.0 ,(s((.1( ...--..--.... I 4 ,...1 Ice 14i ct t At.I IR ...... -= ltt, in I s tI TN MA% to . R ttC it --__.t .: 0.tIL t1(,I It c IOR ------If.- It4 tI US(_--- 0 4t1111. R ,,,,I.,.. iI I trt 1 4.11* -...... ------1.'1 1..0: Pt ClIlt.t .t.It 1.1. tc,..,)I. tsh, ( AJO't (I 1.&:_(.. 1(ts. ts.---. f ICI 11 i ,.n,! 1 tits, tr;t %It ,.? 1 F'Mt F.

,Cl.:,- .'P I I. N. t oI I to i CL (WC, AND SAGES ErCCUPAraNi :",I. s .E 4fe:KrT tut' ..:.,.. 171t.1, '..,%NI......

--"N

9 ,.t 1IP.I% 1 I 0 _- ...... ,...------2 t..1iIi111I Ik ------...... -...... ,------....----,-- l 1 t ,I 0,.. 114: , . .. It ....--.. :it, -- - 7 ht. % 1 11I, lo, 1016 ...1....

il 1 01.. ft%11,11t Itst. , 1,1'1. o ^ ^l ..M ro...... -....

o It o I jo t I1 o. f oto----I ,to t RtIttle

...... -

I :I.. I. P5 ... 1 Ilt .{. 1...../feIl\tI .1,...

:1,1 o . et 1io I S t 1 "fife I If iM1110.11.1$ DIN R $e star MCI P IDCH1131T E-V111(2)

PAI110.11 ..

FINANCEINSURANCEREALESTATE SCHEDULE(Cent.)

Nor. Faeces Steet Comments D is OCCUPATION Current f mllorment COPE Ettirloyment For funs.'7, (Dss-rihe All Possible V4rtations) i4), LS) ---7, i tlt

CL&R1CAL4ftds SAEStsCcto..4 nolo ic.*.) ' 4.41:etsitLr (UNI A, of rstt . ;1. %.,4 ytt te4: rt folk I :1`e 4 .41 4 4.1 61'C t 1 r Art f /i ttf et S,I, otij m%),su ,P ,,, 4 11 NI :11 1 .'

14 ,l 4 I {4114 fI p1th .1%t. ftvss r 1

:9 %... N S As111St I I f tt .4, t t1.04% 4.I 4,1.14 ..-.--.-...,!Aue4), .01i I 1.,,IMVW, ...--,.. t . 7 . 1 11 0 s lt tss. ost sits.s ttcqtr. sc.I si Art :; a, ajo.i 1sst (... N1 4tMatti SERVICt rcctiPArtoms

s*2.14.40Sf etRi is C.+ (RD or AT(t.stAst

1I.14(0Ci ,t11tit%% w d t07.A6..t,GAM NOSh I FAIR STRUCTVPA4, ironic ocectrArtams rsq.s!st0stAttATfNAst-F stAv Fttli OrsG

AOSCPI.LANEOUS OCCUPATIONS 1140.2 AENGEt flt STATIONARY .., --.----..--.-. :7 :. ; r.: 1.:. nr ".." :.-.: :-.N.,t,!:...? .."..1- t's.`....'.`"t" ---....---...... -

000 000 0TOTAI t ...NA's sit NT OF ECTA0LIIIITALNT

OA SIGMA 7URI Ilfet 4414.81,0 1848 Arisswft

PripNE TITLE

PAGE .1, APPENDIX E(6)

Obtained current employment by occupation Compared forecasts and actual employment Attempted to ascertain the rea ms for any discrepancies Some of the reasons given for forecast err(s are summarized below. Reasons for over forecasting: Lost bid for work Decreased work Overhired Eliminated one shift Changed to subcontracts Reasons for under forecasting: More business New large contract Reasons for shifts in occupations: - Realigned duties - Error in original forectists - Reclassified employees - Shifted work to another plant

-(2) Kentucky Resurvey Similar to Oklahoma, resurvey interviewers in Kentucky were given an introduction guide. The Kentucky guide is included in Exhibit IX, following this page.However, the actual data collection was conducted using printouts from the original OTIS survey. A sample printout is included in Exhibit X, following Exhibit IX.Using the printout, each interviewer: Obtained current employment by occupation Compared actual and forecast employment Sought to determine the reason for discrepancies The reasons given for discrepancies included. Reasons for over forecasting: Closed department Business lagging EXHIBIT E-IX

Economic Development Administration

KENTUCKY OTIS SURVEY DESCRIPTION

Macro Systems, Inc. is under contract withthe U. S. Department of Commerce, Economic DevelopmentAdministration to study manpower forecasting techniquesused by the Occupational Training information System (OTIS) operatingin Kentucky. OTIS is a computer based information systemdesigned to effectively supply the required trained manpower tothe Kentucky labor pool.The system is operated by the Kentucky Bureauof Vocational Education. This su-vey is essential to complete the studyand should not require much time. The purpose of the studyis to determine the current level of employment by specificoccupation codes and is identical to the survey conducted last fall by theBureau of Vocatiopl Education (OTIS). o- Any questions about our authority to conduct the surveyshould be referred to: Dr. J. Jones Director of Information Services Kentucky Bureau of Vocational Education Frankfort, Kentucky Telephone: 502/564-3096 plIOV plIeM F &0110T/liC DevelopmentSIT PAC(1) Administiation THE KENTUCKY 5TATE DEPARTMENT OF topEATION RURERU OF %IDEATIONALTM. EDUCATIOh 0EMINO AA.tpuaLk N010,1.1 SAMPLE OTIS KENTUCKY PRINTOUT COUNT/KloolONSTATE KENIuCKY USA06 EDIT ROSTER kT OTAtiLISOENT Of COM/73 3U6700 0 09CODEFIKN INSTkomENT NUNOEk CODE3433St' EULLELT1uh DAIL FINN NAME STREET AD1)$E5S 490 PAuk OTT HAKE 1 OF 1 Zip REPNESENTATIvE 11/7Z PHONE _ KUN,Lit ITEM __ 040E040 NumbEN _LS - OPUENTICE NUNuER kumtan Imp JUNE NEATLAPECTE0 JUNE 4 YRSEIEPt.Oku UGCUuU 004+030oUo0 01300 U000 UOL30000 JCL3000u 041,101J63 _ lieuU14 ._ 00019U0001 OOnuo ______-DODO0uuUOuOU uu01Oullo 4419 06uWO0/9LII 004120042900J3300007 0O .. 0ouo0E10000004000 Eiu00U000U0u00000 uol4v02,0o33000/ 04..120Q790103OiJu7 -U71 13/ .63 --. 1/10400Oots0214...0 0 0uuaU01,0WIDU ijoububbuUt)uuu000 Dolt,0001.1uut.'0004 awaGliii..00044 U74u/U0/4 091101Oodie004;03 ... 0 UoUo00U0 oU0uuuU WA/0403 ov*7Ouu3oula . Ouuo .. 13119uu*06U 000SU0413Oci.OS 00 -- ______tU0000Uoub _ 9UOU404000040 .. u1il3LOISUu4'.., uuur,QUO% 099U94U9109D ^ barn°000010001'Joao!, D 00 0040pup°06u00:140 ouoU001J0004UC0(10 00U1U01000060000 OuJiout,*uul300000018 099UVS41'lb ;Jcuuluulaulu0d.,3(00440 0U 004.0.uuuuuouoUquo 0000twoo1.01..0OUou .._ _ 001..10u0113:.310003 0443Oi.60Uoul064,1 425WI /About04uul 0 . 410p00040 00b0%mull u401u1J01 nuik30u010001 IIBIT D-X12) Tit - $0.00(0 STAIL U.aRTMt4;b0hi6U ur 1.00CAT1ONOF VO(hil.j1+41.EDI, 90sTER7t4t E.00tATIOhof otrANv0160450104T tikNeOfttH 50WILY hur Lh IL Li1PLOYLO hip- Ew LS APPR.0T1CL_, Nek) +WI t01140EM hot, 40NL ULA1EXPLCTL0 404. ,4 EXPI01.0 01302 Y10.0 141:4014% 1.3011.131.),J406000;i2Ujki01 0 0)00U PULJJVU 0000U0000000 000100030006 00000030006 'IV:b1 !.)0U!,401 Jul 00 ______'U W1 0U)00 WU '0000 0000 0228 0001V0U4 Ue28 0002000 a Oit1L(A.hAtrii^.1.MO.4%.AL 101.10 LOAL 0041.41.yuzlziu4S6 0;020100 1100060UU6 006ulia oud12uuld,

JiAr APPENDIX E(7)

Reasons for under forecasting: Picked up new account Opened new department Added a shift Bought another company Reasons for shifts in occupations: New production lines Change in production methods Change in product line

DATA PROCESSING The processing of the resurvey data included an initial edit, keypunching, computer editing, and a complete review of raw data priorlo analyses. Each of these operations is described below.

(1) Initial Edit As the resurvey documents were returned to MSI, they were individ- ually reviewed for completeness and clarity.All data that were inadequate were sent back to the field and completed, if possible, or omitted fromthe resurvey, if still incomplete. After this initial edit, the documents were submitted for machine coding. The specifications utilized by EDA to main- tain the original Kentucky OTIS data were utilized for coding the resurvey data of both states.

(2) Keypunching And Data Verification The keypunching of the resurvey documents required over 8,000 cards. Because of the large volume of cards, the difficult nature of the keypunching, and the need for maximum accuracy, severtl test runs were developed to verify accuracy. After all discernable keypunch eriors were removed, a line-by-line check of the data was initiated. Each employer's survey document was matched against the computer printout for that company. The following data were checked on an individual line basis: Region Company name and address APPENDIX E(8)

Resurvey date Survey document number Occupation line item code Description Original employment Forecast employment Actual employment Control totals Only after these edits were complete were thedata submitted to the Com- mittee for review.

(3) Raw Data Check Printouts of the resurt.ey data weresubmitted to representatives of the Project Monitoring Committeefor review. These represented the resurvey data line item byline item and indicated potential problems and/or questions.Subsequently, the printouts were matched againstthe survey instruments andall questions reviewed with the EDA Project Officer.After this final review and a series ofcorrection runs, the data were arrayed for analysis. APPENDIX, F

RESURVEY DATA ANALYSIS APPENDIX F(1)

RESURVEY DATA ANALYSIS

The resurvey of employersparticipating in the original OTIS survey in Oklahoma and Kentucky provided a comparisonof actual and forecast employment by occupation for eachresurveyed employer. These data wereaggregated in different arrays to compare theactual and forecast employment by occupation by region and statewide.Consequently, it was necessary to select relevant sta- tisticsto test the forecast accuracy.After extensive analysis, the Project Mon- itoring Committee decided to use a setof three statistics to evaluate the accuracy of the OTIS demand data. Each of thesestatistics, including the derivation, limits, special cases, and interpretations, isdescribed below.

1. U STATISTIC The U Statistic was developed by HenriTheilk and has been used previously to evaluate the accuracy of employerforecast of manpower demand.atThe formula for the U Statistic is:

U . where P and A are the predictedemployment change and actual employment change, respectively. The U Statistic measures theseriousness of a given forecast error by its square. For example,the mean square prediction error for a setof n observa- tions is: A)z

The root mean square (RMS)prediction error is obtained by taking the square root of the above mean squareprediction error. Thus, the RMS has the same dimensions as the predictions andrealizations.

I! Theil, Henri, Applied Economic Porecastitvz, Rand McNally & Company,Chicago, 1966. gi Moser, Collette H. , An Evaluation Of Area Skill Surveys As A BasisFor Manpower Policies, an unpublithed doctoral dissertation at the University of Wisconsin, 1971.

, APPENDIX F(2)

The U Stattic, or inequality coefficient of n pairs, is obtained by dividing the RMS prediction error by the square root of themean successive differ- enrte of realizations. The result is the positive square root of:

n P. -A.)2 r 1 1 i =1 A22 z where P. and A. are predicted and actual changes in employment. Thus, U=0 only if tike fore&tsts are all perfect, i.e., perfect for all occupations.Also. U=1 when the prediction procedure leads to the same RMSerror as a naive no- change extrapolation.In summary, by using the inequality coefficient, one measures the seriousness c" a prediction error by the quadratic loss criterion in such a way that the zero corresponds to perfection and the unit with the loss associated with a no-change extrapolation.It will be clear that the inequality coefficient has no finite upper bound, which is tantamount to saying that it is possible to do considerably worse than by extrapolating on a no-change basis. However, the U value of an incorrect no-change forecast is 1.0 regardless of the magnitude of the error because the original and forecast employmentare equal. For computational purposes, the U Statistic was reduced by substitutions, as follows:

CP- A T2 )2 A. i=1 where P. and A. equal predicted and actual changes, respectively.It can be seen fromthiLiithat:

pP. o where F. and o are predicted and original employment, respectively; and that A. - a - 1 where a and o are the actual and original employment, respectively.Substitu- tion shows that: c 4.

APPENDIX F(3)

Ir(p o)(a - o)12 U2 (a - 0)2

E (p (a

2 i(a ci)? Analysis of this formula indicates that several special cases can occur. When the actual level of employment determined by the resurvey is the same as the original level of employment, the denominator of the formula computes as zero.Since division by zero is undefined, the following criteria were selected if a-o 0:

If E ( p- a )0 then U was set equal to zero, representing a perfect forecast. If E (p-a)/0 then U was originally set to coto correspond to the commonly accepted definition of a real umber divided by zero.In order to complete regression com utations, all eases of U.«) were redefined as 10.0.I.ogica is means that an error value of CO percent was redefined to eq1 1000 percent. ALoiher special case occurs whets the numerator of the formula'ormula -iszero. When this happens: If I (a-ol- 0 then U was set equal to zero as above.

If!(a -o)/0 then the U was set equal to zero.The basis for this was a perfect forecast being made as shown by r(p-a) being zero.

2. STATISTIC r Statistic I is a relatively simple statistic that relates the forecast error to total actual employment. The formula for this statistic is: Total Forecast - Total Actual Statistic I Total Actual APPENDIX F(4)

Statistir is computed using only total forecast and total actual employment of the employer or particular aggregatann.Thus, errors in the individual occupational forecast comprising the total forecast can be offset. Special cases occur when the employer either went out of businessor forecasted going out of business. Those cases were handled as follows: If actual employment was zero or the company was out of business and the forecast employment was zero, then:

00-0 Statistic I = z

If actual employment was zero or the company was out of business and the forecast was positive, then:

-0 Statistic I = = 03(reduced to 10.0)

3. STATISTIC II Statistic II relates the forecast error to the actual change in employment over the forecast period and is computed as follows: Total Forecast - Total Actual Statistic IITotal Actual - Total Original Similar to Statistic I, this indicator allows canceling of internal errors and considers only total em,Aoyment of the employer or other aggregation.In the case of a no-change forecast, Statistic II is always 1.0 because total forecasts employment equals total original employment. Special cases occur as follows: If actual employment was zero and forecast employment was zero, then: 0-0 Statistic II0-x 0 If actual employment was zero and forecast employment was not zero, then:

Statistic. IIy0= real number 0x-- APPENDIX F(5)

If forecast employment was zero and actual employment was not zero, then:

Statistic IIy-x=y-x= real number If original employment equals actual employment, then:

Statistic II = (reduced to 10.0) x-x 0 APPENDIX G

DISTRIBUTION BY REGION FOR EACH STATISTIC

F. APPENDIX G-I Economic DevelopmentSAMPLE Administration EMFLOYER PRINTOUT OKLAHOMA 3:2 -v UCCVPATIONAL TRAINING INFORMAT/7N MACRO RESv4YET REPORT SYSTEM AS OF PAGE20 DEC 73- 28 T11 Si ;Tt L I 00(1..4140MA 2 fONIGINA6 SURiLY 'ATE F4ReCAST PLNIO RLSURiLT uArF. 22673871 INSTRUMENT SIC CODE NO. 4)0 OCCVPATIONAL CURRENT CATEGORIESEmPLOMENT 44 238 FIRMDuT lAME 314 J,tIGINAL STiEEt ADDRESS FUqECAsT ABSIFORECST rUKECAST CITY NAME ZIP ACTUAL REPRESEUTATIVE ACTUAL ASSIACTUAL ACTUAL` PION( e.*85108161NA 319/.4CCO.A rC06,UCCeAT.ON SLR SUPER EMPLOTAC:.T 3 "'DmIGINAL - ORIGINAL) EmPLOTPEST 0 0 i FORECAST EMPLOYMENT 0 3 FORECAST, 0 ORIGINAL 0 ACTUAL) 0 r 32:1300 MOWJEKLO 0CC 16 0 00 VI16 0 6416 U U0 0 361oW.I0'LAti4WiT0h47/..1 PER '4URSL AIDE040 6q 1 0 0 I 0 I 0 0 0 09/40J3liiloo4U AAINT.blikA,ils.ECPER Ham, avILD 81 p0 0 0 0 81 U 0 0 TOTAL aTATSTATIJstAl 216 .5909.0546.9369 35 35 251 .13 238 25 22 32 APPENDIX Economic DevelopmentSAMPLE Administration EMPLOYER PRINTOUT-- KENTUCKY CcUPATIONAL TRAINING INFORMATION MACF10 RESURVEY REPORT SYSTEM AS OF 2 DEC 73 679PAGE II! RL4IONSTATEFl4M NAhE KEmILCKY 4 uNIGINAL SOmV0 UATL FORCCAST PLKI00 RESONVO DATE, STREET ADDRESS 674972 9 INSTRUMENT NO. I9D SIC CODE 3622 CITY NAME OCCUPATIONAL CATEGORIES CURRENT EMPLOYMENTZIP REPRESENTATIVE 66 PHONE Gui EIPLOCNILNTonlbIAAL "0416 VIALF001CCA5T ORIGINALTARSIFORECST EMPLOYMENT FORECAST FORECAST EmPtoTmENTACTUAL - FORECAST! ACTUAL ABSIACTUAL ORIGINALACTUAL ORIGINAC/ AOSIACTUAL CV" 1-AYAOLLOCLUPATIoN CLERK 0 0 4 0 I 0 0 0 6133291.31 PEA 4,414;:L CLt.toc o 0 4 , ---\.1 0 I U 03 30 614,21 eROCUALmEil LLLAK 53 0O 00 SS 3 I 66 3 s 1 ! 61115 COmPoTt4ACCOtNr146 °PE.; CLLK 2 O 0 2 2 0I 2 2 62 01147 hi.TroqCn OPEg 7 0 0 97 "Tb I I 6 I 1 ! 92v3361 I'9 PRUOUCriONALL OTH OFFC CLERK MACH OP 02 0 0I I 94 43 0 R3 42 R2 3 62940 ALL UT0 ;LEAK:: TOTAL 6.43 3 se 0 se 0 703 a .22 3 671 0 3511 a 363 33 U 3TATIVICSTATISTICSTATI5TIC 4 . 959710344U IFESTACTI/ACT IfCsTIAAETIttAcTA,ORIG$ APPENDIX G-III

Economic Development Administration VIMINOS. COSY tS'i DISTRIBUTION OF THE. U STATISTIC BY REGION IN OKLAHOMA

Number of Employers with a U Region 6 7 8 9 10 11 Total Statistic of: 1 2 3 4 5

5 15 0 2 1 2 4 1

17 2 .0001 .1999 1 1 - I - 3 .2000 -.3999 - - - - - 2

2 6 1 12 .4000 -.5999 1 1 1

4 2 - 13 .6000 -.7999 - 1 1 3 - 2

8 4 8 1 1 1 30 .8000-.9999 1 4 1 1 -

16 10 16 1 5 3 72 1.0000 4 3 1 6 7

5 - - 1 19 1. 0001 - 1.2500 - 2 - - 1 7 3

2 1 - 2 13 1.2501 - 1.5000 1 1 1 - - 5 -

1 - 2 1.5001 - 1.7` - I - -

1 - - 3 1.7501- 2.0000- 2 q . - - -

4 - 1 9 Over 2.0000 1 2 1 _

44 2c 52 6 7 8 193 Totals by Region 8 16 =7 11 '12 = APPENDIX G-IV

Economic Development Administration t;1". DISTRIBUTION OF STATISTIC I BY REGION IN OKLAHOMA

Number of Employers Region With a - _ --a Statistic 2 of 1 2 3 4 5 6 7 j 8 9 10 11 Total

Over 1.000 .... st m tom =Oa ORM et* O O. M do O. V.

OD do on = to PO. fa = 1 .901 - 1.000 to as to . 1 TOO

o Ot . ow -- -- 3 .801 -.900 te OD IO O.. 3 -- -- .701 -.800 P.M

.601 -.700 Pt. I OP. abm Otte I

etr ...... 301 -.600 ... m, Ot -- ..... 2 -- 2 -- 4

.431 -.300 -- .... 0. Os fa es w ...... a. go..

Ob. 3 .401 -.450 1 ------I I --

o. we Of. Or at Atm . 332 .400 ow.. At et op em 1 1 2

. 302 -.350 .0 -- to .. 1 ...... 1 .r. I -- 3

NO et et/ 5 .231- .300 -- 1 1 I I I -- -- 6 .201 -.250 -- 2 ..,, -- 1 I 1 I -... --

1 9 .151 -:200 ...... 1 .... 2 -- 4 1 --

.101 -.150 -- 2 ... I -- 1 2 I -- 1 -- a .... 18 .051 -.100 2 1 1 2 -- 6 1 4 -- I

.001 -.050 -- 2 -- 2 2 5 2 8 -- I I 23

0 .... 2 I ... 3 6 4 6 2 ... 2 26

0.111, a ado 2 .1 21 .001 -.050 1 I 2 6 3 6 1

3 .1. . et,* 12 .051 .100 1 I I . .. 2 I 1 2

. 101 -.150 2 1 .... I 1 -.. 6 1 12

.151 -.200 -- 2 .... 2 -- 2 1 .. ... -- 7

1 3 .... atelle 7 .201 -.250 1 te 1 -- 1

to.m, .231 -.300 Men 01... -- = = emo. -- 1 Olt.. O. ... 2

mem M 2 .101 - .350 a% wo m .11 ea Oa . 0 2 ... tn.

Otte .111 ODIO itt. 2 .351 -.400 tto to ------1 I

.401 -.4S0 to a mem. tOto 1 oeve ... vow. .451 ...500 II -- .. I 4.08. wet -- .3 N.. 2 2 .501 - .600 -- . 46 eft.* -- ...I I -- --

.601 .700 m pa I 00 Mt m Pa a, we MOP 2

O. ok awe so. moo 1 .701 -.800 .4. MOB 1 .. a..

dam *le tow 2 .801 -.900 we. ... 1 -- I --

ow m a ab ON, OD . 1 tom 2 . 901 1.000% Itt OS a.. No. -. 1

ea ow Otte ...... 4. .... 3 Over 1.000 1 -- I I

Totals by Region 8 16 7 it 12 22 S2 7 193 91= EOM Si= A an APPENDIX Co.V

Economic Development Administration

DISTRIBUTION OF STATISTIC II BEST CPPYPAILIZLE BY REGION IN OKLAHOMA

Number of Employers Region With a 2 1 I S 10 Statistic of: 2 I 6 7 11 1 otal

»1111 .0. .011 3 CO .. -- a

1 Over 2. Ci:.70 1 01. 1 1 -- 10 0... 1.7;1 - 2.000 .11. 1 -- -- s. 2 3 1.501 - 1.740 -- ...... I .... -- 0.... 2 1.251- 2.500 Ob. ...0 M...... -- 2 2 1 5

1.001 - 2.250 ..0 .00 Va. 2 1 2 ...... -- 5

.901- 1.000 S 4 2 7 6 18 10 18 1 5 1 77

...... 10. 1 ob. 3 .801 - :400 -- 1 -- 1

. 06 .0. Y. *ma S . 701 - 800 wt. -- -- 1 1 1

.601 .700 .0 1 1 1 ... 1' M. 01110 M. ... M.e 4

0... 1 .m.. so = ... 4 .502 -.600 1 1 --

11101 010. M. 1 7 .451 -. 500 ..0. 1 -- 0 . 401 .4SC. . I . .. I I ..... -- . ak, .... 3 . ;51 .400 ..* 40. -- .. me. 2 -- ...... 2 ...... 302 . 350 .... -- ...... ! ......

.251 -. 100 .... -- mom 111114. .. w......

. 201 -.250 00 .. 10111 .... 0 mt M. wal MIN

.151 -.200 OM* ...... 110 .. .40 aM0 mom .1.0 Oft .101 -.150 ...... 1.= .. -- .0. .. 2

01. WM M. .051 -.100 1111. ... 40. 1 1 .001 .050

0 ma. 1 2 7 2 -- 2 26

001 .050 MR 40.0 ... mar mem .. 1.0. M. .051 -.100

. de . 201 - so .40 1111.. 4100. Ma! -- M. 1 -. . 151 -.200 . 201 -.250

.251 -.300 am. .. 1 1111. .. am. ems

.301-.350 .1. 1. 1. el. re 1 1000

351 -.400 OM. 4..6 Ms ..... M. 1 10. .. .. mat .401 -.450 .. am= .40 M. .. .1. 00. .0. .. .0.

.451 .500 a0 .0. .00 .. 1 .... .1.

M. of . M. Im . .10 .50E -.600 .1. 1 -- E. ..

/MO M. .601 -.700 ... .M. .10 wa .. 1

moo .4. ... .701 -.800 WM . ... .0 0. .. .801 ..900

...... 1 S . 902 - 1.000 Oa. 1 -- .110 2 2 .... Mt 10. 1.001 - 2.250 010. -- M. 1.1. -- ... 1 1 2

1.251 - 1.500 ... .. amo ... 011. 1 .1.0 1 wt. 2

Me. 11 11. omm. 2 1. 501 1.750 .110 1 aom .... Mt* 1 --

=MI 01. am. 000 wee 11.10 1.7512.000 1 M. 40. M. .01 Mr. 7 Over 2.000 1 0.. 2 ...... -- 1

. . Totals by Region 8 16 7 11 12 44 22 52 6 7 8 193 ssi 91111 MX all w ass

ti ."' .. I

APPENDIX G-VI

Economic Develoinnent Administration po-A, er-f" DISTRIBUTION OF THE U STATISTIC BY REGION IN KENTUCKY

Number of Employers With a IT Region Statistic of: 1 2 3 4 S 6 7 8 9 10 14 15 Total

1 5 4 1 16 2 1 1 1 1 33

.0001 -.1999 - - - - .. ------

.2000 -.3999 - - - - GO Ile MI - - 1 M. 1 2

.4000 -.5999 - 1 - - .. I - .. OD - .. 1 3

.6000 -.7999 1 1 - 2 - 2 1 r., . - 1 1 9

.8000 -.9999 1 1 1 4 1 8 2 1 - - - 6 2S

1.0000 1 8 11 11 2 64 11 - 2 1 6 20 137

1.0001 - 1. 2 SOO 3 2 3 2 - 10 8 2 - 3 1 15 49

1.2501 - 1.5000 1 1 1 1 - S 3 - - - - 3 15

1. 5001 1. 7500 - - 1 1 - 1 .. - .. - 1. 4

1. 7501 - 2. 0000 1 - 2 - - 1 1 - - 1 .. 1 7

Over 2.0000 - - 3 2 1 2 3 :1 - - 1 5 17 MM. 01.1.10 OM 1110

Totals by Region 2. 19 26 23 2 110 31 ri A Z 2 SS 301 s.

APPENDIX 4.1.Arl I

Economic Development Mm istrat ion BE.S1 Cut'i ;'! D1 AtIIIIITION OF STATIS1 IC 1 BY REGION IN KENTUCKY

Number of irmployers -Re ion With a 14 IS 1 4 S 6 7 8 9 20 Total Statisticof 1 2

-- 2 ... -- ...... 3 CO - .. .11 Over 1.000 .901 - 1.000

n10 Rol. .101 0. .901 .900 = . .01 00. M. IR. at.

. 701 44.800 .601 -.700

MIR 1 I 3 10 om 0g. . .0 RR rm. .10 -- . 501 -.600 1 I% 40 ..... IR. 2 ow. = . 2 ...... -- -. .451 -.500 I° I° .1.0 M. IR. 01.1. 10 ...... 401 -.4fi0 0IR = 1 .. S N... 1 1 . 351 -.400 RR 4.10 I 0.. .. ------0g. 3 111110P IR. .R. .301 -.350 .. oft .0 1 1 1 / ..... 2 2 -- -- 2 2 .251 -.100 1 ......

01,.. ... 2 2 .201 -.250 -- 2 1 1

RIR da 1110111 1 -- 10. -- .151 -.200 .... -- 1 1 25 7 1 06. .... IR. 3 .101 -.150 -- I 1 1 1 .... -- 2 1 26 .051 -.100 -- 6 -- a II 3 1 19 4 ...... -- 10 36 .002 -.050 1 -- 2 1 3 .... 1 42 2 16 2 1 2 0 1 8 7 -...... 27 3 4.. 15 2 -- -. 2 5 .001 4..050 -- _ -- .. MI. RR. CI 8 32 .051 -.100 -- 1 -- 14 4 .. 20 1 3 4 IR. -. 1 1 4 .101-.150 2 1 2 1 11 3 3 1 .... -- -- O-- .151 -.200 1 3 7 sw.. I I alb . 2 -- 2 .201 -.250 -- 1 .0, .... -...... S 11 .251 -.300 ... 3 2 ...... 1 .... -- ..... 1 6 .301 .350 -- - Im. 2 .... 3 -- ...... 1 2 6 .351 -.400 ------.06 .01 2 1 -... 2 ...... 1 .401 -.450 1 ... 2 ------

MAI IR. Mi I1110 3 .451 .4..500 ...... -- 1 2 -

ROI ORR .4 ._ a 6 .502 -.600 -- I / I I -- -- OMR 1 2 .601 -.700 -- -- I ... -- ..... -- -- wag .0. 10. ... .701 -.800 .0e /0 . .1. MR RIR M. 0 w 0.0 .a. M. ... sOm .. 40. .801 -.900 el.. MOO IOW .... sada OW= a PI m. I 01.. M. .901 - 1.000 R. 1 RI. OW 2 6 10 = I I I Over 1.000 1 .. ... 55 301 S 110 31 4 3 7 9 Totals by Region 4 19 26 23 Nift Mali SIMP Sii ss 111111

11. APPENDIX CoeVIII

ot Economic Development Administration DISTRIBUTION OE STATISTIC U R\' REGION IN KENTUCKY

of virloyers Region With 6 . 8 9 10 14 IS Total Statistic. II of: 1 2 3 4 5

S 4 2 .. 1 I 26 1 -- 2 -. -- 3 1 4 IS OVV r 2.000 sw , 2 2 - 4 1 .. - c 7 2 -- I -- 3 1. 751 - 2.000 - -- - 1 ------Ob. .. 3 -- 2 1 -_ .. -- 1. `,4)) - 1. 750 ------4 .... ft. -. 6 1 1 .. 1.251 - 1.500 2 I I .- -- 1 4 10 -- We. -- I 2 .. ! - 1.001 - 1.210 1 .. 124 11 1 5 17 901 1 000 2 7 S. 12 1 60 - -. ... 3 2 .. .. -- . . 900 .. 1 -- - .. 2 wft .4 40. Mew M. I I 701 - SOX) .44 -- -- S 2 -. ... 4.4/4 2 .tx11 . 700 .44 .. I .. -- fr. 1 S 1 1 -- .. 1 .. 501 . -- 2 1 2 ...... -- ..... 1 4 . 45I 500 ...... I -- ...... -...... 401 -.450 .. -- ...... - 1 3 .. 1 -- .. -- . 151 .400 -- I -- -- ...... -. 2 . 101 . .... I ... I -- ...... 252 . 300 ...... -- .. -. .. .. 4 ... 1 2 ...... 201 .250 1 -- W. 3 1 .. I . 151 . 200 ------I -- ...... I 2 M. I .... =a 101 150 -- moo W. m...... W . 051 -. 100 wft WM .. .44 001 030 3 1 42 1 Ir) 2 1 2 - 0 1 S .001 -.050 .051 -. 100 .. .. - .10I -.150 -..- - - .. - -- . 151 -.200 -- -.250 -- -- .. .- .. 1 - - - .251 -.100 301 -.150 .151 ..400 .401 .430 I w. I .1St- .500 4. mm -- 2 ------m. wD. wft 4.4 Oa. 1 501 -. 600 qs. I Me. -- 3 .601 - ',"00 mm .. 2 .. .. 1 - -- -- .. 2 701 . 800 -- 1 t -- -- .. .. - 1 ROI . 900 .. - ...... 901 - 1.000 - .. .. -- 1.00111.250 4.. 40. .. I 2 1.251 - 1.500 a444 44. I I 1, 501 - 1.750 3 40. Ma .4. .44 .44 M. I 1.751 -2 m. 414140 S IS 1 5 2 I I Over 2.0000 -- 1 -- -

4 3 9 SS 19 26 21 110 =...,31 -- . Totals by Repot; = = ....._ --- APPENDIX H

RESURVEY SUMMARY DATA OKLAHOMA ott000.10 egis0" 00000101 00 esIS 6kL5 is64 00C006 n7.1.)1nI"0ononllonn04,Or,17nI'nn-inr;" stew-51960 0°600410000001CO'JnOncunn41coLc041 00 ogesOSZS nszrphLk rsrEOh/4 onePnCOennrt q;:nql-norloa.1- ttrrO°hr...,e.0- 0nUnOelectio01 0 hhto, 5h44I Sh4ht 1106CnC orj,30t-nqcvnl-o()con.1- OnnnOsSCCS1hL9L6" 0001)04.tootigaitcouncil'OnoofiI a0 OC6COhCh AtCC PcCcCf 005oneOnsnot sa,no.t-nnr1rrcoans- 9ciht,-98?hle2e9126- ont:nnetonuluilorcnrit 0a SCIfPotIt 62Oh Oh62 nnennhonr006 nrnnost.n.,01` Cnn004,ehhC0ZS4404.1*711° pounnitonuooil0001Cetentko 0U 9262 CC92he 92nc4c9c !irenr,sAnh0116 (101- con?s01:24 OnCnn1036n04,tnnurnt 0 szsz1)2 2 9tSCor Cr97Cc one00inog Ou100:-neJ001- CCC:/0CEnS69014-6.2560'-ChCC14- aounaitrounaitpoor:tt0.07Et 0 61CZtz 91etOE9Zst 519161rz OnCOn,nnr006 f.lanOIf"f^c,!;$on:nn OnnO4csv1V-tioczsa,:nnad anucot00u00100000.1uncro. C CtClLi11 CI911121 Ct91/111 nneonenn2nOL nnoonl-rannoi-onion. CLELZ4Gonna' oounnt00000.1anuna.! 0 at0111 6a11 96t t nnr006On6 reonnt-r,pnot-"o(:nn °oncewits onuno.t0ounnitmicas!onann 0 69e al019 tit01nt0 nnp(Jetenqnr9 OnOnne nose-ounsz' c0000itpounaOCUneittont.: 0 IL6 I61 611 00c006ore 00100!Cmu0Col- In.:. oonnot9.2414t',991'tcgers oronooonunoilammo% S9L ztqZL 2rta1 eraoar009006 °DonannUont"nnnonl-nnonOt orono'ousneonnr9d- ononostanunotoncena*tocono 0 Sh CSI. Cg4 nnione006 oonon.1-ronon oarnannn.aonse. 7o Ont:no.00000 0 Z 1 t t2 t r t t nrcon/ nclan ourr:ni C0unOs . nns nc-ln.onnnnt- vnno.01 3N0 aounOst 311SIIIILS 0 1v4ISIO Snwlw 0 s331oldwl Ha9wON st S33AOldw3 1430WftN st S31Loldio 999wnte nos (1)1H XICINEltrAlif )1,13111111 NO10113yw 09k 3 115 1111S Nu L531 dMI$ 0+:)vw n it101/04 lvni3v ac -Or soiNnaISVI1HO! 36ve 1104M90 A3AbAS x14304 v'tnt QUICK QUERY SURVEY 0415INAE EMPLOY1ESmum8EW GAVE PORTCm EMPLOYEI.:5Num:mitFORECAST EMPLOYEESotim0E44cTu41,AO-30 ::tIGLALFORECAST MACRO 04P 4 stAT1bTIC g TEST 4K ourSTATISTIC M4010.01( TenIT6T1STIC APPENDIX 11.-42 ) VUO900WO 5550 8 5550 6 565952 0 : 1.00o00 -.8b1654,03696 .10000o.1.00000 900 54 Sc 11:10 :1+ULu jO 10b112 .0 ..100000 600800500 836672 b36672sq 6567be 0 i 100n0o ,otiGGO.20769 60,7 .1000000.1.00000 .40L bUO30g 7976 76 76 0(DI '1' .Donuo ::.(001 9110300900 987069 90706979 92ac8C964 0 IIITU! .2313-.13750- .012,0 .14,0010c.I.0co0c, .400900700 101 9579 101 7995 ius109 98 00 111::!!!!!3.gun00 ::1:01-321::-.24,138 .04255 :12:=.1.0;:0c,.1.00NOu 90061J4SUO 131119111101 1311111 19 125113110 V00 1,0u0001.c00002.utinu0 ::%7070(Q)-,00182 .1 *00000 700200IOU 134195 85 134195 45 135134130132 0 ::(=1i*OtinUO 60n00 ...37037 44E .1+64010:::=:r61.1.A., :ff0i:.Cil:; 600800700 258139139 . 258139134 146136 0 11::= 99098 q...240?)1 illliCUa 30U700400900 201)195200109 200195109 *9819319101 0 l'cUnDO119.tng 01b,15,47932 903627 ....6G:.0c.2,..1%-u::.1.040J.1.g1,1.pg4 900auu2007uu 920296251t1b s28251290115200 407266261V.2211 p0 1.00ng0 :::23:7391.*05213 ,0;399 .1.1:::(Utt 0!:000 I 600900300 1095 13 1 095 24 3 770 43 1 1 20 if011:1= .00000 .12208.05160 .l'19.(0= ?T2221 8008e0200100200 *0 484 1111 67 5876 532 :.!!!!!l'i44640,27475 5,:inl .00000 .00DGL; 900100 16133 6 49i1 8 10 a0 28-3 21 1,35401 00,00 3,400:10 **J770 1.1.691".s. "brbi; 6 vUO900700300 12101317 i22 91415 1310 256 21:911t4::::::: .81650 1:91= :23::= 110000Q 466661 YOU900500 le*614 231912 1914 .2 15 Iiiiii 9: ,11310.0000Q %.0000C .0g000 400800 2014 2321 2020 33 6tUn0041 59741 ,:b000605000=133 10e0gO00 .5004.40 QUICKsuRvET (MDT gniGthAL EMPLOYEESNUMBER DAVE WWI EMPLOYEESNUMbLiFORECAST 4030EMPLOYEESACTUALAUM8E4t 0101nALFORECAST IUS MACRO QQP TEST 0: STATISTICU Oh(STATISTIC sAcito.ok 160STATISTIC A mull. H4(3) SuO800 3116 3226 2620 10 1 1.200001.82574 .23077'30000 ..192Uatna 1.50000 700900 2251 2453 2827 271 1.00717 .70/11 .14286 .96296 ,1.08333..666o7 300600 303518 363119 29 1 1.414211.00495 .93182 . 344483 .06897 14138 .2.00000..1.16667..9090Y 300900 2825 2927 30 12 I 1.13389 . 403333.10000 ..501,0G 30)300900 343234 1.943733 2 363331 63 1.9343911.g:: 26 .12121.04452 .2.000GQ10.0OCOJ 6004009013 542735 2823 4/40 28 14 t ...928571.545451d0r0la 900600300700 45453,43 48524644 51504845 15 37 I .62678 4.f.0b882.0416701707 .04000 .I..$...S.0f700 '6.000 .s0;;00 30030034U3OU 54494S651 586063 66605950 11 47 12119.19910!1696171.09322 .56796.61721 .0.06700.00U00 ;000 .9666671.33333 90UUGO 90030U740 45546644 586845 ,8 69 7 13IS 23 .1Alsr.99177 2 2436,12121 .00000.00U00 wo9Se.52.045633.0000000GUO 6449002007uU300 112 73 128 0590799 86658176 161217 6 1.31170 .15942 .0394794837.00000.11111 .1.61F:36 1'125002.00'300 .00500 90060070U 101 51es 1131491114 e? 104 9467 631012 2 6:r1Z .00654s51191.00000 38U000U3+84615 .0000U 900900MUU240 104165119 77 117129119182 128113109 12 4810S 1010 g4.97I:17:44;9=9: .57248 9 .00701.0088503$40.62500.0467 20CU..C7692 911111.11111 1*r.:71 700600901,200 190125114 41 1S6130 7050 144191192132 '"70 2116 9 2.519191.00379 .66773.6207 .507049.09091 .01389 10,0C,P0U.993/10 lUb26 200300600 163394137 161138152179 102169166161 .2vs1615 f.080591929135 .9402463246 v.01105-.16867.07317 .1033331.41.00°.11111...4460944 600200600 191110 218199463 396192190 4527 2 ::t3j0;;V8 .03505.14737 1G00U00.939189les;0003 4110 217 214 1 ! 5:::::: .01867 ./.33333 700 172 215218 218 413 I 14q80 .01376 .06522 450

,0 64.%- : A ,

0 Iv 4, cp. ... ,o tr p, (..., A., VC' 0. .0 0C so C.: G C.". to CI 40 4: ,0 .4 N .0 u. to 4- 42 .4 C' CD UN 4o el ... PJ .4 At 4D CN IN to UN - CP C2 J .0 c C.!. oo CI 4. tr Ct C: Jt 4, 4 0., .4 to to io Co Ct N C., ..61 .- U' wo .., Ao .., 10 t: t3 c. ,4 4 .... 0* C. 0 0 4, .4 21 2- us ... CI .0 ,.... W. v CP L.' ,.. 1.0 Ow Li V CJ to .0 02 .4 .12 .., ..-o t.* *0 t.. 4..a C3 to up o` C. C, C C3 C tI C) et 0 O. X 0 %.1 P. Po '0 .0 to I./ *0 4 C C. CD 0 14 X V. .4 42 o tr um to to it cP 4 P o0 c u, .c 4 ...4 Cr tm Cp CD Po .4 0 40 V kJ L. CD OD la C. C.: if V to 4 0 e. .4 $ so 1 I .0 1 1 - 1 I 4 3 4 4 4 4 8 4 4 1 -- 4 1 4 1 4o 2 o . - - 1 4 i ot FA 20 4.) 4 , il 8 t 8 1 1 X .4 2. n 4 4 .. i. le op CP to U' CD tO to 44 co 1.4 so .0 ..1 N 0 Pte Po .4 CD 0 Nu .../ r CD ". 'N U' C3 to 0 oo Po .2 .- ..1 N 4.0 r. an .0 42 CP 42 T4 .0 .4 :) - ..1 CIT UN Z5 o O. to C3 .. 0. 0 io 42 00 CP 0, eti 02 UN /2 .0 .r CO 0 1.1 .... cp ii.4 PA. ... 4. CD C CO Cr CP V tn c: .t a 4t to .... 4 ..- Ca C PV a V a m- - tr CO P. ir ,o 4,0 NJ V . V ... G V X G 11.4 o ...1 -0 4 .. c*, to t., .0 IT 4:1 V. C.: CJ` i 144 CT .. 4w U, .4 kr tl, CI Po to O .... t to tO ta N C3 C7 Cl .0 0 r2 0 .. 0 ... 0 Cl NJ AO 0. 14 0- t7 Pa tO .0 000 0 C) C7 CI C..., O- 0 O ..* 4 i i 4 4 4 4 1 1, i 4 4 I 4 $ $ 8 4 4 IS S r en + DC o% 0

Co to to CD .. . o t o o CD CP h Po CD . P 24 CD lI )C OP 0 4 .t VV a to C: a CD 24 IP I V WM .4 24 Po P4 vh V% r. 11,0 yr ... 40 40 C: Cp .0 CP kw V cr m- 00 00 0- .... op V cr kr C 0 Po 40 go .4 4 to Po co. fr fr Ns. tO 17 ...4 C2 ...... - pj -. fla 4 a. 0 AV ..2 .2 Po 0 0 0 to Jt 44 A.) p4 j. ka .... .0 wi .C. .4 44 CD p4 J, 42 a) co (3 co .1 CD to cp co 4 to .t ro a cm ur ,c c: Po 4 4: to 0: 0. Ct a fr to Ct 40 PJ * i WL C: JP fr .0 .0 rC CD LW .0 ..4 L4 ID CD CD .0 CO 42 43P .t 4, N C: t3 UM .0 15 02 V Ott. V' 'ON 'C 4C' Po %., C3 to tf 42 0 .0 l. '0 VI ...4 0 1.1 ..I APO ira il ' Oa ..0 Ira Mr .4 2/0 -4 "0 C V3

CP to ga 3 N tp C, 0 Jt i QM he to cv tO V to .4 to ,o 00 .0 to D. CM i Po a co 00 D 2 4/ 0. V 00 to 00 .0 4. 0 NJ N 11 Po cr 44 Po CP NJ OD LP $ *V 0) CP 4. to CD 1 Cr 0' a,t 0 -4 f- N N X V/ Z. et s Z 3. et vt 2 C 20 a, T pc CV

co A a tft CP Co VI CP CD Cb CO tO C7 00 00 .0 00 00 P4 7C C 40 O C 02 00 ) .0 3 Co C) 0 WP CD .t 0 CP UR PC l % CC 40 oo Po P4 0.4 to a 4t t .t tr CD N- to to %.1 .4 OD C7 Po CP PO 04 0 co. 0 tr CD At .4 to NO CP C3 rev W4W4wWWNNNNNNNNNNN V0 .4 .0 PO PO 40 4.1 4 411 42 0. ro PPV'aai DD .4 c: s fm 0 t: is r- $ 2 10 .1 C n c lb X' PI

40 .4 - 4 Ul .4 to .41 0 CI tes NO PO .0 0. Oa a; 43 43 V 0. IP 43 CO 0 02 6. 0 0 0 4 4 NN tri C" IP Sr to 0. A* tri tr VNi v 0. w *4 4 4 IP 0 Co 4/' .t ) 4.0 4/I N vt, 4.4 .4 P' N .6. PO to 40 04 to J NNNNN 14 4wP NY11'UTNW 0. 0 V N N N 4/1 .4) rr 411 Z ..... V Z 4'4 P 0 fr r 'V CoX 7 m "el Z 4C C:2 CZ C: IP .0 go CP tn C: PO Po lo 440 C3 a PC .0 .o fr af CP 4 W1t T4 .4 V Po tJtP an Cr 0 0 -C iWiNO P .$ C7 00 .b a 0,1 4 u, f0 fr 42 Aso ;IA .4 se WtmPCPP .t to 0 .t ro ro hi Po PO Po rti f+a r.) r- 4. u ta sPVssWNW O' *42 4P 11/ X to, C . .4 44 tS; r 21 enC CZ 4C

C C.) 00 CD OD CD C2 CD CD C3 C C2 CD CA C3 C CD CI CD CD CD CD CD CD CD CD C" CD CD CCC C' C2 CD 0 CD CD C: C, CD CD CD CD C C 0 CD CD CD CD CD CC C: CD C: CD C. CD C C0CCiCL CD C2 0 CD N N N .0 Jt 40 , N.0 4) tP P N 2 N N N N N.0 N a N to N to PO 4C P of 10 N 0 04 V 4 KENTUCKY

I SURVEYQUICK QUERY ORIGINAL EMPLOYEESNUNDER DAVE PORYCH EMPLOYEESNWM8E4fORECAST ENPLOyEESACTUALAo930umBEA ORIGINALMINUSFORECAST MACRO Q0P TEST OK STATISTICV ONESTATISTIC Ma Ea 0,0K STATISTICTAO APPENDIX H-D(3) 210393270 30 2 30 2 1 021 0 1.00(100 .0%4E000V0U0 20.0cuo0 ,0(40001,00,J00 .1.30000 .00000 270201190 IS 42 15 2N2 932 00 0000001.00000 st;Lor:CG00000 .33334.00000 .00000 - I.GGGnDL .,30000 201208230190 5244 524 54 00 1.00000 .CL:00 11'3000.3'4c01 .3 .,00 la .GCC:J0.L.tdcUu 190 43 43 5 0 3o0c001.00000 .0DcCO .,4000 .40000 .1.:40c,.1.eccou .0oG:..42;00 2su204170193 6$7 675 65 0 AG;cD0 01.0000V000 .16.67.5000CyouUo0 GrA.C.I.)ULACUU 285210190 867 867 87 0 ..00c0D1.0Dc00 .0,-4OCO.04,0u0.0YU0 +.1,0266 *112E6.D4%.00 100cOu.11uCk500 00uGo 27019032* 87 87 98 00 1.0V000 00000 411111..22722 .0GOCD403:00 C:3 2i028$202190 1013lu 5 01310 5 1040 0 1.000001CDCJO1.04000 00GOO ..5r#000 900u00.30CCO .14000o;) -1rn 2852403931VO 1111 49 11 49 101110 0 1.0u:.OU1.000001.00000 -.40060 4,CLULD.10000 "-Vc: 324250190 14II12 9 141112 9 1113II 0 1.030001.00000 .0000000000 -,3c,169 .0.100009091 -100000; .C1C5a4,0003U, 324tab240281 1514 7S 141 75 ISIs14 0 1.01%0001.000u01.000001.04000 -m.44266 .1/4687 .07143 200C% sUC,s.LL 324240301201 24191917 241917 1716 0 1.00c001.000002.0'400 .53333 .4147611765.11765.06250 .1e00"A00.1.QCM:101.1.0;403 210270440190 202119/8 2022IR19 1810 0C 1.000001.030001.60600 03000 .2272705554.4 .I.CUC;53.1.COL:1;.; .51A.0,5 240270190 272320 27232020 2223ap19 o0 100o001.000002.34000 .00000 .22727,142.16.3104C30 .:.uOupo01.00id00wieciOCAOJ .00(000 APPENDIX H-0(2) SVNYEY(141C QUERYORIG146.1. EmPLOTEESNumaE4 DAVE POMTCH fp/MEAS.!EmPLCTELsNtimaLK EmPLOIME5N10185aACTUAL 0e1GINALm:HuSFORECAST 11AcK0 elip TEST 01( STATISTICu sTATIST1C04 PAE40.01( 963474T1STIC 190240270 212's1618 272414le 252423 0 1.000001001(00 .0000(1 -.21739 912500.00000 -10GLC,0.1.00c0c., 0(4;00 393202IVO 2616 262416 29271'626 0 (1000004.000001904000 '0000000000 ...11.345*00141«06000 .0CCOIrOtJUGO .1.0000Q.1.00000.1.0U000 .0ug00.ggg0g 270230213$190 26842731 26842733 33333010 00 1.0:000019000004.04000 00n00 .21212wo100001.90;ID0 .04OL0 .1.:C.G041UCLCu.1*UO:AJJ GOLLIO 2401911224250 604341330 60439030 44431735 0 1913Cu001,001100 00000 14286 .36364.0CJ00.06:c6 wie0OULU.1.uUuUL;IUUGUD eV4Ouu 270240270240 45554651 554$6145 40464244 0 10.:00001.000001000uo 00000 001174 *00140.0P511 1.1.000UC.1.0000c,.1.00000 .00000 270190 5146 5146 50 0 1000004.00000 ,..0e+.00 4102000.1463 .1.00000 39322U2112u2 45485146 45485146 55535;bg 0 1UU00019000001.00000 , `WA; o1e1112 '^,:,G0 434 10 .1.0000C.1.u0Onu .00(100 204206 6661 6861 - 5857 0 1,000001.00000 .07018,11241 .1.00000.1.0a0U0 270242190 6j606154 62606154 616059 0 3.000004.000001.00000 CU000 .00475 .01619.Ct0000.03390 1.00000IOCCOu0.3 .U0:J61, +40.0id 270301265202264 747269 697472 6766767479 0 1.01+0001.050001.00000 .00000 .02103 ,0L4100402165404045 1.800..!.0a17,00 .cloacu 230200190 98588356 986356be 77949179 0 1.000001.000001s00000 ....06264"27273.02632 .04255.05063 .1.1.0040u.140Cp50 00(,00 33527U3j4321 101 969185 101 159691 100 969c. 00 1,000001.000001900000 ,15000.04211 ,03061,OGg;0 .1.000Cc,.I.00c00.1.00000 .00L01.7, 204150290190 1417226106 $5 147Itas116 85 115111105 00 1o0000100c001,04.000 -.23423 *00952.25641,Cu670 2,17100013.1.40000 262250190. . 100190 100195100 134124122117 00 1.000001,0000014P000001.00000 -.25373-1.1E033 .52344 .1.30000.1.0000u SURVEYQUICK QUERYORICIOIAL EMPLOYEES6UMBEY1 DAVE PORTCH EPPLOYEESNURISERFORECAST IhrLOIICESROWERACTUALApe.20 ORIGINAL6INV:FORECAST NitCRO 641> TEST OX STATISTICU ONESTATISTIC RACROOK 1.047ATZSTIC APPENDIX H-I1(3) 20819024,241 151Ws160155116 151149160155118 161151152146 0 1.040001.00000000n00 0..19176 .01863.09391.011174 .0.040001.1.00000.1.00000..2.00000 250204190194 $90172186166 190172106166 197189184182165 0o 1.000001.04000 .007911..06405,06945 .G1067 1.1.00%.!,0.0.00304;.0.0000D.$.00000 270261194 1991061t7 199105167 199199198 - 00 1.000001,040001.100000 .00000 .07035+.15657.03553 .00000 .1.000001.1.00000 .00000 230335205199 250220223200 250220223200 250231204203 r00 1.00000 +.01478-.04762 .00040.09311 .1.000001.00000.1.04000 .00000 230313190 41.744J7216t99 4172463v9199 280278263 00 '1,00000 1.000081000003UOr00 -.28417-.06144 .1037 .1.00004-1.00a00.1.00000 2311260190270 313367325 367340325 361367309282 0D 1.000001.003301.04000 .00000 +.05637 .1'01.100.05178.47872 91.0000491.04000.1.00000 .90000 206190250 109104936390 409106936390 415142,420906 0 1.000001.00000300000 .049181.493331",:!192;: .1.0000G*1900000::::= CO.04 220332282190 411959373483 95441112373 476965964 00 100c001'04.40030on00 -.05021'09765 .01095 ..1.D0003.1.00000.1.00000 77:1 240230190 1017 582372492 1017 492495372 525520494480 0 1.000001.003001,000301.00000 ...28462-.0040807292 .93111 .01.00003.3.00000.1.0000J.1.00000 r- 33124.03335 652543764 6525s3b02744 6255835617,7 0 1.0001801.04L001000000 go.:413627.06061 001.472 -11,u00fo0"1.6""«I.1.00001; 41000 rn 230230190201 1005 693994787 1045 693998707 109910721013 820 00 0001000000001.000001.0J000 -1,00790-,01:J24 .36993.06903 .10003.1.00609.1.00000 c, 282270/190262 32931941613141119 3293144613141119 329313591207115.9 09 101=0;0018.000001,010CG .:w000.00000 -.03451 .00000.06795.00665 .1.00',U0.1.000.0.00CcJG.1,40030 .00r.00 254290190 150 97 5 139165 6 20 2215 1 1.0954514194701.09656 10.000E0;0.000002.00001 .1.33333...1.10:+00i,22680 240264307 43 SO i5 I93 z 2I 81,000001400Un00 .00000 6.50000 600000.66667 1G.000001004g100 .00309 c4:0( ;Jt.s1 Orom poterch AD-30 MACRO 61Op TEST 010 10180.0.0K APPENDIX 0) SuRvE/ ORIGir.xL EMPLOYEEStOJP18Ed EmPLUYEESKUNdLRFORCAST EMPLOYEES140R8L4ACTUAL ORIGINALMINUSFORECAST STATISTIC ST.IORE IST1C , TOOSTATISTIC 28525U232' 277 8338 7 -3 4.2241744.2247h1.33333 .,57143 04166 .1.33133.4150CD:i 307270190 14 78 19 9 87 S2I3 1.449141.000001.41421 1,375E80 .28571I4286 ftjo63333.2.0GwCJ10.1JUGUJ 324311190 1410 a9 15 a9 133010 9 ( 60 1.552991.06000IIPOUn00.70711 ..2:4uc0-,1CCCO1,0CUO0 5C;;,10 .1.0;,0c0.1s8ULLU20,w,Jou4c,,Goauc 324230190 1216IQ 171116 123211 3 31 1 1.00000.000011.98n56 ,33333.41667 .1.25couristAgAlu tuuc,a5 2JC204190 1212 Is2018t$ 13 36a 2.1300002.144763.31662 .91267 ,1538S90734308462.53646 5.6ti000.7sULJUOti2vOLGG3 230204190 151417 192%Is IS 10 21 1.414216.330121.7371351.00n00 .26667.6(000901,U00 10,0000Z.70U0;w0(.19.:JCLOZ, 240270307190 IS171216 25191615 161716 83 3.672981,004721,172601,0000 -.osaa2 9S62%0,C625CP,18/50 10+05000wtar3i,,won500 270240190 192u18 232119 1918 11 3 1.000001.354011.22474 .21053,05556.16667 ICDOGC30*C.V./COOyo5t4G4a 250281190 201618 23262919 42414044 363 3,240371'022%71.24316 .92195 .,0%167,13636 .2361045U00 031,5000i) ..250 30723)82us190 27al2526 292831 26?4 S23 29+I494929179%52964575 115131 .1)538.11538.07592.29167 .300L0k, 2.00003 CZ 324190 282216 332i21 332846 52 1.16316 .47140 2661 ,14286.,19231 ,00;000 -166667 .L0i)uu -.1 2403351/0 S23937 58452943 424037 26 1.253571.31149).034000 9U992 -..30952 .12500062)6.33395 moilltagAN1C00U.,4,..86661betAK04 324332326230307 4227463533 61944348 424543 1310 96 1,sunu011100000.53321.D0GJO '0,02222 4,3239.0E0u0.14206 .7.sccaa1U+EJSOU..1000u .UtICC,C 240270202208 52354413 3711660 S0514746 1437 2 7,681151.00000 .65q65.53182 .,26000- .02328 .33333,084596 30.00000ft,..33233 664567 .12:21 -4, QUICKSVRVET QUERY ORIGINAL ERPLOTLESNVM5EN DAVE PORN EMPLOYEESNUMIARFORECA$T ENPLOVEESN0016EdACTUAL60-30 ORIGINALm1h0SrORECAST MACRO OOP TEST o STATISTICu OhESTATISTIC NAcROOK THOSTATISIC APPENDIX 1-:.-11(5),3 250210190 55505442 63595645 575452 8923 1.000001.22474 7505597118 11462 .35357.07692 .2.V.):(0310 wo700003.00000 2642u1230393 47583959 0!4158 6059 6324 1,670031.000001e01533 "01647.01695 .01667.1026 .20003 203270307190 Su6970 6567715376 67656263 15 31 1.126671.414211.02273 .99396 -.0208514516 001177.12698 ...117652.00000*60000 307201190 63707956 657677 767368 2220 63 1.783771915470 962017.87287 .00974.00000.05479.11765 .46667.0000004:::n7 [ :1 335240190 9400818605 120147114 9290 9190819382 265711 5 5.901351.0627119600161.47196 .290320902401538.02222.11111 .5.33333.2.25000 .22222 285407190 128109IOC 99 III147124110 103101 9896 251119 1 11.797341.004569.327381.02353 .35300 05595.26531.14583.07767 .1.0769215.00000 f 208210332190 482108104 93 163149109106 119108103 4113 S1 1.000331.11403 .69c85 -.0C9093.97222 02910 30000 270311190 11510810310' 129127119109 116115111110112 1419 76 20412281.362771'557911$14012 .23905 .00540 .11207.10435.33036900000 .9900001.71429 .0000L 270203190 118154115113 168118132122 122119118116 14 93 1.055291.195230,670031.02921 .11869.37705.03390 1.4375J1030=tt GDt03 208IVO190 128159165 129195176 128125121 11 11 1 7664995618 .0078118000.91935 264208190 141169 4 221151173 140138 217 ID 11/ 2.236071.00000 .82631.77182 .07857.57857.75362 10.00000 .59559 209730270190 160152122 159137155 162167195 15 3 1.095'4511.135208 .99960 .01,452031274.05517 *i.50000 -.40008*47E3 202IVO 202127174 219192177 194169178 , 13517 0317781,01960 .72237 .28804.00562 .15873 241190194. 216807169185 220261192 202201195 237612 6 5e1025525 3165 .47207331605 31,gs4.0015311 1343303845 amise:12:: -.625006.60000 SURVEYlit.iCK OVERT hUhEariORIG1h64.EMPLOYEES DAVE FORTCh EMPLOYEt$NUM,FORLCAST E% huMBERACTUALA0.30GMPLO3ECS ORIGINALMINUSr0R1CAST PA010 Opp TEST 0 STATISTICV STATISTICORE sacROsok IITATISTIC180 APP} DIX 1-1-11(6) 282230250190 206249193230 228202212251 236209256243 22332119 2.080331,10772 .94691 .20096 .6.50000200000-.56000-.55814 264230190 2Su243228255 277242233 266261258256 27.1.5 7 :78:93.95429 ' 6.71g74f::..,,,,,i9: -.833336,0000u 230282331190 221314233182 328254372190 268267276274 151 2131$ 8 11:841:74:.1.01)::; .99134 ...2910411,100094160695909 .35273.19708 .aii.1s4spil .9069a1.7963U 2U3190 .407267354 461248436 300293292 54482 1 1.13611111:1; +.08532 .49315 - 2.32258.6154 19030719U281190 324304246270 300353318272 329325313302 295414 92 1.000001,07340 10.0357/.4,021S4....iiliii .07215 -il.r:;4i7(33 4440000..33333 307190 45936479235431$ 459403313370324 42937336536)336 213916 0 1.0011019.:::::ligi ..14247 .069V300b919.02493 1.005Cu SIP5W)OW-E7143 1.i8571'71233 208331220244190 527419430483 413526535455418 530481442433430 9114 s'12107 5225 !!!!!!!1.09294 -.02791 .09356=1111 2294:72'::::,10'0000010.00000 .72581 230248190 755643575741 7777015203'73 714679535763 ..58 582034. 1:393V, .95)29 ...23:4(ji0+2a75 .01573.08024 .2.33333 1.5000034.884.61111 230220190 847635771 848699802 826811780 6431 4 ::= .98019 -.Delo .01777 -.63636 .82353 190 849 850 1101 1 3.00023 .99487 ...22797 .82663 lu476d 210331190190 455231341296 997 4542327413011069 4469343913091144 Om 6710 5 1.00,591.00123 .4672+50533 ":::7:::=9111 0 1'12048 :::"1151::::::::i3s APPENDIX I

OTIS SUPPLY ADJUSTMENT PROCEDURES APPENDIX I(1)

OTIS SUPPLY ADJUSTMENTPROCEDURES

There are five primary sourcesof manpower supply data used inthis report, i.e., theOklahoma State Department of Vocationr,7and Technical Edu- cation, the Governor's ComprehensiveManpower Planning Staff, the State Accrediting Agency, the OklahomaEmployment Security Commission,and the Oklahoma Board of Nurse Registrationand Nurse Education. The Oklahoma State Departmentof Vocational and TechnicalEducation provides information on fourcategories of manpower supply, i.e.,high school vocational program graduates, posthigh school vocational programgraduates, adult vocational programgraduates, and Manpower DevelopmentTraining Act (MDTA) graduates. The figures entered into thetables in Chapter II for secondaryand post- secondary vocational and technicalstudents are calculated in thefollowing manner. The firstphase of the problem was tofind the percentage of the 1971- 1972 students who have dropped out orcompleted each of the vocational programs 14,and to rind out how many areavailable for job placement. This wasdone by searching the student accountingsystem for: Those who dropped out of schooland are employed full time in the field for which they weretrained (VE-6000-R-1, Code 6) The number of students who areunknown (VE-6000-R-2, Code9) Those who are not availablefor placement (VE-6000-R-2, Code 0, 1, 2, and 3) Those who are available forplacement (VE-6000-R-2, Code4, 5, 6, 7, and 8) The total number with completion response(VE-6000-R-1, Code 1 through 8) The total number enrolled inthe program (VE-6000-R) for each occupational objective APPENDIX I(2)

The percentage of students who dropped out of school and are employed full time in the field for which they are trained or completed the program and were available for the labor market were calculated by:

a d PC = x a+d+b+c e If there was insufficient information to calculate the percentage, the percentage was based upon OTIS Cycle Three Report, Supplement Five, "A Follow- Up Study of Oklahoma Vocational and Technical Education Graduates and Drop- outs; 1968-1969 and 1969-1970," August 31, 1972. The second phase was to calculate the number of students available for the labor market by accumulating the number of students who completed the program (VE-6000-R-1, Code 1) from the 1971-1972student accounting system and multiplying it by the percentage worked out above for each occupational objective. Secondary programs, other than reimbursed vocational-technical programs, provide input intc the labor market.Information on. what happens to these graduates is not available.Research that is in progress should provide information so that supply data from this can be included in future OTIS reports. The figures entered into the tables in Chapter II for adults are calculated by searching the student accounting system and ether available records for: The number of adult preparatory students The number of adult supplementary students in adult classes for each occupatir gal objective The formula used to calculate percentage available for the LA )or market was:

PC - 86 (VE-6000-R-2, Code 4,5, 6, 7, 8) g+ h (VE-6000-R-1. Code 1) The factor .86 was used to eliminate duplicated enrollment. The adult enrollment for 1971-1972 was used to calculate the figures in the table in Chapter II. One hundred percent of the graduates in the MDTA programs are listed as available to the labor market. APPENDIX I(3)

The Governor's ComprehensiveManpower Planning Staff providesdata on the numberof graduates to be producedby the federal training programs not associated with the StateDepartment of Vocational andTechnical Education. Data on private school graduates wereobtained ft.om the license application required annually by HouseBill 1403. This licensingprocedure is super- vised by the State Accrediting Agency. The Oklahoma Employment SecurityCommission provides the numberof qualified registrants who areavailable to fill jobs listed in theOTIS Report. The Oklahoma Board of NurseRegistration and Nurse Education andthe Oklahoma State Department ofVocational and Technical Educationprovide data on the numberof nurses that are available forthe labor market. APPENDIX J BLS EMPLOYMENT DATA COLLECTION AND FORECASTING PROGRAMS .APPENDIX J(1)

I.INTRODUCTION

This appendix presents a summary of three employment data collection and forecasting programs conducted by the Bureau of Labor Statistics (BLS) of the Department of Labor. These programs are the: National/State Industry-Occupation Matrix (matrix) Program Tomorrow's Manpower Needs (TMN) Program Occupational Employment Statistics (OES) Program Essentially, these programs are federal/state data collection and analysis efforts.Collectively, their purpose is to gather information to meet federal, state, and local manpower information needs. Several BLS staff members assisted in the preparation of this appendix. Messrs. Richard Dempsey, Russel Flanders, and David Evans, and Ms. Nancy Kent were particularly helpful in reviewing our test and explaining new programs being implemented by BLS. APPENDIX J(2)

II.NATIONAL/STATE INDUSTRY-OCCUPATIONMAI RIX PROGRAM

The National /State Industry-Occupation Matrix Programprovides the occupational composition of the labor force in thestate by industry for 1970. In addition, a multipurpose computer system is providedfor utilization of the matrix and other information to update and project stateand regional matrices. Currently, the base year of the matrix is thecensus year so that detailed information gathered in the decennial censuscan be utilized in developing the matrix. For example, the occupational composition of the laborforce in particular industries is reported in the U. S. Census Bureau publicationOccupation By, Industry.

The industry-occupation matrices present occupational profilescovering approximately 425 occupations in approximately 200industries which represent all types of economic activity.Also, the matrix provides data thatcan be used as a basis for projecting future manpower requirements. Specifically, the tables display changes in the composition of the labor forceover time.In turn, employment levels for a particular year may be predicted by industry andoccu- pational category. Subsequently, the occupational totals ineach industry can be summed to project the manpower requirements byoccupation for the entire economy. This section details the procedures employed by BLS to developa base period matrix.Additionally, the methods utilized to predict occupational patterns and project industry employment are explained in detail.

1. THE DEVELOPMENT OF THE BASE PERIOD MATRIX IS BASEDON INDUSTRY AND OCCUPATIONAL EMPLOYMENT DATA Development of the base period matrix requires:

Data depicting total employment by industry and theoccupational composition of industries Annual employment data.This information facilitates the examination of changes in occupational patterns by industry and an analysis of these changes in relation to external factors such as changes in technology. APPENDIX J(3)

A variety of sources providethese requisite data.Principal sources are described below.

(1) Two Data Collection ProgramsProvide Information On Industry Employment Two ongoing data collection programsprovide information which is used to develop estimates of thetotal employment in the U. S. and in each matrix industry.Specifically, these programs are: Current Population Survey (CPS)Program--Data collected each month through a nationwide sampleof 52, 500 households are recognized as theauthoritative basis for estimating total employment in the U. S.All classes of workers are sampled, including self-employed and unpaid familyworkers as well as wage and salaryworkers. The monthly estimates for a year are averaged to providethe estimate of total employment pre- serted in the Industry-OccupationMatrix. Current Employment Statistics(CES) Program--Data collected monthly and annually from a sampleof approximately 160,000 employees provide detailedindustry estimates for private wage and salary workers whocomprise the majority of the labor force. The data collected through the CES areaugmented by information from the following sources. Survey Of Governments Unemployment Insurance Statistics U.S. Civil Service Commission Population Census Data collected through theCES and other sources facilitateestimating total employment by industry.The estimates for the industries aremanip-' ulated to agree with the estimateof total employment derived from the CPS. Hence, a link is establishedbetween the BLS wage and salary worker statistics and the total employmentstatistics derived from the population survey.

30,c;;-), APPENDIX J(4)

(2) Two Methods Are Used To Estimate Occuyational Employment ELIndtistsz The decennial census provides the information used to distribute total U. S. employment among occupations and industries.Nonetheless, several factors limit the use of census data in compiling occupational employment information for the matrix. Census data are compiled every 10 years; thus, the use of these data for trend analyses is limited. Census data differ from the CPS data used to project total U.S. employment and from CES data used to estimate detailed industry employment. Occupational information for the entire 1Poor force may not be available from the census due to non-responses and undercounting. Therefore, other reliable sources which provide data more frequently are used to estimate occupationalemployment. These sources include: BLS employer surveys for scientists and engineers Office of Education data on the employment of teachers and s librarians Regulatory agency data on the occupational composition of interstate industries, e.g., railroads, airlines, telephone and telegraph companies, and pipeline companies Professional society data for medical and health occupations U.S. Postal Service occupational employment information Federal Civil Service Commission statistics on employment by occupation in Federal government agencies A second method of collecting occupational data is used for occupations for which employment data are not available annually orperiodically. The population census Industry By Occupation tables are used for estimating employment in these occupations.However, the tables must be adjusted to allJw for undercounting and other differences withdata from preferred APPENDIX J(5)

sources.In particular, adju tmentakinvolve forcing census estimates alternately to industry emplolent ,totals and occupational group totals. This iterative manipulation isrepeated until each matrix cell is consistent with both sets of marginal controls.

(3) The Industry And Occupational Employment Data Are Combined To Produce The Base Period Matrix The industry and occupational data gathered through the processes described above are used to produce a matrix. This matrix displays the ratio of the employment for each occupation in the industry to thetotal employment in the industry. These ratios are used in conjunction with detailed industry employment information to provide meaningfuldata for estimating future manpower requirements.

2. OCCUPATIONAL EMPLOYMENT PROJECTIONS ARE DEVELOPEDUSING ESTIMATES OF INDUSTRY EMPLOYMENT AND OCCUPATIONALSTAFFING PATTERNS BLS utilizes the Industry-Occupational Matrix to project future manpower requirements. These projections are based on a seriesof predictions about the economy in the projection year, e.g., theestimated real Gross National Product (GNP) and the unemployment rate for the target year. Giventhese predictions, two steps are followed to project future labor demandby occupation. Total manpower requirements in each detailed industry a re projected. Trends in the proportion of each occupation in each industry are analyzed and staffing patterns(ratios) for the target year are developed.

(1) Three Methods Of Analysis Are Used To Project IndustryEmployment The first step in the development of projections for industryemployment is to estimate the level of real GNP.Once real GNP has been estimated, total employment is distributed by industry in several ways.The actual distribution depends on the data available, thelevel of industry detail required, and the characteristics of the industry. Three different methods cf analysis are used to projectindustry requirements. APPENDIX J(6)

Input/Output Analysis is used to translate GNP into industry employment requirements.In this method, GNP is distributed to the final demand sectors. The output of each industry is determined through input/output relations which show the value addea to output in the industry of final demand. Pro- ductivity in each industry is projected to derive the labor required per unit of output.Together, total output and labor required per unit determine labor requirements by industry. Regression Analysis also is used to'estimate employment in each industry. Employment is projected through a system of equations which relate total employment in each industry to significant variables or model assumptions, e.g.GNP, population, and unemployment rate. Analysis Of Ly

(2) Data From Many Sources Are Analyzed To Project Industry Occupational Structure The occupational composition of the work forces in particular industries are projected after careful analysis of historical occupational composition data.Data on occupational employment in each industry are obtained from the population census.!Is noted earlier, these data are supplemented with data from the following sources: U. S. Postal Service data on employment by occupation Regulatory agency data on the occupational composition of certain interstate industries APPENDIX J(7)

Federal Civil Service Commission statistics on employment by occupation in Federal government agencies Office of Education data on the employment of teachers and librarians Public Health Service data on the medical &nd health occupations BLS employer surveys for scientists and engineers Initially, historical patterns of changes in an industry's occupational composition are projected to the target year by the use of trend lines. The trend for each occupational ratio, whether projected from census data or other sources, is extended to the target year. The difference of the ratio in the base year of the matrix and the ratio in the target year isadded to the ratio in the base period Industry-Occupational Matrix.In result, a first approximation of industry occupational structure in the target year is produced. Occupational trend data are analyzed further to identify factors which caused past changes in the occupational structure.Subsequently, the probable impacts of these factors on occupational employment are estimated for the target year.Occupational ratios for the target year are adjusted, based on the impact of these factors and of emerging factors notreflected in the historical statistics.The resulting ratios depict the projected occupational composition of particular industries in the target year.

(3) Estimated Occupational Ratios Are Used In Conjunction With Projected Industry Employment Future Man Requirements The final step in the projection process involves applying the occupational ratios to the detailed industry employment projections. The figures produced by this comparison represent an estimate ofthe occupational requirements by industry for the target year. Data for theindividual occupations may be summed across industries to provide estimatesof total occupational demand. Further, other industry employment projections based on different assumptions may be developed because the occupational ratios are developed independently of industry employmentdata. When the occupational ratios are applied to these industryemployment projections, occupational requirements by industry maybe identified for varying economic conditions. APPENDIX J(8)

HI. TMN PROGRAM

During the past several years, BLS has developed information on current and projected state and local occupational manpower requirements while continu- ing to compile and analyze national manpower information. BLS published a four-volume bulletin called "Tomorrow's Manpower Needs." in 1969. This bulletin contained national occupational and industrial employment data and provided techniques for projecting occupational manpower requirements at the state and local level. More importantly, the projection techniques formed the core of the TMN Program. Program pkinciples include: Local projections should be developed within the context of national economic and industrial trends. Mechanically produced projections should be reviewed by knowledgeable local analysts. Projections which are consistent with other factors Affecting the future of the area should be developed systemaacally. k.. The projection techniques utilized must ensure flexibility and program continuity. The techniques adopted for the program involve application of National/State Industry-Occupation Matrix patterns to estimate the occupational composition of industrial employment in the state for base and target years. Results are summed to occupational totals and a change factor for each occupation is calculated. These change factors are then applied to estimates of base period occupational employment to yield projections of occupational requirements.

r APPENDIX .1(9)

IV. OES PROGRAM

Three BLS programs weremerged under the heading of theOES program in May, 1973. Thethree component programs were .The OES Survey The National /StateIndustry-Occupation Matrix System The State And AreaOccupational Manpower ProjectionsProgram The resulting OES programis a national /state cooperativeventure involving BLS, the ManpowerAdministration, and stateemployment security agencies. The program is designedto respond to theincreasing number of requestsfor reliable information on currentand projected occupationalemployment at the state and local levels.

1. THE OES SURVEY In 1970, BLS, theManpower Administration,and a number of stateemployment security agenciesjoined to initiate theOES survey. This mail survey program is designedto produce estimatesof occupational employmentby industry for the entire nonagricultural wageand salary labor force.While the immediate goal of the programfocuses on producingoccupational employment estimates for states and subdivisionsof states, ultimately,the program alsowill produce national level estimates. Currently, 26 states areparticipating in this pilotproject. The survey provides for the collectionof sample data onoccupational employment patterns These data, after expansionto represent thepopulation of indus- of industries. of tries, provide meanitIgfuland useful estimates onthe occupational composition of the labor force.

(1) The OES Survey SatisfiesFive Information Needs Prior to developmentof the ,OES survey, manyother programs collected data on the currentoccupational composition ofindustries. These programs includedthe Census of the Population,Current Population Sur- vey, IndustryWage Survey, and AreaSkill Survey.Information compiled .4k

APPENDIX 3(10)

by these programs was fragmented and inadequate to meet the needs of manpower planners and other data users. Therefore, the OES survey was designed to achieve the following objectives: Develop occupational employment estimates by industry. These estimates will provide the basis for producinga time series of estimates of employment by occupation. A Provide data about the occupational composition of industries, including: Differences in occupational composition withinan industry by size of establishment, process, etc. Extent of inter-establishment variability Changes in occupational composition in response to external factors such as changing technology Provide occupational information in a form that permits development and improvement of industry-occupational matrices at the state and area levels and, ultimately, at the national level

(2) The OES Survey Collects Occupational Data The OES survey collects data from a sample of employers on the occupationsi composition of industries within each of the cooperating states. state employment security agencies analyze and edit the data and produce occupational employment estimates by industry. Specifically, occupational employment data are collected far ,. industries through mail questionnaires. The OES survey is designed to cover all nonagricultural wage and salary employment. Selected establishments receive questionnaires through the mail and are asked to provide information on the occupational composition of the labor force within their establishments as of a specified date. The date is chosen to provide data from a typical month ip terms of employment reasonability.- Data are collected for pre-listed and employer listed occupations. Only pre-listed occupations are used from the first round of sampling. APPENDIX 3(11)

The survey of manufacturingindustries conducted during1972 utilized 33 separatequestionnaires to collectoccupational employment data at the three-digitStandard IndustrialClassification (SIC) level. On list of approximately80 occupations that are com- each questionnaire, a In addition mon to manydifferent manufacturingindustries was provided. to the standardlist, each questionnaireincluded a second list ofother occupations that are importantto the particularindustry. Importance by the number ofemployees in these occupations.Addi- was determined period of education or tionally, other occupationsrequiring a substantial training were listed forselected industries. Employers receivingquestionnaires are requestedto provide the following informationfor the designated referencedate: Total number of workersemployed in the listedoccupations Total number of apprenticesin specifically designated occupations

Total number of workersengaged primarily in researchand development activities forspecific scientific andtechnical occupations the data supplied byemployers for completeness BLS personnel validate generated. and reasonablenessafter occupationalemployment estimates are The Em lo er Sam le IsUsed To Produce (3) Data Collected From Occupational Em lo mentEstimates B Industr

Occupational employmentestimates are prepared foreachindustry size class, e.g.,500-999 employees. on anindustry-by-industry basis. The occupationalestimates for each sizeclass are developed at the most detailed SIC level permittedby the sample. Theseestimates are relatively occupational employmentpatterns are generally most detailed because in similar amongestablishments which are ofthe same size and engaged manufacturing similarproducts or performingsimilar services. The process used toderive occupationalemployment estimates from the sample data involvesfour basic steps: APPENDIX J(12)

The reported occupational employment figure for an establishment is multiplied by the establishment's sample weight. The weighted occupational employment figure for the establishment is summed with similar data for all other sample establishments in the same industry size class (cell). The sum of the weighted occupational employment data is divided by the sum of the weighted total employment for all reporting establishments in the cell.This produces the occupational ratio for the cell. The ratio is multiplied by the total industry employment figure for the cell, as determined from ES 202 data, to yield the estimated number of workers in each nccupn tion in the cell. This process can be expressed by the formula below.

Pt:,:eP i wi)M Eel W1 where:

Pc Estimate for occupation P in an industry size cell

pi Reported employment for occupation P by number of establishments in an industry size cell

M The ES 202 benchmark employment for the industry size cell

ei Reported total employment in the ith establishment in an industry size cell

wi The reciprocal of the sample weight or the sampling ratio of the ith establishment in an industry size cell The occupational estimates produced for each cell can be aggregated at various levels to produce data on the occupational composition of different levels of industry detail, e.g., three-digit or two-digit Sic code industries. These estimates provide valuable state and local level information hereto- fore unavailable from a single data source.Specifically, occupational APPENDIX J(13)

employment estimates for states and state subdivisions can be used to construct/ state and local level industry-occupational matrices for all wage and salary employment classifications. These matrices, in turn, may provide the basis for projecting occupational employment requirements.

2. THE NATIONAL/STATE INDUSTRY-OCCUPATION MATRIX SYSTEM Early in 1972, BLS inaugurated a new program called the National/State Industry-Occupation Matrix System. This system was developed with the cooperation of the Manpower Administration and state employment security agencies. The purpose of the system is to develop occupational information at the state and local le;1s. The system is designed to provide a set of 51 (all states and the District of Columbia) individual matrices consistent in format, concept, and data base with the BLS national matrix. A matrix will be developed for each state and the District of Columbia. The data base for the program is a special tabulation of industry by occupation by class of worker from the 1970 census for each state. The system also includes preparation of occupation-specific death and retirement rates for each state that can be used to help estimate total occupational demand. These rates were developed using special census tabulations of occupational employment distributed by age and the BLS standard working life tables. The first goal of the system is to provide an employment matrix for each state which covers over 400 occupations and 200 industries and can be used to generate projections of occupational requirements with greater accuracy than projections based on national patterns alone. Another goal is to provide a flexible, multipurpose computer system that will allow a state to update its matrix, e.g., supplement the census-based matrix with data from the OES survey and prepare ' sub-state estimates. Currently, a number of states are nearing completion of the design phase, including preparation of the new 1970 national matrix and development and documentation of the computer software package. Most state agencies have reviewed the basic census input data and prepared special industrial employment controls covering 200 industrial sectors and six classes of worker categories.Plans call for implementation of the system to begin in early 1975. APPENDIX J(14)

3. THE STATE AND AREA OCCUPATIONAL MANPOWER PROJECTIONS PROGRAM Effective November, 1972, BLS was charged with the responsibilities for providing consultation and technical assistance to state employment security agencies concerning state and area occupational manpower projections. This assistance program requires: Continuous research efforts to develop and improve methods of preparing state and local manpower projections Development and maintenance of technical manuals, guidelines, and other appropriate materials presenting projection procedures to the stateagencies Centralized processing by state agencies of state data related to the development of occupational demand projections

4. INTERIM MANPOWER PROJECTION PROGRAM

Projections basedon the state matrix program cannot be available in time for Fiscal Year 1975 planning.Therefore, the Bureau has recently undertaken an interim project to develop stateand sub-stateprojections for the FY 1975 manpower planning cycle: This project is being conducted in conjunction with the Manpower Adminis- tration.Specifically, BLS is responsible for developing and coordinating technical aspects of the program, while state employment agencies are responsible for organizing data files and determining and disseminating the occupational projections. A centralized data processing capability is being funded by the Manpower Administration. During the first phase of the project, BLS will research and test various procedures for developing sub-state area projections and pilot test the project in one or more local labor market areas.State projections for some 400 occupations will be developed using a modified TMN methodology. These projections are scheduled for completion in mid-October, while projections for smaller geographic areas were completed in December,- 1973.Efforts were made to maximize the level of detail of both occupational and geographic data, according to the constraints of available data and projection reliability.