INFORMATION TO USERS
This manuscript has been reproduced from the microfilm master. UMI films the text directly from the original or copy submitted. Thus, some thesis and dissertation copies are in typewriter face, while others may be from any type of computer printer.
The quality of this reproduction is dependent upon the quality of the copy submitted. Broken or indistinct print, colored or poor quality illustrations and photographs, print bleedthrough, substandard margins, and improper alignment can adversely affect reproduction.
In the unlikely event that the author did not send UMI a complete manuscript and there are missing pages, these will be noted. Also, if unauthorized copyright material had to be removed, a note will indicate the deletion.
Oversize materials (e.g., maps, drawings, charts) are reproduced by sectioning the original, beginning at the upper left-hand corner and continuing from left to right in equal sections with small overlaps. Each original is also photographed in one exposure and is included in reduced form at the back of the book.
Photographs included in the original manuscript have been reproduced xerographically in this copy. Higher quality 6" x 9" black and white photographic prints are available for any photographs or illustrations appearing in this copy for an additional charge. Contact UMI directly to order.
University Microfilms International A Bell & Howell Information Company 300 North Zeeb Road. Ann Arbor. Ml 48106-1346 USA 313/761-4700 800/521-0600 Order Number 9401385
The relationship between Automotive Service Excellence certification and Ohio Vocational Education Achievement Test scores
Wiblin, James Howard, Ph.D.
The Ohio State University, 1993
Copyright ©1993 by Wiblin, James Howard. All rights reserved.
UMI 300 N. ZeebRd. Ann Arbor, MI 48106 THE RELATIONSHIP BETWEEN
AUTOMOTIVE SERVICE EXCELLENCE
CERTIFICATION AND OHIO VOCATIONAL
EDUCATION ACHIEVEMENT TEST SCORES
DISSERTATION
Presented in Partial Fulfillment of the Requirements for the Degree Doctor of Philosophy in the Graduate School of The Ohio State University
By
James Howard Wiblin, B.S., M.A.
* * * * *
The Ohio State University 1993
Dissertation Committee: Approved:
Dr. Dewey A. Adams Dr. Joseph L. Davis Adviser 0 Dr. Aaron J. Miller Comprehensive Vocational Education Graduate Program "The society which scorns excellence in plumbing because plumbing is a humble activity and tolerates shoddiness in philosophy because it is an exalted activity will neither have good plumbing or good philosophy. Neither its pipes or its theories will hold M ater."
John W. Gardner Copyright 1993
James H. Wiblin
All rights reserved Dedicated to my wife, Jo Ann,
my children, Dan and Mandi,
and my mother, Roxie. ACKNOWLEDGEMENTS
The following study is a creation of many - my advisors, instructors, co workers, mentors, and friends. I am a conduit for their knowledge and training. I wish to acknowledge their guidance, leadership, and example in my life and work. I especially want to thank my doctoral committee, which included Dr. Dewey Adams,
Dr. Aaron J. Miller, and Dr. Joseph Davis. My advisor, Dr. Adams, was especially supportive, and without him I would not have finished this task. Dr. Byrl Shoemaker has been a role model for me throughout most of my career, and continues to be today.
He is a leader in the nation for vocational education, and I thank him for his friendship.
Dr. Darrell Parks encouraged me to pursue this degree, and supported me during the process. My first supervisor in vocational education was Sig Guckenheimer, who gave me a sound vocational foundation which has served me well for many years. My first teacher educators, James Provost, Russell Riley, and Bernard Densmore, taught me the tools of the trade, including how to enjoy it.
I have great respect for the teachers in Trade and Industrial programs in Ohio.
They have hard-won skills which they are eager to share with tomorrow's tradesmen and women. Their skills are shown in the results of this study. Also, the help which
Ohio's businesses and industries are giving to schools is right now improving our programs, our graduates, and Ohio's future. I have followed Ohio's vocational students around the state, to national competitions many years, and to the Netherlands, to see them compete against other nations. They are a wonderful example of what our youth can and should be. I salute them. They deserve the best education we can provide.
To my wife, who has always supported me, and first encouraged me to pursue this goal, I am eternally grateful. VITA
May 24, 1941 ...... Bom, Richwood, West Virginia
1958 ...... High School Diploma Richwood High School Richwood, West Virginia
1959-1964 ...... North American Aviation Patuxent River, Maryland Palmdale, California Columbus, Ohio
1961...... Western School of Heavy Equipment Operation Weiser, Idaho
1965 ...... Bachelor of Science in Education The Ohio State University Columbus, Ohio
1965-1968 ...... North American Aviation Training Equipment Design Specialist Columbus, Ohio
1967 ...... Master of Arts in Education The Ohio State University Columbus, Ohio
1968-1970 ...... Industrial Arts Teacher Licking Valley School District Newark, Ohio
1970-1973 ...... Diversified Cooperative Training Coordinator Newark City School District Newark, Ohio
v 1973-1979 ...... Trade and Industrial Supervisor Licking County Joint Vocational School Newark, Ohio
1979-1982 ...... Supervisor, Trade and Industrial Education Northeast Region Ohio Department of Education Columbus, Ohio
1982-present ...... Assistant Director of Vocational Education for Trade and Industrial Education Columbus, Ohio
1989-present ...... Doctoral studies The Ohio State University Columbus, Ohio
Publications
1992 ...... "Ohio's World Class Partnerships," Journal o f American Technical Education Association
Field of Study
Major field: Comprehensive Vocational Education Dr. Dewey A. Adams Table of Contents
ACKNOWLEDGEMENTS...... iii
VITA ...... v
LIST OF TABLES...... ix
LIST OF FIGURES ...... xi
CHAPTER PAGE
I. INTRODUCTION ...... :...... 1
Background and Need ...... 1 Statement of the Problem ...... 8 Objectives of the Study ...... 9 Statement of the Hypotheses ...... 11 Significance of the Study ...... 12 Definition of Term s ...... 15 Limitations ...... 17 Summary and Outline o f Dissertation by C hapter...... 18
II. REVIEW OF LITERATURE AND THEORETICAL FRAMEWORK ...... 20
The Educational Reform Movement of the 1980s ...... 21 Cultural Factors Affecting Vocational-Technical Training ...... 30 National Education Goals ...... 33 Current Efforts to Implement Industry-Based Standards ...... 38 Summary and Theoretical Framework ...... 45 III. METHODOLOGY 46
Description of Sample ...... 48 Research Design ...... 52 Data Collection ...... 53 Data Analysis ...... 55 Summary ...... 57
IV. RESULTS OF THE STUDY 59
Introduction ...... 59 Schools and Individuals Selected for Study ...... 59 Methods of Reporting Findings ...... 61 Hypotheses Testing ...... 80 Results of Hypothesis 1 ...... 81 Results of Hypothesis 2 ...... 83 Results of Hypothesis 3 ...... 85 Results of Hypothesis 4 ...... 87 Results of Hypothesis 5 ...... 89 Results of Hypothesis 6 ...... 91 Summary ...... 94
V. SUMMARY, CONCLUSIONS, AND RECOMMENDATIONS 95
Introduction ...... 95 Summary ...... 96 Background ...... 96 Objectives...... 96 Research Questions ...... 97 Procedures ...... 99 Findings ...... 99 Conclusions ...... 100 Recommendations for Practice and Further Research ...... 102 Practice...... 102 Research...... 104
BIBLIOGRAPHY 105 List of Tables
TABLE PAGE
1. Schools Included in Study ...... 51
2. Junior Language Class Means ...... 63
3. Senior Language Class Means ...... 64
4. Junior Nonlanguage Class Means ...... 65
5. Senior Nonlanguage Class Means ...... 66
6. Junior Automotive Class Means ...... 67
7. Senior Automotive Class Means ...... 68
8. Descriptive Statistics - Junior Language ...... 70
9. Descriptive Statistics - Senior Language 71
10. Descriptive Statistics - Junior Nonlanguage 72
11. Descriptive Statistics - Senior Nonlanguage 73
12. Descriptive Statistics - Junior Automotive 74
13. Descriptive Statistics - Senior Automotive 75
14. Language Test Results for Junior Level ...... 82
15. Language Test Results for Senior Level 84
16. Non-Language Test Results for Junior Level 86
17. Non-Language Test Results for Senior Level 88
18. Automotive Test Results for Junior Level 90
19. Automotive Test Results for Senior Level 92 ix Achievement Test Scores of Students Completing
ASE-certified Programs ...... 93 List of Figures
FIGURE PAGE
1. Language Mean Scores ...... 77
2. Nonlanguage Mean Scores ...... 78
3. Automotive Mean Scores ...... 79 CHAPTER I
INTRODUCTION
Background and Need
America's educational system is in trouble. The public, the policy makers, and the educators know it. The issue now is not whether it is in trouble, but what must be done to correct it. For a decade now, some disturbing facts have begun to surface.
The facts are compelling evidence that change and reform of public education are crucial.
American high school graduates are not competing well with students in other industrialized nations. "The ominous news seems relentless, declining test scores, growing numbers of high school students unable to perform simple arithmetic, poor showing of American youth in international comparisons of academic ability, a national dropout rate of roughly 25 percent" (Office of Technology Assessment
Report, 1989, p. 10). A recent comparison study of a selected group of senior level
American college prep students and similar Japanese students on an international math test showed that only 2 to 3 % of the Americans could match the median score for their Japanese counterparts (Marshall & Tucker, 1992). Other studies have shown that Scholastic Aptitude Test (SAT) scores have declined steadily since the mid-
1980s and that U.S. students rank in the bottom half of international tests for math and science (Marshall & Schram, 1993).
1 2
Business and industry leaders complain that they must retrain graduates in
basic academic and technical skills needed to succeed in jobs. They say that their
employees' skills in reading, writing, math, and science are inadequate for the job. A
recent National Alliance of Business survey revealed that more than half of small
business owners have trouble finding entry-level workers with even the basic skills of
reading, writing, and computation, and that the average manufacturer rejects live of
every six job applicants (National Alliance ofBusiness,1992). Many companies, such
as Motorola, have started their own in-house remedial training programs because they
cannot hire workers with the needed skills. Initially, the Motorola Training and
Education Center was established to provide training in basic industrial problem solving, statistical process control, and flexible manufacturing for their own workers.
Motorola managers were surprised when few of their workers signed up for the voluntary training; they later discovered that the reason was that less than half of their work force could pass a test containing such questions as "Ten is what percent of
lOO*7" (Marshall & Tucker, 1992, p. 100). Another surprise came to Motorola when they began to investigate the skills problem at other levels in their company. They found skill deficiencies at all levels throughout Motorola - deficiencies that should have been taken care of by the public educational system through which the employees moved. Marshall and Tucker (1992) stated that after 11 years of concentrating on improving their worker's skills, Motorola is now spending $120 million annually on education and training. From an employer's viewpoint, ihey are paying three times for the failure of America's educational system: first, through local school taxes; second, through lost revenue during the nonproductive time that employees are in training; and third, through direct training costs. Unlike Motorola, many of the smaller employers simply do not have resources to provide training for 3
front-line workers, and they are forced to operate at a disadvantage when competing with foreign suppliers or larger U.S. companies.
From education's perspective, and in a gradual shift in emphasis, schools have been asked to deal with problems of malnutrition, family breakups, drug abuse, violence in the homes and in the schools, handicapped and special needs students, nontraditional occupations for men and women, mental and emotional problems of students and families, crime, and physical assaults on teachers and administrators.
Vocational education has been asked to do all of these things and often to accept students based upon their special needs rather than their potential for succeeding in the trades they have selected. Although a general education must consider all of these things to provide an education for its students, vocational education does so at the risk of an inability to provide skilled workers for America.
Priorities in this country are focused on preparation for college. The consensus of the American public is that one must attend college to be successful.
What was in America's past a respect for skilled craftsmen has declined into silent disregard for the trades. Although this attitude has not prevented students from attending vocational schools for training, it may have deterred the brightest from choosing a career in the trades. Worse, it may have convinced students who select vocational training that they are not as valuable or skilled as those who attend college.
Policy makers are not exempt from this attitude: America currently spends about $55 on college preparatory courses for every dollar it spends on programs for noncollege bound students (Thurow, 1992, p. 275). It is difficult to overcome disparities like this to produce a highly skilled work force that can compete internationally. Garrick Utley, NBC News, stated that if America's educational system is to be competitive with other industrialized countries, then its people must 4
abandon the philosophy that a college degree is required for success and must once
again respect the ability to work with one's hands (Utley, 1993).
Even in the late 1800s, Booker T. Washington knew this well. In his book Up
from Sla\>ery\ Washington tells about requiring his students at Tuskegee to learn the
art of brick making:
The making of these bricks taught me an important lesson in regard to the relations of the two races in the South. Many white people who had no contact with the school, and perhaps no sympathy with it, came to us to buy bricks because they found out that ours were good bricks. They discovered that we were supplying a real want in the community. The making of these bricks caused many of the white residents of the neighbourhood to begin to feel that the education of the Negro was not making him worthless, but that in educating our students we were adding something to the wealth and comfort of the community. As the people of the neighbourhood came to us to buy bricks, we got acquainted with them; they traded with us and we with them. Our business interests became intermingled. We had something which they wanted; they had something which we wanted. This, in a large measure, helped to lay the foundation for the pleasant relations that have continued to exist between us and the white people in that section, and which now extend throughout the South. (Washington, 1901, pp. 110-111)
In international student competitions, the varying attitudes among nations as to the importance of vocational skills are clear. In the 1991 VIC A International
Youth Skill Olympics held in Amsterdam, Netherlands, the United States won only one gold medal. Japan, Germany, Taiwan, and Korea won the bulk of the medals awarded, and their students' desire to win was obvious. Teams from the Asian countries arrived loaded with piles of equipment and with coaches intent on pressuring their students to be the best. These students rarely smiled, reflecting the tension under which they worked. Respect for the trades by these nations was 5
evident even in the contests themselves. In one contest, German, Austrian, and Swiss
teams carefully laid beautifully patterned cobblestone streets by hand in sand.
Americans did not participate in this contest, since cobblestone streets are regularly
being paved over with asphalt in this country.
After the Skill Olympics, an editorial appeared in the Washington Post titled
"Our Unsung Olympic Hero: Why has Robert Pope's Electrifying Victory Gone
Unnoticed?" The authors of the editorial stated,
Robert E. Pope won a gold medal in his last Olympics. But his event wasn't televised, he wasn't offered any endorsement contracts and he hasn't been drafted by a professional team. The only newspaper that noted his victory was in his home town o f St. Petersburg, Fla. As far as the public was concerned, Pope was in the wrong Olympics. But as far as the future of the United States is concerned, he offers far more hope than the athletes who are representing the nation this year in France and Spain. ..In America...we discourage our youth from seeking careers in so-called blue-collar vocations and insist that college educations are essential for happiness and success in life. Those who do not receive college degrees (about 70 % of the population) are cast adrift to make their way—with little if any training in skills or preparation for work. (Kolberg & Smith, 1992)
America must change the myth that in order to be successful, a person must have a college degree. "The skills of the labor force are going to be the key competitive weapon in the 21 st century. Brainpower will create new technologies, but skilled labor will be the arms and legs that allow one to employ the new product and process technologies that are being generated" (Thurow, 1992, p. 51).
"The front lines of the war on crime, drugs, teenage pregnancy, or housing desegregation should be established [outside the classroom]...The school's primary responsibility is to ensure that their students are educated. . . The energy of our school systems should be focused on education - not dissipated on other goals, no matter 6 how laudable" (Ibid. p. 279). Teachers are not prepared for, nor are they capable of, teaching high performance skills and attending to social ills simultaneously.
Finally, vocational education must be able to test students as a basis for admission to vocational programs. Schools in Germany have rigorous entrance examinations for their youth apprenticeship programs to ensure that students who enroll can succeed and profit from the training. America's students must also be evaluated before admission to vocational programs to earn the privilege of learning a skill in school and to conserve our scarce resources. All students should have the opportunity to enter vocational programs, but only those who have the interest, ability, and will to succeed in a trade should have the privilege to remain enrolled.
According to Rita Kramer,
We have set aside equality of opportunity—the idea of opening doors to anyone—and replaced it with equality of results. Everyone has to pass, to be promoted, to enter college, to get a degree. No child should be passed from one grade to another without demonstrating by means of a dependable system of testing that he or she knows what is supposed to be learned in that grade. It is a matter—and nothing less will do it—of raising standards all along the spectrum of schooling from first grade to graduate study. (Kramer, 1990, p.215)
Some would say that preventing students from entering vocational programs smacks of punishing the students for the sins of their schools. They assert that schools are not doing the job of educating students and that standards should not be used for students until schools and teachers' skills can be vastly improved. U.S. News and World Report agrees that many students are handicapped by terrible schools and terrible teachers, but does not agree that standards must be postponed until every substandard school is fixed and every poor teacher is replaced. The magazine quotes
Albert Shanker, president of the American Federation of Teachers. "We don't abolish medical-school exams because not everyone has had the opportunity for top-notch pre-med education. Nor do we say that tests for airline pilots shouldn't count because not everyone has the opportunity to do well on them" (Leo, 1993, p. 19).
Other critics of standards for student entry might say that it is unfair. In
America, land of opportunity, everyone has an equal chance to succeed, to learn, to be exposed to training that can lead them from the quagmire of welfare, poverty, and other social evils that befall some segments of our society. Doesn't education have a responsibility at least to attempt to break this cycle? In The Dream and the
Nightmare, Magnet contends that it is attitudes like this which do an injustice to the people they intend to serve. "The beliefs and values [they transmit] to the underclass are all the wrong ones, retarding rather than promoting self-development, you've been marred by victimization, you can't succeed without special treatment, your success or failure is really not in your own hands, the values that allow us Haves to succeed have no application to you Have-Nots and will only oppress you, your own self-destructive behaviour is a legitimate expression of your history and your oppression" (Magnet, 1993, p. 30).
Philosophies that have made America great have also made her weak. In striving to give everyone the same educational opportunities, we have lowered our sights. We are teaching to the middle, or slightly below, and losing our world economic position in doing so.
This notion of standards must be further explored. National standards must be adopted that will motivate students to achieve at higher levels and that will make
American vocational training programs the best in the world. According to Thurow,
"international bench marking reveals that no one turns out a high-quality product unless someone sets quality standards. The world's best school systems operate under 8 a strong centralized ministry of education that sets tough standards that everyone must meet" (Thurow, 1992, p. 260).
Statement of the Problem
Employers need a method of determining that potential employees who have completed vocational training meet certain standards. One example of such standards is the National Institute for Automotive Service Excellence (ASE) voluntary industry- based standards for automotive programs. ASE standards are industry-developed standards for vocational-technical training programs for automotive repair, auto collision, and truck repair. In Ohio, ASE standards for automotive repair programs were pilot-tested in 1983; on October 10, 1983, Ohio was the first state in the nation to have a program receive ASE certification. Since then, approximately 67 programs in Ohio have received certification. Although ASE is a voluntary program, Ohio and
10 other states have made ASE certification mandatory for programs to receive state and federal funds. The deadline in Ohio for ASE certification is October 1994.
To become certified, schools must purchase a packet of materials from ASE and conduct a program self-review. They then return the materials to ASE for evaluation. On receiving the self-review packet, ASE determines whether a program is ready for certification. In case there are areas of deficiency, schools are so notified, and the process is halted until improvements are made. After all noted deficiencies are corrected, ASE notifies schools that the on-site review can proceed. An independent evaluation team leader and three to four practicing automotive technicians conduct the review, spending approximately 2 days at the facility reviewing equipment, curriculum, students, schedules, faculty, administration, and 9 other facets of the program. Programs can request between three and eight areas in which to be certified. There are eight specialty areas in which a program can be certified: automatic transmission/ transaxle; brakes; electrical systems; engine performance; engine repair; heating and air conditioning; manual drive train and axles; and suspension and steering. The on-site review determines whether a program can be certified in the requested areas. Certification is valid for 5 years.
Industry consistently recognizes the certification process as an indication of quality. However, no one has conducted any research to determine whether ASE program certification makes a difference in the quality of the program. This research is designed to determine the relationship between ASE certification and performance on the Ohio Vocational Education Achievement Test.
Objectives of the Study
Based on a review of research and literature about industry-based standards, six research questions have been formulated to guide this investigation. This study seeks to answer the following questions:
1. Do junior students who complete an ASE-certified program score significantly higher on the language section of the Short Form Test of Academic Achievement (SFTAA) than students who complete a non certified program?
2. Do senior students who complete an ASE-certified program score significantly higher on the language section of the SFTAA than students who complete a noncertified program? 10
3. Do junior students who complete an ASE-certified program score significantly higher on the nonlanguage section of the SFTAA than students who complete a noncertified program9
4. Do senior students who complete an ASE-certified program score significantly higher on the nonlanguage section of the SFTAA than students who complete a noncertified program?
5. Do junior students who complete an ASE-certified program score significantly higher on the automotive section of the Ohio Vocational Education Achievement Test (OVEAT) than students who complete a noncertified program?
6. Do senior students who complete an ASE-certified program score significantly higher on the automotive section of the OVEAT than students who complete a noncertified program?
The nonlanguage section of the SFTAA measures recognition, ability to draw
pictorial analogies (opposites and similarities), and comprehension of numerical and
geometric pattern sequence changes, whereas the language section measures verbal
comprehension, word meanings, and retentive ability. The SFTAA is used to measure
higher-level cognitive skills and the academic aptitude of students enrolled in
vocational education programs (Ohio Vocational Education Achievement Test
Program, 1989, p.3). An increase in mean scores of the automotive test could be a result of improved student academic ability rather than ASE certification. If the scores in nonlanguage or language sections remain constant or decline and the automotive mean scores improve, then there may be a relationship between ASE certification and the automotive scores.
Although the study seeks to determine the answers to the above questions, it is expected that the academic aptitude of the students during this time period remained constant or declined. Increased emphasis and public pressure on academic requirements for graduation during this period caused some of the more capable students who traditionally would have taken vocational education courses to opt for college preparatory courses instead. As a result, vocational programs may have been forced to accept less able students. Obviously, this would have an impact on students' scores on achievement tests, both academic and technical, and therefore must be considered in this study.
Statement of the Hypotheses
Because the ASE certification program emphasizes improvements in equipment, curriculum, and industry linkages, students trained in a program that is
ASE-certified should be better prepared to answer questions on the achievement test in the automotive section. The following directional hypotheses are derived from the research questions presented in the previous section and will be used to examine the trend of mean scores on the language and nonlanguage section of the SFTAA during the period of the study, and to explore the relationship between student mean scores on the automotive section of the OVEAT and ASE certification.
1. The student mean language scores for juniors will not show a significant increase during the period of the study.
2. The student mean language scores for seniors will not show a significant increase during the period of the study.
3. The student mean nonlanguage scores for juniors will not show a significant increase during the period of the study. 12
4. The student mean nonlanguage scores for seniors will not show a significant increase during the period of the study.
5. The student mean automotive scores for juniors will show a significant improvement after programs receive ASE certification.
6. The student mean automotive scores for seniors will show a significant improvement after programs receive ASE certification.
Significance of the Study
There is an emerging consensus in America on the part of vocational
educators, policy makers, and industry and labor representatives that national
standards are needed to bring our vocational-technical training programs up to the
quality of our foreign competitors. President Bush in his education strategy America
2000 (U.S. Department of Education, 1991) asked the Departments of Labor and
Education to spearhead a public-private industry partnership to develop voluntary
industry-based standards for all major industries. In order to maintain a high standard
of living, America must be competitive in the international market. Competitiveness
requires producing goods and services more efficiently and cheaper than others.
Well-trained and motivated workers are critical in defining how efficiently and
effectively industries operate. Industry-based standards will help determine whether
workers are well trained.
Few studies have been conducted to determine the effectiveness of industry- based standards for vocational training programs. Most previous research deals with comparing the American educational system to European or Japanese systems where national standards exist. The author researched the Educational Resources 13
Information Center (ERIC) data base and files on doctoral dissertations completed
since 1985 to conduct a review of literature on business-education standards related to vocational education programs No formal research data were found for this area.
However, some studies have begun to better define industry-based standards and how they should be developed. According to an October 9, 1992, progress report issued by the National Working Group on Skill Standards and Certification, there are currently at least five mapping studies of skill standards systems in the
United States (National Advisory Commission on Work-Based Learning, 1992).
These studies, funded by private foundations and the Departments of Labor and
Education, will not be completed until 1993.
This study will compare student scores on the OVEAT from 1980 to 1990 to determine the relationship between ASE certification and the mean scores on the automotive section of the test. Specifically, the study will use OVEAT data collected from 26 different vocational automotive programs throughout Ohio from 1980 to
1990. The national ASE certification program was initiated in 1985, and achievement test data are available for at most 7 years prior to becoming ASE certified and 5 years after. By analyzing the mean scores on the automotive portion of the OVEAT, the researcher intends to show the relationship between ASE certification and student mean scores on the OVEAT. All schools will be analyzed based on the year of certification, with scores prior to that year compared to scores during and after certification.
Since current efforts to establish national standards promote voluntary compliance, research data will be valuable in showing the advantages of adopting national standards and in convincing doubting instructors to participate. Although local administrators have been largely positive, there are no quantifiable data to 14
support their views, and some instructors do not share the enthusiasm, especially those whose technical skills are outdated. Research results from this study could be helpful in convincing those who question the value of the ASE certification program and could encourage them to consider adoption of standards. Adoption of industry- based standards on a widespread basis would link industry representatives more closely with vocational educators and would, in effect, improve communication between the two groups. This alone could improve the quality of programs. In addition, exposure to current practices in industry would help to update teacher skills and school lab equipment to current technologies.
In addition to technical aptitudes, this study shows the academic aptitude of students in vocational automotive programs during the study period by analyzing the language and nonlanguage scores on the OVEAT. If the quality of students has remained constant or has improved, then vocational education must look to the quality of its programs to explain the lack of appropriate skills in graduates.
As a result of this study, a number of questions will be posed that could lead to further research, such as the structure of the certification program, the manner in which standards are implemented, and the manner in which standards are developed.
Certified programs have to meet certain standards for equipment and instructor skills that other programs do not have to meet. A possible future research study could investigate the effects of program certification through industry-based standards on the two factors of teacher skills and laboratory equipment. 15
Definition of Terms
Accreditation Refers to the process of evaluating and recognizing a program of study as meeting predetermined standards or qualifications.
Achievement Tests Designed to measure accomplished skills.
Aptitude Tests Designed to predict what a person can accomplish with training.
ASE National Institute for Automotive Service Excellence.
Certification (Individual) Refers to the process by which verification of occupational-specific competencies is issued by various institutions, agencies, or associations to individuals who meet predetermined standards.
Certification (Program) Refers to the process by which vocational- technical programs are evaluated using industry- based standards. The process usually includes an on-site evaluation of facilities, equipment, and curriculum by a committee of practitioners to determine if the program meets the standards.
Competency An observable and measurable behavior that has a definite beginning and ending and one that indicates the ability to perform the activities within an occupation to the set standard. 16
Criterion-Referenced Tests Tests that measure whether a person has acquired the skills that are required for the job.
Licensing A process of mandatory certification widely used in health care and public safety occupations which may involve both written and performance testing. Licensing is required to protect public health and safety.
Norm-Referenced Tests Tests that measure how much a person knows and can do in relation to the average person and all other test takers.
OVEAT Ohio Vocational Education Achievement Test.
Performance Standard The level of performance considered acceptable for (1) advancing to the next level of education or training, (2) successfully entering a job related to training, (3) continuing to offer a course or program, or (4) establishing acceptable program performance. Performance standards may be defined for teachers, students, courses, programs, districts, or states.
PrintEd National voluntary certification for vocational printing programs developed by the Printing Industries of America.
Proficiency Indicates the ability to perform the activities within an occupation to a set standard. It may incorporate the ability to apply the relevant skills and knowledge to new situations within the occupational area as well as generic skills. (Skill Standards and Certi fication Issues Paper, 1992, p. 8).
SFTAA Short Form Test of Academic Aptitude. 17
Limitations
The study was limited to a purposive sample of vocational automotive
programs that were involved in both the voluntary OVEAT program and the voluntary National Institute for Automotive Service Excellence (ASE) certification program. A purposive sample is a nonprobability sample and carries with it the following limitations. (1) cannot infer to larger populations, (2) cannot specify probabilities of units being included in the sample, and (3) cannot estimate sampling error (McCracken, 1990).
The first selection criterion was that schools must have administered the Ohio
Vocational Education Achievement Test to junior and/or senior classes between 1980 and 1990. This time period was selected because it provided achievement test data several years before and after the initial ASE certification. In addition, ASE certification is valid only for 5 years so that schools that received ASE certification in
1985 would have been due for recertification in 1990.
A second criterion for being included in the sample was that schools must have received ASE certification between 1985 and 1987. Of the 67 Ohio vocational automotive programs that have ASE certification, 26 met both of the above conditions. Because of the nonprobabilistic nature of the sample, the results of this research apply only to the 26 schools selected.
An adaptation of a time-series design (Campbell & Stanley, 1963) using one group with repeated measurements or observations over a period of time both before and after treatment was used in this study. With the time-series design, no control group is employed (Frankel & Wallen, 1990, p. 244). Since the treatment (i.e., the program's becoming ASE certified) had previously occurred, the study was ex post 18
facto in nature. Although ex post facto research may compromise some of the rigors
of the controlled experiment, it does maintain the argument and logic of experimental research. Ex post facto research is a systematic and empirical approach in which the investigator does not employ experimental manipulation nor random assignment of subjects to conditions because events have already occurred or they are inherently not manipulate (Kerlinger, 1979). For example, there was no opportunity to control conditions such as changes in teachers, vocational administrators, facilities and/or equipment, advisory committees, the ability of students who enrolled in the programs being studied, or revisions that were made in test instruments.
Summary and Outline of Dissertation by Chapter
If America is to succeed in the new world economy, it must concentrate its efforts in retraining its work force for the technological changes of today. A great number of young people who go diivctly into the work force have neither the benefit of specific job training nor support services available to college-bound youth. The lack of emphasis on skill training reflects the philosophy that success is measured by a college degree. Perhaps a signal has been heard: The introduction of national standards for skill training could be the beginning of a change in attitudes toward skilled workers.
The literature review in this study summarizes the major reports and studies that led to the public's acceptance of using national standards. Academic education was a major focus of these initial reports, but later the emphasis shifted to vocational- technical education and work force preparation. Cultural factors also affect vocational training, as we shall see. Five major proposals are briefly reviewed to 19
provide an understanding of the importance that national standards will have in future vocational-technical programs.
Six research objectives were developed to guide the study. A description of the 26 vocational automotive repair programs included in this study shows that all geographic areas of the state, both city and rural districts, and both city and joint vocational districts were represented. An adaptation of a time-series research design was used to evaluate the relationship between Automotive Service Excellence certification and scores on the Ohio Vocational Education Achievement Test.
Data used in the study, including several tables and charts that contain descriptive and inferential statistics, show the relationship between ASE certification and mean scores on the OVEAT. Findings and conclusions of the study are used to make inferences about practical applications to the operation of vocational programs.
Some unanswered questions as noted in the last chapter should serve as the basis for further investigation. CHAPTER II
REVIEW OF LITERATURE AND THEORETICAL FRAMEWORK
A review of the literature on business-education standards documented in the
Educational Resources Information Center data base and files on doctoral dissertations completed since 1985 did not result in any studies that show the advantages of using such standards in vocational-technical education. The debate over the advantages of national standards is relatively new in our country, and most of the research and literature on this topic deals with comparing the American educational system with the European and Japanese systems where national standards do exist.
Therefore, this review will examine the major events that have led to the acceptance of national education standards in America, the philosophy and rationale for existing industry-based standards such as the National Institute for Automotive
Service Excellence (ASE) certification program and the Printing Industries of
America Certification Program, and a review of the work of the National Advisory
Commission on Work-Based Learning and its Working Group on Skill Standards and
Certification.
The review of literature and theoretical framework is divided into three sections: (1) The Educational Reform Movement of the 1980s, (2) Cultural Factors
20 21
affecting Vocational-Technical Training, and (3) Current Efforts to Implement
Industry-Based Standards.
The Educational Reform Movement of the 1980s
Those concerned with improving education are most likely bewildered by the attention that education has received over the last 10 years. Unfortunately, most of the attention has been negative. Education was blamed for its graduates' poor performance on international comparison tests. Legislators, state governors, and other policy makers became actively involved in education reform as never before. In most cases, educators were not consulted.
The National Commission on Excellence in Education launched the current wave of educational reform in the early '80s with its report, A Nation at Risk: The
Imperative for Educational Reform. In this report, educators were chastised for the mediocre performance of students, primarily in math and science. According to the report, standards had fallen and educators were primarily responsible for the decline.
"Twenty-five percent of the credits earned by general track high school students are in physical and health education, work experience outside the school, remedial English and mathematics, and personal development courses, such as training for adulthood and marriage" (National Commission on Excellence in Education, 1983, p. 18).
Numerous other reports and studies quickly followed A Nation at Risk.
However, few of these addressed vocational education or students not attending college. Perhaps the first national report to address the concerns of noncollege bound students was the William T. Grant Foundation report, The
Forgotten Half. Published in 1988, the report marked a turning point in the education reform movement and warned that more resources should be focused on
students not attending college. The report noted inequities between resources
available for college and noncollege bound students According to the report, "each
student enrolled in an institution of higher education can typically receive a combined
public and private subsidy of about $5,000 per academic year" (Commision on Work,
Family, & Citizenship, 1988, p. 7). Only about 5% of youth not going on to college
can expect to receive any federally supported job training. The report noted that
those who get help receive an average of $1,800 to $2,300 per student for about 4
months of training.
The Forgotten H alf was the first of a series of reports to focus on reform of
work force education, and emphasized the need for schools to prepare students for
the world of work as well as for college. A common feature of subsequent reports
was the focus on student performance outcomes rather than method or process of
training (i.e., did the students achieve the competencies expected or required for the job). The use of performance outcome measures necessarily requires the use of
standards.
There were at least five major work force education reform proposals released
after The Forgotten H alf that included standards as a key element. One of the most
significant of the reform reports was America's Choice: High Skills or Low Wages
This bipartisan report noted that American industries can no longer afford to pay their workers high wages for low skills. There were five major recommendations in the report, but the one dealing with standards stated that "a new educational performance standard should be set for all students, to be met by age 16. This standard should be established nationally and benchmarked to the highest in the world" (Commission on the Skills of the American Work Force, 1990, p. 9). Students meeting the prescribed 23 standards would be awarded a Certificate of Initial Mastery (CIM), which would qualify them to choose among going to work, entering a college preparatory program, or studying for a Technical and Professional Certificate Any student not meeting the requirements for a CIM would have access to alternative training programs
(Commission on the Skills of the American Work Force, 1990)
Connecting School and Employment, released in 1991 by the Council of Chief
State School Officers, recommended national standards for work force training programs. The Council of Chief State School Officers recognized that a variety of approaches would be needed to meet the needs of the students and employers, and it recommended that cooperative education and youth apprenticeship be considered as possible models. It is important to note that the Council recommendations focused on student outcomes and not the method of training.
In 1989, President Bush convened a national education summit with the nation's governors at the University of Virginia. National Education Goals (U.S.
Department of Education, 1990) was a joint proposal of the National Governors
Association and the President. It included six goals for education that are to be met by the year 2000 and that are designed to make American students competitive with those from other industrialized nations. It is significant to note that the President chose the governors and not the chief state school officers to participate in this strategic planning session for the nation's educational system. The goals focused on expectations for students and adults, but offered no recommendations as to how they should be achieved. Included among the goals was the following: "Every adult
American will be literate and will possess the knowledge and skills necessary to compete in a global economy..." ("Workforce Education Reform," 1992, p.31). The report said that a procedure was needed to measure progress toward the goals. 24
In the spring of 1991 and following the development of the National
Education Goals, the Bush administration submitted America 2000: An Education
Strategy (U.S. Department of Education, 1991). It contained several recommendations for achieving the national education goals but did not directly address the issue of work force training. For example, America 2000 recommended increased flexibility for institutions in using federal, state, and local funds for projects that improve student outcomes for local schools. Although both the House and
Senate rejected the President's proposal for a 10-part categorical program, a private sector group, the New American Schools Development Corporation, has given $50 million in grants to 11 design teams to create innovative schools to implement the
National Education Goals (National Education Goals Panel, 1992)
Job Training 2000 was President Bush's plan for improving work force training. It included a recommendation for establishing a "certification system for federal vocational training that would prevent substandard vocational programs from receiving federal funds" ("Workforce Education Reform", 1992, p. 31). In addition, the President made a companion proposal for a National Youth Apprenticeship Act, which would link students and employers in a school-to-work transition program using industry-based standards as a means of ensuring high quality worker training.
At least four bills on youth apprenticeship were introduced in Congress in 1992, all of which included some form of industry-based training standards.
Ray Marshall and Marc Tucker (1992) presented a compelling argument in their book, Thinking for a Living, for adopting a system of educational standards designed to improve the performance of graduates. They maintain that American education is not without standards, but that the standards we use are not the kind that motivate students to achieve. The standards currently used deal with input. For 25
example, there are state standards for length of school year, length of school day, minimum time for class periods, maximum number of students assigned per teacher, and number of credits required for graduation. All of these standards relate to input in that they place limits on the process side of the educational system. None of the standards address the output or performance side of education. According to
Marshall and Tucker, "as matters stand now, teachers and principals cannot be held accountable for student performance outcomes for two reasons: they have never been clearly specified, and in any case, they are responsible not for student performance outcomes but for following the rules laid out in the design standards"
(Marshall & Tucker, 1992, p. 145). Marshall and Tucker suggest a system of standards with four components The first standard would be for the 20 % of students who go to college or who enter a program for technical or professional careers. The second would be for students entering the work force directly from high school. The third would be for teacher certification and the fourth for students at the time of entry into college and again at the time of graduation.
The decade of reform in the 1980s started with a broad-based attack on education and ended with a call for adopting national standards to improve work force training. Throughout this debate, little attention or recognition was given to the industry-based skill standards developed by the National Institute for Automotive
Service Excellence (ASE). The ASE program began during the mid-1970s when auto manufacturers became concerned that Congress would legislate rules for automotive technicians. This fear was the result of an outcry from consumers about low quality of service available from dealers and auto repair shops. Prior to 1975, most automobiles were "low tech," did not have sophisticated electronic controls, and could generally be repaired by persons with limited mechanical training. However, in 26
the mid-1970s pressure to design fuel-efficient and less polluting vehicles caused
manufacturers to use "high tech" electronics for precise control over fuel use and
emission. These devices required advanced diagnostic skills to repair, which most
mechanics lacked. Often, the customer had to bear the cost for the lack of skills,
since the mechanics simply removed and replaced components until the problem was found. The customer paid for unneeded parts.
In 1978, the Industry Planning Council of the American Vocational
Association, composed of representatives from the Motor Vehicle Manufacturers
Association and vocational-technical educators from the American Vocational
Association, was concerned about the perceived low quality of training programs and the lack of standards to motivate improvement in the programs. The IPC felt that the best way to improve technicians and training programs was to develop industry-based standards. Therefore, the Motor Vehicle Manufacturers Association invested approximately $400,000 over a 4 year period to develop materials for a national voluntary certification program. The project was directed by the Industry Planning
Council, but the detail work was done by the Southern Association of Colleges and
Schools (SACS). This is one of the first successful examples where representatives from competing U.S. automotive industries came together to work on a common system for evaluating technicians and training programs (Shoemaker, 1992).
In 1982 , SACS produced an evaluation guide and task and equipment lists for schools. The Industry Planning Council then developed evaluation material, and ASE developed a process for certification, maintaining certification records, and periodically updating the task and equipment lists. The ASE Board accepted the responsibility of evaluating and certifying programs, and conducted a market study to define the certification process. The study revealed that schools supported industry- based standards, but could not bear all of the costs for certification. A continued investment from the automotive industry would be needed to make the certification program successful. In 1983, ASE formed the National Automotive Technicians
Education Foundation to provide a tax-deductible means for industries to make contributions because ASE is not classified as a 501(c)(3) organization. Although the
National Automotive Technicians Education Foundation serves as the source of funding for the certification program, the ASE Board remains involved in approving policies and procedures associated with certification. The plan that the Foundation and ASE approved for certification included the following:
A set of standards for program certification; a plan for self study, review of self study materials at the national level, a team review of those programs whose self study met industry standards, and certification of programs that were reported by the review team as meeting ASE standards; provision for training evaluation team leaders in each state after application by the state and two programs being ready for team review; standards for evaluation team leaders and team members; methods of measuring evaluations; procedures for appeal of certification decisions. (National Institute for Automotive Service Excellence, undated, p. 2)
A school wishing to become certified must first request a self-review packet from ASE and conduct a self-review of the program. If the self-review is approved by ASE, then an Evaluation Team Leader is assigned to assemble an evaluation team to conduct a two-day on-site review of the program. There are eight specialty areas in which a program can be certified: automatic transmission/transaxle, brakes, electrical systems, engine performance, engine repair, heating and air conditioning, manual drive train and axfes, and suspension and steering. Programs must qualify for at least three areas in order to be certified and the instructor(s) must also be ASE certified in the areas they teach. There are a minimum number of instructional hours 28 required for each specialty area. Each specialty area is subdivided into tasks, some of which are designated as high priority. At least 80 % of the high priority tasks must be included in the curriculum in order for a program to be considered for certification.
All programs are evaluated on the basis of 10 standards: purpose, administration, learning resources, finances, student services, instruction, equipment, facilities, instructional staff, and cooperative agreements. If the on-site review team finds that a program meets all of the criteria then it is recommended for certification in the areas requested. Programs meeting the criteria for certification are awarded a plaque by
ASE which lists the areas of certification (Shoemaker, 1992). In addition, graduates from ASE certified programs are eligible to receive a certificate showing the areas of certification. The program certification is effective for 5 years, at which time the program must go through a similar, but less intensive review. Although the ASE certification program is voluntary, approximately 11 states have implemented policies that will deny state and federal funds to those not gaining certification (ibid). Ohio leads the nation in programs certified with 67.
Because the automotive industry is interested in helping schools improve their automotive technician training programs, they make a significant annual contribution to the National Automotive Technicians Education Foundation to ensure that the costs of program certification for schools are kept to a minimum. The ASE program is economical and effective because of its broad base of industry support. Auto manufacturers, tire companies, after-market parts suppliers, auto dealers, and independent garage owners are among the industry groups that endorse and support
ASE certification. Their financial support and technical advice has made it possible to establish a national office and to hire staff to evaluate program applications, keep the program up-to-date, maintain records, and develop public relations materials. Costs 29
of certification include $35 for a packet of self-review materials and about $535 for
an on-site team review and review materials (National Automotive Technicians
Education Foundation, undated, p.3). Local car dealers and independent auto service
agencies are encouraged to help local schools conduct the on-site review by providing
technicians to serve on the two-day review at no charge to the schools. The $535 fee
helps to pay for the expenses of the trained evaluation team leader.
In September, 1989, ASE approved a certification program for auto body
repair training programs that was patterned after the automotive program. Ohio also
leads the nation in this area with 11 auto body programs certified. In 1992, ASE
approved a truck repair certification program, which means ASE now offers three
different sets of national voluntary standards for vocational-technical training
programs.
In addition to the certification programs offered by ASE, in 1988 the Printing
Industries of America developed a certification program for vocational-technical
programs that was patterned after the ASE program. It is called the PrintEd program and is based on 335 tasks divided into six certification areas. Students are eligible to receive a certificate in one or more of the skill areas if they meet 80 % or more of the tasks at specified levels (Center for Remediation Design, 1991). Ohio is currently pilot-testing the PrintEd program in three different schools with plans to expand to all schools offering printing programs in 1993. The PrintEd program has not been as widely publicized as the ASE program and therefore has not been universally accepted among vocational-technical educators.
Numerous other industry or trade associations have developed, or are in the process of developing, industry-based standards for vocational-technical programs.
These include the American Welding Association, the Association of General 30
Contractors, the Society of Manufacturing Engineers, and the Air-Conditioning and
Refrigeration Institute. None of these efforts have enjoyed the national reputation of
ASE, but they are effective in states where the association has actively promoted
linkages with vocational-technical education.
Cultural Factors Affecting Vocational-Technical Training
The author visited the Netherlands in 1991 and observed their excellent
vocational training system first hand. Eleven different Dutch trade associations have
established standards that students must meet before they can be employed. The
trade associations work closely with employers, education, and government to
monitor training programs to make sure they are up to date and meet the needs of
employers. Dutch educators are keenly aware that as soon as the European Common
Market is organized, the issue o f training standards will become even more critical.
Then, if all goes as planned, people, goods, and services will be permitted to move
freely among member countries and a general system of standards that will be
recognized throughout Europe will be required for all training programs.
In May, 1993, the author visited the Federal Republic of Germany to learn
more about their system for training skilled workers and specifically to study their dual system of vocational training. The dual system, so named because part of the training is provided by the employer and part is provided by a vocational school
(berufschule), is the system through which most Germans must pass for skilled training. There are no predetermined standards that dictate how much will be taught on the job and how much will be taught in school, since this decision is determined by the nature of the occupation. Committees composed of educators and industry 31
representatives who are familiar with the occupation recommend a format for
achieving the required training to the Federal Minister for Education for approval.
German and American employer attitudes differ in that in Germany it is considered an honor for an employer to hold a license for training apprentices. About 50 % of
German employers enthusiastically support skill training for apprentices and say that training of youth is too important to be left entirely up to educators. Apprentices usually spend 1 or 2 days in the vocational school and the remainder of the week with their firms. A portion of the time with the firm may be spent in interfirm workshops.
These special facilities, built with federal subsidies to assist consortia o f smaller firms, provide the full range of training in an occupation. Local administration of the training is carried out by the state and local branches of the Chambers of Industry and
Commerce, Chambers of Handicrafts, Chambers of Agriculture, and Chambers of
Professions. These groups, known as "competent bodies," supervise the execution of training, determine the suitability of enterprises to train, and conduct examinations for apprentices and for trainers ( meisters). All firms with five or more employees must belong to a Chamber and pay dues, and a majority of the dues are spent on administering the training system.
In addition to the excellent German and Dutch vocational training systems which use nationally recognized standards, Australia and New Zealand have become the most recent industrialized nations to adopt a system of national industry-based training standards. According to a recent Carl Deusberg Society publication, an
Australian National Training Board was established through legislation in 1990 to meet the challenges of international competition. This board is a three-part organization, with representatives from labor, industry, and government, whose function is to develop national occupational competency standards. The National 32
Training Board brings industry partners together to determine standards that are
broad based and allow for further specialization in later years of training. This broad
training also allows trainees to move around within an occupational area (Anderson,
1993, p. 2).
In New Zealand, new national training qualifications for the aircraft industry
are being adopted to improve the quality of training and to reduce the duplication of training offered by various providers. According to a New Zealand Qualifications
Authority report, "there’s not going to be duplication of training in the industry any more. Wherever people train they will come out with nationally-recognized qualifications" (Hall, 1992, p.2).
Unlike their European and Asian counterparts, America's schools have not traditionally had national standards. Education was a local responsibility, under local control. Education was a privilege and a right reserved to the states by the nation's founders. However, unlike today, the founding fathers were not in an international economic war with the nation's future at stake. America no longer has the luxury of entrusting its national well-being to local boards' perception of educational needs.
Most young people who do not plan to go to college leave school without the knowledge or foundation required to find and hold a good job. Consequently, employers no longer value the high school diploma as a certificate of mastery.
Further, the American educational system has adopted the philosophy that no one should fail, that it is better to lower standards so that everyone can pass. Students are free to select unchallenging courses that fail to prepare them adequately for the workplace. "The sad fact is that not enough is expected of American students. They have it too easy academically. Yet, we expect — and get — a good deal in athletics and other extracurricular pursuits" (Kearns & Doyle, 1991, p. 79). The SCANS 33 report stated "unless all of us work together to turn this situation around, our nation will pay a very high price - low skills lead to low wages and low profits" (Secretary's
Commission on Achieving Necessary Skills, 1991, p. 7).
When President Bush met with the nation's governors in September 1989, he introduced the idea of national education goals. Twenty years ago, his announcement would have meant political suicide because of the emphasis in this country on local control. Today, however, the U.S. is in an economic war of survival with other industrial nations and education has a critical role in its success as a world leader. In a pamphlet titled National Goals for Education, Lauro F. Cavazos stated that
"America's educational performance must be second to none in the 21st century.
Education is central to our quality of life.... Education is the key to America's international competitiveness.... All of our people, not just a few, must be able to think for a living, adapt to changing environments, and to understand the world around them" (U.S. Department of Education, 1990, p. I).
The six goals that the President and governors defined are about excellence and will force education to accept a new direction. Although all of the goals apply to vocational education, Goal Five specifically addresses the acquisition of skills and knowledge necessary to compete in a global economy.
National Education Goals
1. By the year 2000, all children in America will start school ready to learn.
2. By the year 2000, the high school graduation rate will increase to at least 90 percent.
3. By the year 2000, American students will leave grades four, eight, and twelve having demonstrated competency in challenging subject matter including 34 English, mathematics, science, history, and geography; and every school in America will ensure that all students learn to use their minds well, so they may be prepared for responsible citizenship, further learning, and productive employment in our modem economy.
4. By the year 2000, U.S. students will be first in the world in science and mathematics achievement.
5. By the year 2000, every adult American will be literate and will possess the knowledge and skills necessary to compete in a global economy and exercise the right and responsibilities of citizenship.
6. By the year 2000, every school in America will be free of drugs and violence and will offer a disciplined environment conducive to learning. (U.S. Department of Education, 1990, pp. 4-8)
Among the factors that have led to the public's acceptance of using national
standards are rapid technological change, intense international competition, poor performance of students when compared to students from foreign industrialized nations, and a national interest in increased accountability for the educational system.
Although most Americans will agree that using national standards for education is a good idea, implementing the standards at the local level may be a difficult task. The decentralized governance of U. S. public schools will create the greatest roadblock.
In a recent U. S. News and World Report article, Thomas Toth states that "it is tremendously difficult to build national momentum behind a particular reform when
50 state legislatures, 50 state boards, and more than 15,000 local school boards have veto power" (Toth, 1993, p. 61). It is worth noting that all of the current reform efforts to establish national goals or standards are careful to imply that they are national voluntary standards as opposed to required federal standards. This concession is provided to allow time to make the transition, but eventually, required 35
standards will be needed to define student outcomes. Local agencies will still have
the flexibility to decide how the standards will be met.
Decisions about what is taught in schools rest primarily with local boards of
education. This concept of local autonomy served America well before the onset of
international economic competition. It was a relatively easy task for local boards of
education to determine what skills were needed in local industries and plan their
training programs accordingly. Up to this time, "local control" had been a sacred part
of our education system, but now surveys make it clear that most Americans favor
national education goals — and endorse the six that the president and governors have
set. There are some who argue that adoption of national standards will eliminate local
autonomy and lead to a rigid learning environment that could prevent access for
minority children. However, Diane Ravitch disagreed: "Far from standardizing and
homogenizing American education and culture, these voluntary standards will offer a vision of excellence that can be reached through many different approaches. And far from creating new barriers for minority children, the new standards will provide new hope by raising expectations and by establishing a vision of what is possible for all children" (Ravitch, 1992, p. 13).
In the early days of this nation, no one worried about competition from abroad
— concerns centered around local interests and teaching the three R's. Technology was not an issue. Many unskilled jobs were available at acceptable wages. The educational system did a reasonably good job of meeting the demands of society. It provided the country with an excellent supply of well-trained scientists, engineers, and medical practitioners, who helped keep America's industrial society in a world leadership role that was relatively unchallenged, and one that enjoyed a very high standard of living. During this time, concepts such as statistical process control, total 36
quality management, and world-class manufacturing were not in the everyday
vocabulary. Most factory jobs did not require complex skills and could be filled by
workers with a high school education or less Americans were generally satisfied with the quality of their education system and were content to give local citizens the power to decide what was best for the education of youth. For most of this century, as it took its goods and know-how to the world, America did not have to worry about competition from abroad. At home, the technology of mass production required discipline on the assembly line, rather than thinking skills or individual initiative. It is worth noting that after World War II, when Americans occupied Japan, one of the changes imposed by the occupation forces was locally elected school boards.
However, this practice was soon abandoned, and the Japanese reverted to national control of schools because "local control produced inequities among schools, especially in the standards set for students" (Hildebrand, 1992, p. 27).
However, the world has changed, and American industry is facing stiff competition from abroad that demands constant improvement in quality and productivity. Firms must meet world-class standards and so must workers.
Employers seek employees with adaptability and ability to learn and work in teams.
No longer available are high-paying assembly line jobs that require little problem solving ability or training. Employees must be able to set goals and plan a means of attaining them. They must be able to operate high-tech equipment and apply their knowledge in new ways to complex problems. They must have people skills and the work ethic to succeed, and they must have adequate training.
The United States is the only industrialized nation in the world that does not have a national policy for training, nor any national industry-based training standards.
The result is a weak link between training that students receive and job standards set 37
by industry. "America's competitors, on the other hand, have high educational
standards for front-line workers, make intensive efforts to help every student meet
those standards (their dropout rate is dramatically lower than in the U. S.), and make
extensive provision for the transition from school to work" (Marshall & Tucker,
1992, p. 69). Fortunately, a model exists in this country that can be used to develop
the kind of training standards needed to grant skill credentials and to improve
America's competitiveness and productivity. This model is the one developed by the
automotive industry, which is called the National Institute for Automotive Service
Excellence (ASE) voluntary certification program.
Now national standards seem possible. At about the time the President was
meeting with the governors, the Carl D. Perkins Vocational and Applied Technology
Education Act (U.S. Congress, 1990) was signed into law. For the first time in the
history of federal vocational acts, a section was included that provided for the development of national industry-based standards for vocational training programs.
Section 416 of the act was patterned after the ASE standards for automotive programs. In 1992, approximately $3.5 million were authorized to develop industry- based standards for seven occupational areas. These projects were funded for 18 months with a possible renewal o f 18 months. An additional $3.5 million will be awarded in 1993 for seven different occupational areas (PY1992 Strategic Plan fo r
Developing Industry-based Skill Standards and Certification, 1992).
As a follow-up to the interest in national standards, the National Association of Trade and Industrial Education (NATIE) sponsored Workforce 2020, a national summit meeting of key trade and industrial educators, policy makers, and industry and labor leaders, to develop strategies for addressing national, voluntary, standards for vocational education programs (NATIE, 1992). 38
Current Efforts to Implement Industry-based Standards
When legislation was being written for the reauthorization of the Carl D.
Perkins Vocational Education Act of 1984, a number of vocational-technical
educators who were familiar with the ASE certification program lobbied to have the
concept of industry-based standards included as a part of the Act. Dr. Byrl
Shoemaker, serving as a consultant to the National Automotive Technicians
Education Foundation, was successful in getting language inserted in Section 416 of the act that provided for developing industry-based standards similar to the ASE program. Initially, this section received little attention and there was concern that it would not be funded at all. However, sometime after the approval of the Perkins Act and the unveiling of President Bush's America 2000: An Education Strategy (U.S.
Department of Education, 1991), key policy makers, especially those connected with the Department of Labor, began to view Section 416 of the Perkins Act as a means for promoting the America 2000 effort. In a Federal Register notice on public hearings for Section 416 of the Perkins Act (Skill Standards and Certification), the
Departments of Labor and Education were asked to jointly develop industry-based skill standards in response to the needs of business, workers, educators, training providers and governments. "Track III of Goal Five asks business and labor to adopt a strategy to establish job-related and industry-specific skill standards, built around core proficiencies, and to develop skill certificates to accompany these standards"
(Federal Register, 1992, p. 9488). The Federal Register called for comments on industry-based standards through a series of five public meetings held in major cities throughout the nation Oral and written testimony was presented by over 200 people who represented government, education, industry, organized labor, joint labor- 39 management committees, community-based organizations, and other service providers and private citizens. An analysis of the testimony revealed almost unanimous support for developing industry-based standards. "Of the eight stakeholder groups, the joint labor/management group was the only one for which unfavorable responses outnumbered favorable" ( Voluntary Industry-based Skill Standards and
Certification: A Final Report, 1992, p. 41). Organized labor has traditionally opposed any efforts that pose a potential threat to the existing national registered apprenticeship system. Organized labor viewed the effort to develop voluntary, industry-based skill standards as a threat because of the possibility of diverting federal resources from the Department of Labor's Bureau of Apprenticeship and Training, undermining the collective bargaining process, and diluting standards already in place
(Office of Work-Based Learning, 1992).
Both the Departments of Education and Labor are now cooperating in a joint effort to address the issue of industry-based standards through the National Advisory
Commission on Work-Based Learning. Approximately $3.5 million was appropriated for FY 91 through grants to develop industry-based standards in seven different occupational areas. The purpose of the grants was to organize and operate business- labor-education technical committees ( Federal Register, 1992). Grant applications were limited to industrial trade organizations, labor organizations, national joint apprenticeship committees, or comparable national organizations. Grant recipients were to propose national standards for competencies in industries and trades.
According to the Federal Register, the standards as a minimum shall include criteria for "major divisions or specialty areas identified within occupations studied, minimum hours of study to be competent in such divisions or specialty areas, minimum tools and equipment required for such divisions..., minimum qualifications for instructional 40
staff, and minimum tasks to be included in any course of study purporting to prepare
individuals for work. . . " (Federal Register, 1992, p. 9488).
The Departments of Labor and Education have adopted, with the assistance of
the National Advisory Commission on Work-Based Learning, a four-point approach
to develop and implement voluntary industry-based skill standards and certification,
which includes the following.
1. To develop a working partnership with key players in industry, education, labor, state and federal governments to determine the demand for and shape of a national framework for skills standards through a public dialogue;
2. To initiate a number of pilot projects with key industries to develop, on a trial basis, approaches to voluntary industry-based standards and certifications;
3. To establish a research and technical assistance program to support the public dialogue and pilot projects; and
4. To provide national leadership and coordination to the industry, labor, state and education groups working to develop voluntary industry-based skills standards and certification arrangements. (Voluntary Industry-Based Skill Standards and Certification:P Y 1992 Strategic Plan, July 6, 1992, p. 2)
The initial grant recipients, announced on October 26, 1992, included the
Graphic Arts Technical Foundation, the Electronic Industries Foundation, the
Foundation for Industrial Modernization, the Far West Lab for Education and
Research and Development, Southern Association of Colleges and Schools,
Educational Development Center, and the National Automotive Technicians
Education Foundation. The respective occupational areas in which each of these groups will be developing standards are printing, electronics, computer-aided 41
drafting, health science and technology, air conditioning and refrigeration,
biotechnical sciences, and automotive, auto body, and truck repair The initial grants
were for approximately $500,000 each for 18 months, after which progress will be
evaluated and grants could be extended for an additional 18 months.
A parallel but separate set of similar grants was announced by the Department
of Labor on June 3, 1992, when a "Notice Inviting Applications" was published in the
Federal Register. On September 30, 1992, the Department of Labor funded six
projects from this first competition for approximately $300,000 each (Van Erden,
1992, p. 2). The groups receiving grants and the respective occupational areas were the American Electronics Association (electronics), Council on Hotel Restaurant and
Institutional Education (tourism, travel, and hospitality), National Electrical
Contractors Association (electrical construction), and National Retail Federation
(retail trade).
Other initiatives further illustrate the interest in adopting national skill standards and certification. One example is the Technical Planning Subgroup of the
National Education Goals Panel, which is led by Marc Tucker and John Bishop
(National Education Goals Panel, 1992). This group will develop a plan to compare the skills of American workers to those of other countries, and that will include participation in an international study by the World Bank, the Educational Testing
Service, and the Organization for Economic Cooperation and Development. They also plan to define occupational skill standards, complete case studies of specific industries, and conduct interviews with managers familiar with the skills of workers in other countries.
Another example of initiatives to develop standards is the General Accounting
Office's (GAO) report of May, 1990, which compared the transition from school to 42
work in the U.S. and other industrialized countries (U.S. General Accounting Office,
May, 1990). In August, 1991, the GAO followed up with a study linking worksite
training and education through cooperative education in the U.S. (U.S. General
Accounting Office, August, 1991). This report revealed a considerable amount of
variability in the quality of cooperative education programs and recommended that
standards be developed and applied to these programs. A May, 1993, GAO report on occupational skill standards found that "organizations and industries sponsoring skill standards and certification systems believe that the time and resources devoted to developing and managing such systems were well-spent and represent wise investments in the future of their industry" (U.S. General Accounting Office, May,
1993, p. 1).
Finally, the National Governors Association is conducting research on the role of states in developing skill standards and certifications. The Great Lakes Governors
Association, which is an affiliate of the National Governors Association, has formed a consortium to work with the printing and metalworking industries in the Great Lakes states to develop skill standards and voluntary occupational skill certifications. The metalworking industry is represented in this consortium by the National Machining and Tooling Association (NMTA). NMTA is using funds from a Department of
Education grant to help develop skill standards and certifications. After development, the plan is for the standards and certifications to be recognized by employers throughout the Great Lakes region (Whitten, 1992).
Although the results of these grants will not be known for 2 to 3 years, it is clear that national voluntary industry-based standards are being viewed as a means of improving the accountability and quality of vocational-technical programs. According to one of the Department of Education grant recipients, Don Hatton, the response of 43
industry to the Department of Education grants has been overwhelming. He states,
"We have been inundated by companies and educators alike volunteering key
personnel to be a part of the development of these national standards" (Hatton, 1993).
Another example of industry support is reflected in a recent National Coalition for
Advanced Manufacturing Productivity position paper in which a key recommendation
for continuous quality improvement in vocational-technical education is to "sustain
federal support for the industry-led effort to. . .create voluntary national skill standards
for key industrial occupations in support of which the vocational-technical education
community can develop curricula" (National Coalition for Advanced Manufacturing
Productivity, 1993, p. 21).
There is growing interest in adopting voluntary, national, industry-based
standards as a means of improving the quality of training for front-line workers. As
has been noted, experience in vocational-technical education thus far with voluntary
industry-based standards has been limited to the ASE standards for the automotive
areas and PrintEd for the printing area. The literature review on the efforts of the
Departments of Labor and Education has revealed that standards for an additional 12
occupational areas could be completed within three years (PY1992 Strategic Plan fo r
Developing Voluntary Industry-Based Skill Standards and Certification, 1992).
These voluntary standards could produce dramatic impacts on stakeholders.
What are those impacts?
For students and workers, national standards would provide a means of
recognizing competencies in a particular occupation. Since standards would be jointly developed and recognized by industry, students and workers would benefit
because they could easily move ffom one area of employment to another, and because 44
they would graduate with a diploma and certificate widely recognized and valued by
industry.
For teachers, standards would provide a road map for developing curricula.
Standards would clearly define the expected outcomes for graduates of programs and
tools and equipment required to teach the curriculum. The use of industry-based
standards and performance measures could cause teaching styles to shift from the
teacher as deliverer of information to more student-centered instruction, with the
teacher as facilitator.
For employers, much of the mystery in hiring workers would be eliminated
because standards would ensure that anyone with a certificate would possess the
needed competencies. In addition, if employers were directly involved in helping
vocational-technical educators specify what students should be able to do when they
graduate, program quality should improve and students should be better prepared for
the workplace.
Taxpayers would benefit from adoption of national standards because of
improved efficiency and effectiveness in training programs. According to a Texas
Department of Commerce report, adoption of standards would convert our
"inefficient non-system of training into a coherent, standards-driven system that
improves both information and incentives for workers...and accountability for
program outcomes that will ensure taxpayer investments" (Texas Department of
Commerce, 1993, p. 4).
Finally, society would benefit because standards can "ensure that training supported by public funds includes a core set of general skills, even when the training is provided by private firms" (Batt & Osterman, 1993, p.46). 45
Summary and Theoretical Framework
The review of literature has revealed that most industrialized nations use some form of national industry-based standards to guide the training of their workers.
There are no surprises with this approach, since industry, labor, and education work closely to define the standards. In the U.S., however, training is somewhat haphazard with local schools setting standards that are often not recognized by the industries that hire graduates. The result is that many students enter the work force totally unprepared for the jobs that await them. Conditions such as this do not help industries to be competitive in world markets. There is, therefore, increasing interest in this country to adopt national, voluntary, industry-based standards similar to those developed by the National Institute for Automotive Service Excellence (ASE), with the hope that the quality of training will be improved. Current literature does not identify what effect industry-based standards have on vocational programs. This study should substantiate the relationship between ASE certification and standardized test scores.
Based on the literature, the theoretical framework for this study is depicted below:
Main Independent Rival Independent Dependent Variable Variables Variable
Language scores ASE certification Automotive scores Nonlanguage scores CHAPTER III
METHODOLOGY
The major purpose of the study was to determine whether or not there was a relationship between schools achieving the National Institute for Automotive Service
Excellence (ASE) certification and automotive scores on the junior and senior levels of the Ohio Vocational Education Achievement Test (OVEAT). The study further sought to identify whether academic aptitudes of automotive students increased or decreased during the study period for junior and senior level students.
This chapter includes the research procedures and methodology used to determine if ASE certification had any influence on the mean scores obtained by students taking the OVEAT during the years 1980 to 1990. The procedures used to conduct this study are reported in the following sections. Description of Sample,
Research Design, Data Collection, Data Analysis, and Summary.
The study was guided by the following research questions:
1. Do junior students who complete an ASE-certified program score significantly higher on the language section of the Short Form Test of Academic Achievement (SFTAA) than students who complete a non certified program?
2. Do senior students who complete an ASE-certified program score significantly higher on the language section of the SFTAA than students who complete a noncertified program? 46 47
3. Do junior students who complete an ASE-certified program score significantly higher on the nonlanguage section of the SFTAA than students who complete a noncertified program?
4. Do senior students who complete an ASE-certified program score significantly higher on the nonlanguage section of the SFTAA than students who complete a noncertified program?
5. Do junior students who complete an ASE-certified program score significantly higher on the automotive section of the OVEAT than students who complete a noncertified program?
6. Do senior students who complete an ASE-certified program score significantly higher on the automotive section of the OVEAT than students who complete a noncertified program?
The following directional hypotheses are derived from the research questions presented above and will be used to examine the trend of mean scores on the language and nonlanguage section of the SFTAA during the period of the study and to explore the relationship between student mean scores on the automotive section of the OVEAT and ASE certification:
1. The student mean language scores for juniors will not show a significant increase during the period of the study.
2. The student mean language scores for seniors will not show a significant increase during the period of the study.
3. The student mean nonlanguage scores for juniors will not show a significant increase during the period of the study.
4. The student mean nonlanguage scores for seniors will not show a significant increase during the period of the study. 48
5. The student mean automotive scores for juniors will show a significant improvement after programs receive ASE certification.
6. The student mean automotive scores for seniors will show a significant improvement after programs receive ASE certification.
Description of Sample
The sample for this study was selected from Ohio schools that offered
vocational automotive repair programs at the junior and senior grade levels, that
achieved ASE certification, and that administered the Ohio Vocational Education
Achievement Test during the 11-year period from 1980 to 1990. Since a primary
purpose of the study was to determine the relationship between ASE certification and
mean scores on the automotive section of the OVEAT, a purposive sample was
drawn based on these criteria. Twenty-six automotive programs were identified that
met the criteria outlined above.
The National ASE office in Herndon, Virginia, where ASE historical records are kept, was contacted to determine which schools had automotive repair programs that were ASE-certified and when they were certified. The ASE certification program is voluntary and therefore not all schools have received certification. In addition, schools were encouraged to try to become certified in all eight of the specialty areas but some chose to select fewer areas because they were unprepared to meet the stringent requirements for all eight areas. From the list of over 60 Ohio ASE certified schools provided by the national ASE office, only those in which the automotive programs received ASE certification in 1985 to 1987 were selected. The certification process often took 1 to 2 years, especially when large equipment 49
purchases had to be made in order for the program to meet the criteria, so that
schools which were officially certified in 1987 actually began making program
improvements a year or two earlier, starting in 1984 when the ASE certification
program was first initiated. The first opportunity for vocational automotive programs
to obtain ASE certification was in 1985. The researcher felt that a span of 7 years
prior to certification and 6 six years during and after (including the year of
certification) would provide a sufficient time period to observe the trends in the
automotive and academic aptitude test scores. It must be noted that not all schools
had this generous time span, but that data were collected representing these extremes
of years.
Like the ASE certification, the OVEAT was also voluntary and a list of schools that had administered the automotive section of the test was obtained from the Instructional Materials Lab at The Ohio State University. In selecting schools on the basis of the achievement test, those which administered the OVEAT to junior and senior level automotive students between 1980 and 1990 were selected. The reasons for selecting the 1980 to 1990 period were that (1) 1990 was the last year that the
OVEAT was available and (2) it was felt that an 11-year period (several years before and after gaining ASE certification) would provide sufficient data to show significant trends in the test scores.
Twenty-six programs were identified that met the criteria described above.
Table 1 lists the schools in which the programs were located, the year of program certification, and the number of ASE specialty areas that were achieved by each program. The 26 different programs included in the study represent a mixture of urban and rural, joint vocational schools and city districts, and geographic regions from north, south, east, and west portions of Ohio. Only two of the schools were city 50
districts. Twenty-four were joint vocational schools. Urban areas included both the
two city schools as well as several joint vocational schools. Many rural areas were
represented. All schools except two were certified in five or more ASE specialty
areas. Thirteen schools were certified in all eight of the ASE specialty areas. Two
schools were certified in only three of eight specialty areas. ASE areas of
specialization include automatic transmission/transaxle, brakes, electrical systems,
engine performance, engine repair, heating and air conditioning, manual drive train
and axles, and suspension and steering.
In all succeeding tables, scores will be assigned to schools listed by letter rather than school names to permit anonymity. 51
Table 1
Schools Included in Study
School Name Year of Areas of Certification Certification
Eastland JVS 1985 7 Montgomery Co. JVS 1985 8 Montgomery South 1985 8 Montgomery North 1985 8 Montgomery East 1985 8 Montgomery West 1985 8 Ashtabula JVS 1986 8 Buckeye JVS 1986 8 Central Ohio JVS 1986 8 Ehove JVS 1986 5 Guernsey-Noble JVS 1986 5 Jefferson Co. JVS 1986 7 Laurel Oaks JVS 1986 5 Tri-County JVS 1986 5 Belmont Co. JVS 1987 8 Columbiana Co. JVS 1987 7 Diamond Oaks JVS 1987 5 Licking Co. JVS 1987 7 Live Oaks JVS 1987 3 Northland, Columbus City 1987 8 Northwest Career Ctr 1987 5 Penta JVS 1987 8 Penta South 1987 8 Pickaway-Ross JVS 1987 8 Scarlet Oaks JVS 1987 5 Vanguard JVS 1987 3 52 Research Design
This study reviewed test scores from the Ohio Vocational Education
Achievement Test given to automotive students over an 11-year period from 1980 to
1990 to determine whether there were significant changes in the mean test scores for
the automotive section after the programs became ASE certified. Since there was no
opportunity to manipulate an independent variable and the sample was not randomly
selected, the study was ex post facto in nature. The design that most closely
describes this study is the time-series experiment, in which several measurements are
taken on a group or individual, then a treatment is introduced and more measurements
are taken to determine if the treatment had any effect on the measurements (Campbell
6 Stanley, 1963). Since it was impossible to take measurements on the same group
of students, the author used an adaptation of the time-series design and compared test
data from different groups of students over the 11-year period from 1980 to 1990.
Graphically, the design can be shown as follows:
0j0203040^)7X 08090jq0jj
WhereX represents the ASE certification obtained by the 26 programs, 0 2 -
0 7 represent the students' mean score on the automotive section of the OVEAT before ASE certification, and 0 8-O u represent the students' mean score on the automotive section of the OVEAT after ASE certification. 53 Data Collection
The data for this study were obtained from two sources: (1) the Instructional
Materials Laboratory at The Ohio State University in Columbus, Ohio, and (2) the
National Institute for Automotive Service Excellence in Herndon, Virginia. The
information obtained from the Instructional Materials Laboratory included scores
from the Ohio Vocational Education Achievement Test. "The [OVEAT] tests are
specially designed instruments for use by teachers, supervisors, and administrators
evaluating and diagnosing vocational achievement in order to improve instruction"
{Ohio Vocational Education Achievement Test Program, 1989, p. 3). The OVEAT
test includes two parts: the achievement test relating to the trade area and the
academic aptitude test known as the Short Form Test of Academic Aptitude
(SFTAA). The OVEAT was published by the Ohio Department of Education,
Division of Vocational and Career Education. Data from the achievement test section
relating to the automotive trade were used to determine if implementing the ASE
certification program had any effect on the mean scores for this section.
According to the OVEAT manual, tests are administered during any three consecutive
school days. Ohio schools administer the tests during the first 3 weeks in March.
The first day of testing, which requires approximately 1 hour, is devoted to obtaining demographic information and administering the SFTAA. The automotive portion of the OVEAT is administered during the second and third days of testing and requires 2 hours each day. The language section vocabulary subtest contains 25 items, and the memory subtest contains 20 items. The nonlanguage section analogy subtest contains
20 items, and the sequences subtest contains 20 items. The occupation portion of the test contains between 340 and 395 items {OVEATProgram, 1989). Seniors take the same tests as juniors, according to Instructional Materials Laboratory staff. 54
The SFTAA was published by CTB Macmillan/McGraw-Hill Publishing
Company, and is a series of academic aptitude tests for use with students in grades
1.5 through 12.9. Level 5 is the recommended test for grades 9 through 12.9, and is
used in conjunction with the OVEAT. The SFTAA contains two sections: a language
section that is composed of two separately timed subtests, vocabulary and memory; and a nonlanguage section that includes analogies and sequences, both of which are also separately timed. The emphasis in the SFTAA is on the measurement of academic aptitude rather than on speed (Sullivan, Clark, and Tiegs, 1971). Data from language and nonlanguage sections are used to show whether academic aptitudes of automotive students changed significantly during the 11-year period.
The OVEAT was developed by a panel of experts, primarily vocational automotive instructors, to measure skills and understanding. The test items measure students' ability to:
1. Solve problems
2. Analyze data
3. Recall specific facts
4. Use one's knowledge of principles
5. React to generalizations
6. Use abstract thinking in specific situations
7. Put together parts to form a complete structure
(OVEATProgram, 1989, p. 3).
Twenty-six schools were included in the sample: most schools had a junior and a senior class taking the OVEAT each year. Therefore, there were approximately
50 classes contributing test data. The average class size was 18, so that roughly 900 students were involved each year. Since many students took the test in both their 55
junior and senior years, approximately 5,000 different students took the tests during
the 11-year period from 1980 to 1990, and approximately 9,900 scores were used to
compute the data. Three sets of data were collected for each student: (1) a score on
the automotive section, (2) a score on the nonlanguage section, and (3) a score on
the language section. The mean scores for each class were then computed for the
automotive, language, and nonlanguage sections of the OVEAT. The mean scores
were among several descriptive statistics that were computed and listed on the forms
that were distributed to local schools as a part of the Instructional Materials
Laboratory testing program.
Data Analysis
The initial step in analyzing the data was to identify those schools that were
among the first to achieve ASE certification. A list of these schools was obtained
from the national office of ASE. Because of the preparation time required for schools
to get their automotive programs ready for the ASE self-review and on-site review,
the researcher felt that using a certification date and comparing scores prior to and
after that date would accurately represent the impact of certification on schools. The
certification process often took 1 to 2 years, especially when large equipment
purchases had to be made in order for the program to meet the criteria. Therefore,
schools that were certified in 1987 actually were making program improvements
starting as early as 1984 when the ASE application process was officially initiated.
Schools that had given the OVEAT over the 11 -year period were next matched with those that had achieved ASE certification . This resulted in a purposive sample of 26 programs. The range of years for which data were available was 1980 to 1990. All schools in the sample received certification in the years 1985, 1986, or 56
1987. Multiple years of certification were necessary to obtain a sufficient number of
schools for the study. The year of ASE certification became the common baseline
from which comparisons in the study were made. All data collected prior to the year
of ASE certification were placed in the pre-ASE group, and ranged from 5 to 7 years,
depending on the year of ASE certification. All data collected during and after the
year of ASE certification were placed in the post-ASE group, and ranged from 4 to 6
years. For example, a program that became ASE-certified in 1985 would have 5 years in the pre-ASE phase and 6 years in the post-ASE phase.
Not all years are complete with data for all programs. Limiting the selection of schools to those which participated in the OVEAT test program every year resulted in an inadequate sample of schools. In the final analysis, the author selected all schools that had data available and were certified in the years 1985 to 1987. In some cases, there were branch campuses that had few scores. However, it was felt that since the main campus was included with the branch campus in terms of certification, all branch campus scores should be included. Seven of the programs studied were in actuality only two school districts.
All test scores were analyzed with the Statistical Analysis System (SAS) package for the social sciences, and Microsoft Excel's statistical program. Both types of software are designed primarily to perform statistical analyses and to provide a means for reporting the results. The mean score data collected from the Instructional
Materials Laboratory for the automotive, language, and nonlanguage sections of the tests were transferred to a format that was recognizable by SAS. A special SAS program was used to select and group all of the junior mean scores for each of the three tests and all of the senior mean scores for each of the three tests for each year.
Then, using the 26 junior mean scores for each test, a mean was computed for each 57 year for each test. A similar procedure was completed for the senior mean scores.
Both groups were then divided into pre-and post-ASE groups to compare their performance on all three sections of the test. Graphs were then plotted to show results before and after ASE certification. The next step was to apply a two-tailed t- test for independent groups to determine whether there was a significant difference between the two groups. The final step was to obtain a calculated value of t and to compare it to a critical value of t to determine the probability of the difference in mean scores occurring by chance. The t-test is a statistical model that is used for testing the significant difference between two populations based on the means and distributions of two samples (Williams, 1986).
Given the exploratory nature of this study and the nonprobabilistic nature of the sample, the determination of an alpha level was unnecessary. However, an alpha level of .05 was used as a guide to help interpret the practical significance of the data.
Summary
The purposes of this study were as follows: (1), to determine if there was a relationship between ASE certification of automotive programs and student performance on automotive sections of the OVEAT, and (2), to determine whether during the same time period, the academic aptitude of students taking the SFTAA test changed significantly. The researcher used an ex post facto study and therefore had to limit the extent to which generalizations could be made about the results.
This chapter presented a description of the sample, the research design, data collection procedures, and data analysis. In order to compare the performance of pre- and post-ASE groups on the test scores, a two-tailed t-test for independent groups was calculated. CHAPTER IV
RESULTS OF THE STUDY
Introduction
Chapter IV, Results of the Study, is organized into three sections. The first section is an introduction to the results of the study, which includes a discussion of the participants in the study and of methods of reporting the findings. The second section presents the hypotheses testing. A table of results accompanies each research hypothesis to help the reader in analyzing the data. Finally, a summary of the research findings is presented in the last section.
Schools and Individuals Selected for Study
There were 26 vocational automotive programs selected for this study, all of which were among the first in Ohio to receive ASE certification. Since the ASE certification program is voluntary, it is likely that schools which participated early in the program were progressive and open to new ideas. In that sense, the schools included in this sample are somewhat self-selected and unique among their peers. The sample, therefore, cannot be considered a random sample.
All of the programs had also participated in the Ohio Vocational Education
Achievement Test (OVEAT) program. The average class enrollment was 18 students. Approximately 9,900 student scores on the automotive OVEAT from the
59 60
26 programs for the 11 -year period were used to compute mean scores for the study.
Only those programs which obtained ASE certification in 1985, 1986, or 1987 were
included. The maximum time range for each school's scores is 7 years prior to ASE
certification and 6 years during and after ASE certification. Data from the OVEAT
were available only for the years 1980 to 1990; therefore, schools that were certified
in 1985 have data 5 years before and 6 years during and after ASE certification
Schools that obtained ASE certification in 1987 have data 7 years before and 4 years
during and after ASE certification. Programs that did not participate in the OVEAT
every year have missing scores.
The 26 programs included in the study were a mixture of urban and rural, joint
vocational schools and city districts, and geographic regions from north, south, east,
and west portions of Ohio. Only two of the schools were city districts. Twenty-four
were joint vocational schools. Urban areas included two city schools as well as
several joint vocational schools. Many rural areas were represented. All schools
except two were certified in five or more ASE specialty areas. Thirteen schools were
certified in all eight of the ASE specialty areas. Two -schools were certified in only three of eight specialty areas. ASE areas of specialization include automatic transmission/transaxle, brakes, electrical systems, engine performance, engine repair, heating and air conditioning, manual drive train and axles, and suspension and steering. Methods of Reporting Findings
The research design used in this study is an adaptation of a time-series design
in which the OVEAT scores for the automotive, language, and nonlanguage sections
served as the observations before and after introduction of the National Institute for
Automotive Service Excellence (ASE) treatment. The study was ex post facto
because the treatment had already occurred and there was no opportunity to
manipulate an independent variable and no possibility of selecting a random sample.
The study is a quasi-experimental design that uses inferential statistics to explain the
changes that occurred after automotive programs received ASE certification.
Individual student scores in all 26 programs were used to compute class
means as a measure of central tendency for each of the three OVEAT sections. The
junior and senior level mean scores for the language, nonlanguage, and automotive
tests are presented in Tables 2 through 7 respectively. All junior scores from 25
programs were grouped for each year from 1980 to 1990 — in one program, juniors
did not participate in the OVEAT. A similar procedure was used for senior scores.
There was also one program that did not have seniors participate. The scores were
separated into junior and senior levels to allow an evaluation of the relationship
between ASE certification and OVEAT scores for junior and senior levels. In
addition, both junior and senior scores were arranged according to the year of ASE
certification. That is, mean scores in years prior to ASE certification were placed in
the pre-ASE category, and mean scores during and after the ASE certification year
were placed in the post-ASE category. Once data were assigned to the two study
groups, a mean for each year in relation to the ASE certification year for each level
and for each test was computed. These figures produced the results of the study, which included pre- and post-ASE scores, both junior and senior level, for each year 62 for the three tests. Actual data for the junior and senior mean scores of each of the tests, language, nonlanguage, and automotive, appear in Tables 2 through 7. Table 2
Junior Language Class Means
{school 7 yr pre 6 yr pre 6 yr pre 4 yr pre 3 yr pre 2 yr pre A Junior 20 30 25.40 25.60 25.40 23 80 B Junior 21.50 21.30 21.90 19.60 19.50 20.10 21.20 21.20 20.30 19.70 C Junior D Junior 22.60 21.60 18.50 22.80 24.20 19.50 27.90 21.60 27.60 19.80 20.00 E Junior 20.90 21.10 20.20 19.40 19.20 21.50 27.00 21.20 21.60 21.10 14.50 F Junior 23 60 23.90 20.30 21.40 20.80 22.10 21.40 20 70 21.60 G Junior 25.70 2420 22.00 25.60 23.40 22.60 21.70 26 20 24.10 22 20 21.10 H Junior 27.50 18.70 1 Junior 21.30 20.50 J Junior 22.70 23.30 20.40 2260 28.50 20.90 25.00 21.10 20.50 K Junior 19.60 19.70 21.10 20.60 20.20 21.50 23.20 22 20 15.70 17.00 L Junior 28.50 27.40 25.20 23.00 22.80 20.60 22 80 22.30 24.80 21 00 16.50 M Junior 24.40 20.70 26.20 21.30 24.80 18.30 N Junior 19.50 22.60 21.90 25.50 20.80 24.30 23 50 25.40 20 90 O Junior 21.30 23.50 27.00 22.80 23.90 18.40 21.70 P Junior 22.80 25.80 20.90 2430 23.00 22.20 Q Junior 26.00 R Junior 24.60 26.70 S Junior 22.70 23.20 T Junior 26 90 24.20 25.00 18.80 24.10 21.60 18.00 23.30 25.90 22.50 23.10 U Junior 21.90 21.50 23.30 21.70 22.80 22.10 17.60 26.10 23 60 22.40 V Junior 22.90 17.30 20.90 19.70 W Junior 22.20 2480 25.00 27.10 23.60 21 50 23.00 23.90 26.00 22 90 22.50 X Junior 22.30 24.30 21.60 18.80 23.80 19.60 2350 23.10 21 40 18.00 Y Junior 20.20 19.70 20.00 18.90 20.00 18.90 21 10 19.10 1860 16.70 14 00 Z Junior 21.90 19.40 21.80 19.20 23 30 25.90 18.90 20.80 16.60 20.00 16.40
Mean 23.65 22.81 22.44 21.38 21.93 22.34 22.51 22.84 22.13 22.87 19.71 19.93 21.23 Table 3
Senior Language Class Means
(school 7 yr pre 6 yr pre 6 yr pre 4 yr pre 3 yr pre 1 yr pre A SE yr jyrpost 2yrpost 3yrpost 4 yr post 5 yr post A Senior 23.10 24.50 23 90 23.30 28.00 24.90 26.60 B Senior 25.00 28.40 24.50 26.00 22.20 22.90 20.90 24.10 25.40 C Senior 28.30 24.00 25.30 D Senior 24.90 21.30 22.30 21.40 21.10 23.70 23.10 29.80 22.70 30.00 16.90 E Senior 22 60 22.40 24.80 23.40 24.90 17.90 23.20 21.90 F Senior 28.80 24.50 27.10 23.00 28.30 2450 21.60 26.50 19.50 G Senior 22.50 25.90 22.80 24.90 27 20 24.40 23.90 24.10 27 90 25.10 20.80 H Senior 21.80 25.50 22.20 22.20 23 80 24.30 26.40 24.30 22.30 27.20 23 70 I Senior 22.70 24.00 21.90 J Senior 24.80 25.80 26.30 22.80 17.40 K Senior 21.60 23.50 20.80 24.60 22.80 27.30 21.60 19.70 24.20 14.60 L Senior 22.10 27.80 26.10 23.40 25.60 24.10 24.40 27.10 25.00 29.30 15.60 M Senior 26.10 30.60 26.80 27.40 24.20 21.80 N Senior 21.40 25.70 27.50 28.20 28.40 25.30 22.60 24.40 22 30 O Senior 21.70 27.40 28.80 20.30 22.50 18 50 P Senior 26.60 28.00 26.00 27.30 26.40 2480 30.50 2440 24.00 Q Senior 24.50 R Senior S Senior 20.70 20.30 T Senior 28.10 26.00 28.70 20.50 28 60 21.40 17.20 24.80 24.10 16.20 17 50 U Senior 7.40 23.40 22.70 21.20 21.50 25.20 24.70 20.40 23 80 22.60 V Senior 21.60 25.70 16.90 25 50 W Senior 25.90 24.10 29.00 27.20 27.70 25.10 22.40 25.00 24.80 26 20 22.10 X Senior 20.80 2420 23.60 22 20 24.00 24.60 27.10 16.70 Y Senior 25.70 22 00 22 30 22.10 23 90 23.10 22.10 20 60 22 50 18.60 Z Senior 21.00 24.40 20.40 23.30 22 60 23.80 20.70 21.00 22.00 25.00 Table 4
Junior Nonlanguage Class Means
{school 7 yr pre 6 yr pre 6 yr pre 4 yr pre 3 yr pre 2 yr pre1 yr pre ASE yr 1 yr post 2 yr post 3 yr post 4 yr post 5 yr post | A Junior 25.10 27.60 25.50 24.50 26.40 27.20 B Junior 21.80 25.00 25.10 22.70 21.90 20.00 20.80 19.20 20.50 21.30 C Junior D Junior 25.30 25.90 23.80 23.60 23.50 24.70 27.60 25.80 28.40 20.90 23.80 E Junior 26.00 23.10 24.10 25.50 24.20 22.50 23.00 26.10 20.90 24.90 21.30 F Junior 25.60 26.80 25.20 26.10 26.40 25.60 26.10 19.40 27.30 G Junior 28.00 26.30 28.10 25.60 25.60 23.90 25.90 25.90 27.60 27 30 24.40 H Junior 30.00 24.30 I Junior 25.40 20.60 J Junior 2420 23.30 24.70 21.90 27.00 23.60 25.20 25.70 22 90 K Junior 22 90 22.90 22.70 22.50 22.10 22.90 23.80 22.70 21.70 20 80 L Junior 29.20 33.40 30.10 24.50 22.20 24.40 22.50 25.10 24.90 23.30 20.50 M Junior 25.00 27.40 25.00 25.60 27.10 24.00 24.80 28.50 19.60 25.20 N Junior 24.00 25.30 24.00 25.80 26.30 27.70 21.00 30.10 2400 0 Junior 24.10 27.00 25.60 25.60 25.70 23.50 24.70 P Junior 25 30 25.50 23.00 24.90 22.70 26.00 Q Junior 25.50 R Junior 25.60 26.20 S Junior 23.90 27.90 T Junior 29.40 24.80 28.50 20.70 25.20 21.80 16.00 26.70 25.40 21.10 2460 U Junior 25.10 23.00 25.00 24.40 25.00 21.60 20.90 27.00 25.00 25.60 V Junior 26.00 22.70 26.40 19.80 W Junior 25.10 2610 25.50 23.70 23.90 21 30 21.90 22.30 24.00 24.60 24.00 X Junior 26.20 25.40 2260 22.00 21.50 21.20 23.70 24.40 22.80 25.80 Y Junior 22.10 22.20 23.40 2210 21.80 21.60 22.70 21.60 20.40 22 90 17.00 Z Junior 26.90 24.50 24.90 23.10 25.80 22.80 24.90 24.00 21.60 27.50 14.40 O'* Mean 26.40 26.70 26.42 24.04 24.16 23.71 23.19 24.38 23.91 26.06 22.69 23.64 24.37 Table 5
Senior Nonlanguage Class Means
jschool 7 yr pre 6 yrpre S yr pre 4 yrpre 3 yr pre 2 yr pre lyrpre A SE yr lyrpost 2yrpost 3yrpost 4yrpost Syr post | A Senior 26 90 27.10 27.20 25.50 28.50 22.40 27.20 B Senior 28.00 31.10 2480 2430 22.80 22.40 23 20 19.80 25.00 C Senior 28 00 28.10 25.80 D Senior 27.90 25.80 27.20 24.60 22.60 25.80 25.50 27.50 27.70 30.60 19.70 E Senior 32.80 26 60 24.40 27.00 24.90 26.20 25.20 23.60 F Senior 27.90 2910 29.70 28.30 26.50 29.60 25.70 31.00 18.40 G Senior 24.70 30.50 26.40 28.30 29.20 27.40 25.30 25.20 27.80 29 90 22.10 H Senior 25.30 27.70 28.20 25.10 25.90 24.80 25.40 22.80 24.40 28.40 2420 I Senior 24.10 26.30 25.10 J Senior 2710 26.00 24.00 23 60 21 90 K Senior 24.80 2910 24.90 27.60 24.60 23.30 22.90 24.30 26.70 20 00 L Senior 29 30 30.10 32.20 25.90 27 50 24.70 27.00 28 40 27.20 26.60 21.10 M Senior 26.00 26.90 25.60 28.70 29.80 30.40 25.90 26.60 24.80 N Senior 24.90 27.50 28.30 28.80 29.10 27.70 2420 27 40 24.10 0 Senior 24.40 28.10 28.00 22.50 26 30 25.00 P Senior 25.50 24.80 27 60 26.20 26.10 28.80 29.80 31 90 26 20 Q Senior 27.00 R Senior S Senior 25.10 19.70 T Senior 30 80 29 40 31.10 22 00 28.80 24.90 18.90 27.50 28.60 24.50 16.40 U Senior 9.00 29.20 25.40 23.20 23.80 27.30 27.80 25.20 28.80 28 10 V Senior 23.30 26.50 23.80 26 70 W Senior 3010 26 80 30.20 26 90 26.00 24.80 23 70 26.80 26.50 27.00 23 00 X Senior 23 20 26 40 25.50 24.40 23 90 25.50 26 30 1860 Y Senior O n 26.40 24.50 24 30 25.80 22.00 22.00 23.90 20.60 23 80 19.10 O n Z Senior 25.30 28.70 23.20 26.00 2490 26.30 2510 24.80 24.50 25.70 27.40 Table 6
Junior Automotive Class Means
| Class 7 yr pre 6 yrpre 5yrpre 4 yr pre i yr pre 2 yr pre 1 yr pre ASE year 1 yr post 2 yr post 3 yr post 4 yr post S yr post A Junior 163.70 157.70 182.10 173.30 152.90 189.60 B Junior 125.00 129.80 146.60 139.40 119.30 137.30 146.00 172.30 153.80 139.00 C Junior D Junior 125.10 138.20 126.00 137.90 125.10 150.60 177.90 158.00 176.10 160.00 152.10 E Junior 127.70 117.30 127.40 104.70 118.90 160.50 147.00 135.20 131.30 133.90 118.90 F Junior 142.20 136.20 139.00 142.10 196.30 149.60 151.70 12210 145.90 G Junior 159 50 158.90 153.40 157.20 152.40 152.20 152.10 167.10 174.40 163.00 163 50 H Junior 124.50 128.40 I Junior 177.20 164.20 J Junior 134.30 149.50 140.20 150.70 206.50 162.80 174.30 154.00 162.00 K Junior 117.90 132.10 142.90 107.10 135.50 146.70 141.30 147.40 140.80 137.80 L Junior 163.40 170.10 165.30 153.70 159.50 166.70 172.50 147.60 180.50 162 20 171.00 M Junior 154.60 140.20 153.60 148.40 174.70 182.10 151.10 150.00 142.30 N Junior 129.00 142.40 138.20 184.20 187.10 182.80 142.00 176.30 173 20 O Junior 139.40 138.30 193.10 161.80 173.80 130 40 161.10 P Junior 142.90 129.90 145.30 163.10 162.30 160.50 Q Junior 131.20 R Junior 162.70 180.70 S Junior 147.30 151.50 T Junior 138 50 115.7 113.60 115.80 132.30 130.10 115.00 14660 134.00 161.00 255.00 U Junior 138.70 120.00 126.30 123.50 134.10 147.00 112.90 132 30 124.80 144.40 V Junior 148.20 135.30 136.60 157.10 W Junior 140.90 152.70 142.10 129.70 131.60 161.30 158.00 161.40 166.00 152.40 157.00 X Junior 141.40 139 40 132 90 117.70 149.00 131.10 148.60 140.50 147.20 161.30 Y Junior 135.20 124.60 139.20 120.30 131.20 150.10 172.50 158.90 135.60 148.00 190.00 Z Junior 141.50 122 90 112.00 115.10 115.30 146.30 143.70 132.10 109.30 142.00 103 00
Mean Scores 139.50 137.88 136.50 136.79 135.76 147.84 149.24 156.99 150.61 154.02 156.12 158.80 165.93 0\ '■J Table 7
Senior Automotive Class Means
|6iass /yrpre 6 yr pre 5yr pre 4 yr pre 3yr pre 2yrpr ■I yr pre A^E year 1 yr post 2 yr post i yr post 4 yr post 5 yr post | A Senior 218.10 202.40 168.80 187.30 194.30 218.00 216.00 B Senior 133 00 155.20 157.20 165.40 159.50 164.00 169.00 180.00 213.00 C Senior 195.70 183.80 173.20 D Senior 167.60 147.90 163.20 145.90 164.10 193.90 163.90 210.00 192.00 218.00 168.00 E Senior 154.90 147.80 164.50 138.90 186.10 126.30 159.30 162.20 F Senior 169.00 168.00 178.70 187.20 205.50 222 60 164.80 233.60 121.50 G Senior 190.40 195.20 182.20 196.50 188.80 249.30 216.50 220.30 220 20 209.20 201 20 H Senior 170.80 175.70 155.50 143.60 153.50 197.70 207.30 185.40 190.90 179.00 199 00 I Senior 158.10 216.30 202.60 J Senior 201.70 171.10 170.40 170.90 187.00 K Senior 143.90 148.10 155.20 179.00 187.60 192.90 192.00 166.10 157 60 157.00 L Senior 173.80 160.80 157.90 183.60 170.20 263.60 224.60 233.50 251.00 225 00 210.00 M Senior 163.50 193.00 183.80 194.70 241.70 193.00 186.90 143.40 190 30 N Senior 155.00 162.10 167.50 210.90 205.30 241.00 160.80 160.70 206.80 O Senior 148.90 178.70 242.70 187.60 179.20 226.10 P Senior 173.40 166.50 145.40 211.10 213.50 199.20 222.60 221.40 237.60 Q Senior 158.40 R Senior S Senior 205.90 147.00 T Senior 147.40 143.1 161.80 116.40 144.90 159.30 104.60 165.60 175.70 123.00 153.00 U Senior 179.10 174.30 138.50 143.60 143.80 189.20 199.30 168.50 173 80 159.60 V Senior 185.00 201.60 170.80 211.30 W Senior 194.50 189.00 210.00 205.20 185 30 226.40 218 00 226.60 209.00 222 00 211.00 X Senior 145.10 159.20 170.40 151.00 167.60 173.90 196.00 127.70 Y Senior 185.30 168.00 156.80 145.40 181.30 192.80 218.10 184.50 179.00 205.00 Z Senior 158 60 175.90 141.60 147.90 132.50 169.30 140.70 186.30 137.50 166 00
Mean 142.47 172.41 169.99164.99 160.78 172.60 186.04 194.28 199.47 177.22 186.67 185.16 69
The descriptive statistics for the junior and senior pre- and post-ASE mean scores are found in Tables 8 through 13. Tables 8 and 9, which contain language results, reveal that the means for pre- and post-ASE groups are nearly equal, with
0.99 separating the junior pre- and post-ASE groups, and 2.56 separating the senior pre- and post-ASE groups. Furthermore, the post-ASE groups on both grade levels actually scored lower than the pre-ASE groups and demonstrated a slight decline in scores on academic aptitude sections of the test for the study period.
Table 10, which contains the junior level nonlanguage results, shows a difference between the pre- and post-ASE groups of 0.55 points. The range of scores in the pre-ASE junior group was only 3 .21 points. The range for the post-ASE junior group was even less, at 1.72 points. Table 11, which contains the senior level nonlanguage results, shows a difference in means between the pre- and post-ASE senior groups of 1.41 points. The range of scores in the pre-ASE senior groups was
2.88 points, and for the post-ASE senior groups it was 1.70 points. In both the junior and senior level groups, the post-ASE mean scores are actually lower than the pre-
ASE mean scores, again demonstrating a slight decline in academic aptitude during the study period. .
Tables 12 and 13 contain the junior and senior level automotive mean scores.
There is a marked difference in the automotive section scores when compared to the language and nonlanguage sections. In both the junior and senior levels, the post-
ASE groups scored higher than the pre-ASE groups. For example, Table 12, which contains the junior level automotive results, shows a difference in means between the pre- and post-ASE groups of 29.36 points. The range for the pre-ASE junior group Table 8
Descriptive Statistics - Junior Language
Pre-ASE Junior Post-ASE Junior
Mean 22.44 Mean 21.45 Standard Error 0.27 Standard Error 0.57 Median 22.44 Median 21.68 Mode #N/A Mode #N/A Standard Deviation 0.71 Standard Deviation 1.40 Variance 0.50 Variance 1.96 Kurtosis 1.16 Kurtosis -2.13 Skewness 0.35 Skewness -0.33 Range 227 Range 3.16 Minimum 21.38 Minimum 19.71 Maximum 23.65 Maximum 22.87 Sum 157.06 Sum 128.71 Count 7 Count 6 Table 9
Descriptive Statistics - Senior Language
Pre- A SE Senior Post-ASE Senior
Mean 24.01 Mean 22 97 Standard Error 0.12 Standard Error 0 63 Median 23.93 Median 22 93 Mode #N/A Mode #N/A Standard Deviation 0.32 Standard Deviation 1.55 Variance 0.10 Variance 2.39 Kurtosis -1.01 Kurtosis -2.61 Skewness 0.77 Skewness 0.05 Range 0.83 Range 337 Minimum 23.64 Minimum 21.31 Maximum 24.47 Maximum 2468 Sum 168.10 Sum 137 83 Count 7 Count 6 Table 10
Descriptive Statistics - Junior Non-Language
Pre A SE Junior Post A SE Junior
Mean 24.65 Mean 24 10 Standard Error 0.45 Standard Error 0.25 Median 24.10 Median 2420 Mode #N/A Mode #N/A Standard Deviation 1.19 Standard Deviation 061 Variance 1.42 Variance 0.37 Kurtosis -1.66 Kurtosis -0.40 Skewness 0.23 Skewness -0 22 Range 3.21 Range 1 72 Minimum 23.09 Minimum 23.21 Maximum 26.30 Maximum 2493 Sum 172.53 Sum 144.59 Count 7.00 Count 600 Table 11
Descriptive Statistics - Senior Non-Language
Pre A SE Senior Post A SE Senior
Mean 26.52 Mean 25 11 Standard Error 0.39 Standard Error 0.27 Median 26.17 Median 25 05 Mode #N/A Mode #N/A Standard Deviation 1.04 Standard Deviation 067 Variance 1.08 Variance 0.45 Kurtosis 1.39 Kurtosis -1 79 Skewness 1.44 Skewness 029 Range 2.88 Range 1 70 Minimum 25.62 Minimum 24 35 Maximum 28.50 Maximum 26 05 Sum 185.61 Sum 150 66 Count 7 Count 6
UJ Table 12
Descriptive Statistics - Junior Automotive
Pre-ASE Junior Post-ASE Junior
Mean 140.50 Mean 157.08 Standard Error 2.13 Standard Error 2.11 Median 137.88 Median 156.56 Mode #N/A Mode #N/A Standard Deviation 5.63 Standard Deviation 5.17 Variance 31.75 Variance 26.70 Kurtosis -0.89 Kurtosis 1.60 Skewness 1.07 Skewness 0.87 Range 13 48 Range 15 32 Minimum 135.76 Minimum 150.61 Maximum 149.24 Maximum 165 93 Sum 983.52 Sum 94248 Count 7 Count 6 Table 13
Descriptive Statistics - Senior Automotive
Pre-ASE Senior Post-ASE Senior
Mean 169.86 Mean 194.60 Standard Error 3.17 Standard Error 5.46 Median 170.74 Median 192.14 Mode #N/A Mode #N/A Standard Deviation 8.40 Standard Deviation 13.38 Variance 70.53 Variance 178.96 Kurtosis -1.77 Kurtosis 1.86 Skewness -0.20 Skewness 1.00 Range 21.32 Range 39.74 Minimum 158.19 Minimum 178.19 Maximum 179.51 Maximum 217 93 Sum 1189.02 Sum 1167.58 Count 7 Count 6 76
was 13.48 points and for the post-ASE junior group was 15 32 points. Table 13,
which contains the senior level automotive results, shows a difference in means
between the two groups of 37.52 points. Ranges for the senior pre- and post-ASE groups were 21.32 points and 39.74 points respectively.
To view the change in mean scores for each of the test sections graphically, the data were plotted against time in a line graph format to show pre-ASE junior scores, post-ASE junior scores, pre-ASE senior scores, and post-ASE senior scores for each of the three test areas. These graphs are presented in Figures 1, 2, and 3.
Graphs for the language (Figure 1) and nonlanguage (Figure 2) mean scores appear to show neither a positive nor negative trend for juniors or seniors in the academic sections. A purpose of the study was to examine the academic aptitude for students taking the automotive test to determine if academic abilities had improved or decreased during the test period. Flad the academic abilities improved significantly, this could have partially explained any positive change in the automotive scores.
Based on the data shown in Figures 1 and 2, it would not appear that the academic aptitude of students in the automotive classes changed significantly during the 11 -year period 1980 to 1990. At the very least, there does not appear to be a positive shift in the mean scores from the pre-ASE groups to the post-ASE groups for the language and nonlanguage sections of the OVEAT.
It is, however, readily apparent from Figure 3 that a change in mean scores for the automotive section did occur at approximately the same time the program was
ASE certified. A slight improvement in mean scores can be detected 1 to 2 years Scores 20 21 19 22 23 24 25 . u . Q k_ ) 0 >% CD I . Q w s > m SD =SD0.71 Mean=44 22 m . Q a> L_ * > Mean=24.01 SD =SD0.32 . Q 2 S > Figure 1 Language Mean Scores Mean Language 1 Figure Years Before and After Certification After and Before Years CO a a> > > CM u . Q
! Scores 26.00 28.00 29.00 23.00 24.00 25.00 27.00 h- 2 l— L C >. Om CO LCL CL s L. >* I s >* 1 Figure 2 Nonlanguage Mean Scores Mean Nonlanguage 2 Figure SD = 1.04=SD SD = 1.19=SD Mean= 24.65 Mean=26.52 Q. £ >* Years Before and After Certification After and Before Years CO . o k- 0 s > i ) CM C g> -u. L- > { l T— ■
SD =SD0.67 Mean=25.11 \ u \ MCO CM iV Vi CL .u. >N w. o 1 - - -- - f -- r f Q. o >> " /) ( Q. u. o >* t in O (/) Q. l >s i . D S Mean =Mean24.10 Post ASE Junior ASE Post Junior ASE Pre otAE eir ! Senior ASE Post r S Senior ASE Pre
00 ~4 220
Mean = 194.60 SD= 13.38 Mean = 169.86 SD = 8.40 i> ■- - -C) Pre ASE Junior
Vi £ Post ASE Junior © 170 (J C /3 Pre ASE Senior
Post ASE Senior i
Mean = 157.08 SD = 5.17 « Mean = 140.50 SD = 5.63
120 i I *
CD 0 ) Q. Q.Q. Q. Q. £ £ £ £ £ £ >* i— > N > s > % > > N > > U i >* £ £ e £ CO > . > * (O m ■ o - CO c m > < e g CO LD Years Before and After Certification
Figure 3 Automotive Mean Scores
'vi <3 80
prior to actual award of certification. One purpose of this study was to show whether
there was a difference in the mean scores on the automotive section of the OVEAT
after schools obtained ASE certification for their automotive program.
Finally, a t-test was used to determine the statistical significance between pre-
and post-ASE group performance.
Hypotheses Testing
A purposive sample of 26 schools was drawn from the population of schools that had automotive repair programs that obtained ASE certification in 1985-87 and that participated in the OVEAT program from 1980 to 1990. Group means were computed for the pre-ASE and post-ASE groups. 81 Results of Hypothesis 1
Research Question 1: Do junior students who complete an ASE-certified
program score significantly higher on the language
section of the Short Form Test of Academic
Achievement (SFTAA) than students who complete a
noncertified program?
Null Hypothesis 1. There will be no significant improvement in junior level
student mean scores on the language section of the
SFTAA after programs receive ASE certification.
Ho' Ml = JL12
H l ; * H2
Table 14 shows that the junior level pre-ASE group mean was 22.44 with a standard deviation of 0.71. Students in the junior level post-ASE group had a mean of 21.45 with a standard deviation of 1.40. Based on the sample, the calculated value of t was 1.64. At 11 degrees of freedom, the critical value of t is equal to 2.20.
Therefore, the null hypothesis was accepted at a = .05. There is no statistically significant difference in the mean scores for the junior level pre-and post-ASE groups on the language section of the SFTAA. This means the academic aptitude of students changed very little during the period of study and that the change was in a negative direction. A further implication is that academic aptitude cannot be used to explain changes in automotive scores mean scores. 82
Table 14
Language Test Results for Junior Level
Group N Mean Standard * calc Deviation
Pre-ASE 7 22.44 0.71 1.64
Post-ASE 6 21.45 1.40 tcalc= 164 alpha =05 tcrit = 2.20 df = 11 83 Results of Hypothesis 2
Research Question 2: Do senior students who complete an ASE-certified
program score significantly higher on the language
section of the SFTAA than students who complete a
noncertified program?
Null Hypothesis 2: There will be no significant improvement in senior level
student mean scores on the language section of the
SFTAA after programs receive ASE certification.
H0: m = P2
Hl: Ml * M2
Table 15 shows that the senior level pre-ASE group mean was 24.01 with a
standard deviation of 0.32. Students in the senior level post-ASE group had a mean
of 22.97 with a standard deviation of 1.55. Based on the sample, the calculated value
of t was 1.75. At 11 degrees of freedom, the critical value of t is equal to 2.20.
Therefore, the null hypothesis was accepted at a = .05. There is no statistically significant difference in the mean scores for the senior level pre-and post-ASE groups on the language section of the SFTAA. This means the academic aptitude of students changed very little during the period of study and that the change was in a negative direction. A further implication is that academic aptitude cannot be used to explain changes in automotive scores mean scores. 84
Table 15
Language Test Results for Senior Level
Group N Mean Standard * calc Deviation
Pre-ASE 7 24.01 0.32 1.75
Post-ASE 6 22.97 1.55
tcalc= 1 75 alpha =.05
tcrjt = 2.20 df = 11 85 Results of Hypothesis 3
Research Question 3: Do junior students who complete an ASE-certified
program score significantly higher on the nonlanguage
section of the SFTAA than students who complete a
noncertified program?
Null Hypothesis 3: There will be no significant improvement in senior level
student mean scores on the nonlanguage section of the
SFTAA after programs receive ASE certification.
H o: Ml = M2
H l : Ml * M2
Table 16 shows that the junior level pre-ASE group mean was 24.65 with a standard deviation of 1.19. Students in the junior level post-ASE group had a mean of 24.10 with a standard deviation of 0.61. Based on the sample, the calculated value of t was 1.01. At 11 degrees of freedom, the critical value of t is equal to 2.20.
Therefore, the null hypothesis was accepted at a = .05. There is no statistically significant difference in the mean scores for the junior level pre-and post-ASE groups on the nonlanguage section of the SFTAA. This means the academic aptitude of students changed very little during the period of study and that the change was in a negative direction. A further implication is that academic aptitude cannot be used to explain changes in automotive scores mean scores. 86
Table 16
Nonlanguage Test Results for Junior Level
Group N Mean Standard * calc Deviation
Pre-ASE 7 24.65 1.19 1.01
Post-ASE 6 2.4.01 0.61
tcalc= 101 alpha =.05
tcrit = 2.20 df = 11 87 Results of Hypothesis 4
Research Question 4: Do senior students who complete an ASE-certified
program score significantly higher on the nonlanguage
section of the SFTAA than students who complete a
noncertified program?
Null Hypothesis 4: There will be no significant improvement in senior level
student mean scores on the nonlanguage section of the
SFTAA after programs receive ASE certification.
Ho: Hi =H2
H l : Hi * H2
Table 17 shows that the senior level pre-ASE group mean was 26.52 with a standard deviation of 1.04. Students in the senior level post-ASE group had a mean of 25.11 with a standard deviation of 0.67. Based on the sample, the calculated value of t was 1.01. At 11 degrees of freedom, the critical value of t is equal to 2.20.
Therefore, the null hypothesis was accepted at a = .05. There is no statistically significant difference in the mean scores for the senior level pre-and post-ASE groups on the nonlanguage section of the SFTAA. This means the academic aptitude of students changed very little during the period of study and that the change was in a negative direction. A further implication is that academic aptitude cannot be used to explain changes in automotive scores mean scores. 88
Table 17
Nonlanguage Test Results for Senior Level
Group N Mean Standard t ca|c Deviation
Pre-ASE 7 26.52 1.04 1.01
Post-ASE 6 25.11 0.67 tcalc=^°^ alpha =.05 tcrit = 2.20 df = 11 89 Results of Hypothesis 5
Research Question 5 . Do junior students who complete an ASE-certified
program score significantly higher on the automotive
section of the Ohio Vocational Education Achievement
Test (OVEAT) than students who complete a non
certified program?
Null Hypothesis 5: There will not be a significant improvement in junior
level student mean scores on the automotive section of
the OVEAT after programs receive ASE certification.
Ho: Ml = M2
H l '- Ml * M2
Table 18 shows that the junior level pre-ASE group mean was 140.50 with a standard deviation of 5.63. Students in the junior level post-ASE group had a mean of 157.08 with a standard deviation of 5.17. Based on the sample, the calculated value of t was 5.49. At 11 degrees of freedom, the critical value of t is equal to 2.20.
Therefore, the null hypothesis was rejected at a = .05. There is a statistically significant difference in the automotive mean scores for the junior level pre-and post-
ASE group. This means that students who were enrolled in automotive programs after they became ASE certified performed significantly better than their counterparts who were enrolled in programs before the ASE certification. 90
Table 18
Automotive Test Results for Junior Level
Group N Mean Standard * calc Deviation
Pre-ASE 7 140.50 5.63 5.49*
Post-ASE 6 157.08 5.17 lcalc = ^-49 alpha = .05 lcrit = 2-20 df =11
* p <.05
I 91 Results of Hypothesis 6
Research Question 6: Do senior students who complete an ASE-certified
program score significantly higher on the automotive
section of the OVEAT than students who complete a
noncertified program?
Null Hypothesis 6: There will not be a significant improvement in senior
level student mean scores on the automotive section of
the OVEAT after programs receive ASE certification.
Ho: Ml = M2
H l : Ml * M2
Table 19 shows that the senior level pre-ASE group mean was 169.86 with a standard deviation of 8.40. Students in the senior level post-ASE group had a mean o f 194.60 with a standard deviation of 13.38. Based on the sample, the calculated value of t was 4.06. At 11 degrees of freedom, the critical value of / is equal to 2.20.
Therefore, the null hypothesis was rejected at a = .05. There is a statistically significant difference in the automotive mean scores for the senior level pre-and post-
ASE groups. This means that students who were enrolled in automotive programs after they became ASE certified performed significantly better than their counterparts who were enrolled in programs before the ASE certification. 92
Table 19
Automotive Test Results for Senior Level
Group N Mean Standard * calc Deviation
Pre-ASE 7 169.86 8.40 4.06*
Post-ASE 6 194.60 13.38 tcalc = 4.06 alpha = .05 tcrit = 2-20 d f = 11 * p < .05 93
Table 20 is a summary of achievement test scores for students completing ASE- certified programs. It summarizes all six test results for both pre- and post-ASE certified programs.
Table 20
Achievement Test Scores of Students Completing ASE-Certified Programs
ASE-Certified programs
Test Juniors Senior
SFTAA Language Section N N SFTAA Nonlanguage Section N N OVEAT Automotive Achievement Section HH
Note: N = no statistically significant difference
H = statistically significant difference - Higher (.05 level) 94 Summary
This study investigated the relationship of ASE certification and student scores on the automotive section of the OVEAT. The study analyzed data collected from 26 vocational programs in Ohio before and after gaining ASE certification.
Data were also collected on the academic aptitudes of the same students as reported on the SFTAA section of the OVEAT. These data were used to gain an understanding of whether or not the academic abilities fiftthe students changed over the 11-year period from 1980 to 1990.
This study revealed that when junior and senior level automotive students were divided into pre-ASE and post-ASE groups and the mean scores compared for each group, there were no significant differences for language and nonlanguage variables. The implication of these results is that the academic abilities of the automotive students during the period 1980 to 1990 did not change significantly. The changes that did occur tended to be downward.
A similar analysis was conducted for the automotive section of the OVEAT.
A comparison of the mean scores for both junior and senior students before and after
ASE certification revealed that a significant difference existed between the pre-ASE and post-ASE scores. The differences were in an upward direction. Although a causal relationship cannot be inferred from these results because of the quasi- experimental nature of the study, ASE certification has to be considered as one of the contributing factors to the increase in mean scores on the automotive section of the
OVEAT. CHAPTER V
SUMMARY, CONCLUSIONS, AND RECOMMENDATIONS
Introduction
If America is to succeed as a leading industrial power in the 21st century, it must revitalize its educational system by adopting rigorous standards. This study examines ASE certification as an example of industry-based standards currently used in this country for vocational training programs and the impact of ASE certification on the technical test scores of the students in those programs. It is hoped that this study will help to gain wider acceptance of voluntary industry-based standards across ail occupational areas.
The results presented in this chapter document a positive relationship between
ASE certification and higher scores on the automotive section of the OVEAT.
Recommendations for practice and research found in this chapter have the potential for improving the quality of vocational programs and graduates, linkages with industry, and the image of vocational programs in the community.
95 96 Summary
Background
American high school graduates are not competing well with students from
other industrialized nations. Recent studies have shown that Scholastic Aptitude Test
scores have declined steadily since the mid-1980s and that U.S. students rank in the
bottom half of international tests for math and science (Marshall & Schram, 1993). In
addition, they do not fare much better when competing in the International Youth
Skill Olympics (IYSO). At the most recent IYSO held in Amsterdam, the
Netherlands, the U.S. team entered 26 different contests but won only one gold medal
and one silver medal. Most of the medals in the IYSO competitions were won by the
Germans, Japanese, Koreans, or Taiwanese. Both Germany and Japan use rigorous
standards for their vocational programs whereas the U.S. does not.
Current efforts are underway in this country to define industry-based
standards for a dozen different occupational areas. Although these efforts are an
important part of the reform movement, they do not address the question of whether
industry-based standards make a difference in the quality of a training program. Little
or no research data exist to show the effectiveness of using industry-based standards
for vocational training programs.
Objectives
The purposes of this study were as follows: (1) to determine if there was a relationship between ASE certification of automotive programs and student performance on automotive sections of the OVEAT, and (2) to determine whether during the same time period, the academic aptitude of students taking the SFTAA 97
changed significantly. The researcher used an ex post facto study and therefore had
to limit the extent to which generalizations could be made about the results.
Research Questions
The study was guided by the following research questions:
1. Do junior students who complete an ASE-certified program score significantly higher on the language section of the Short Form Test of Academic Aptitude (SFTAA) than students who complete a noncertified program?
2. Do senior students who complete an ASE-certified program score significantly higher on the language section of the SFTAA than students who complete a noncertified program?
3. Do junior students who complete an ASE-certified program score significantly higher on the nonlanguage section of the SFTAA than students who complete a noncertified program?
4. Do senior students who complete an ASE-certified program score significantly higher on the nonlanguage section of the SFTAA than students who complete a noncertified program?
5. Do junior students who complete an ASE-certified program score significantly higher on the automotive section of the Ohio Vocational Achievement Test (OVEAT) than students who complete a noncertified program?
6. Do senior students who complete an ASE-certified program score significantly higher on the automotive section of the OVEAT than students who complete a noncertified program?
Due to societal pressures on education during this period, the author suspected that the academic aptitude of the students declined. Increased emphasis 98 and public pressure on academic requirements for graduation during this period could have caused some of the more capable students who traditionally would have taken vocational education courses to opt for college preparatory courses instead As a result, programs may have been forced to accept less able students. Obviously, this would have an impact on students' scores on achievement tests and therefore had to be considered in this study.
The nonlanguage section on the SFTAA measures recognition, ability to draw pictorial analogies (opposites and similarities), and comprehension of numerical and geometric pattern sequence changes, and the language section measures verbal comprehension, word meanings, and retentive ability. The SFTAA is "designed to yield a useful measure of general ability by combining several intellectual tasks into a single score. Total score on the test is the most dependable estimate of general ability, with language and nonlanguage scores constituting two separate estimates of more specific abilities" (Sullivan, Clark, & Tiegs, 1970, p. 6). "[The SFTAA] measures students' ability to comprehend the relationships among ideas presented both verbally and non-verbally" (Ohio Vocational Education Achievement Test
Program, 1989, p.3). An increase in mean scores of the automotive test after ASE certification could be a result of improved student academic ability rather than ASE certification. However, if the scores in nonlanguage or language sections remained constant or declined and the automotive mean scores improved, then ASE certification could be at least one of the factors that positively affected the automotive scores. 99
Procedures
There were 26 vocational automotive programs selected for this study, all of
which received ASE certification in 1985, 1986, or 1987. The sample is not a random
sample. All of the programs had also participated in the OVEAT program. Data
used in the study represent a maximum time range for each school's scores of 7 years
prior to ASE certification and 5 years after ASE certification, depending upon when
the automotive program became ASE certified.
The 26 programs in the study were a mixture of urban and rural, joint vocational schools and city districts, and geographic regions from north, south, east, and west sections of Ohio. Two schools were from city school districts, and 24 were from joint vocational school districts. Urban areas included both city schools and joint vocational schools. Many rural areas were represented. All schools except two were certified in five or more ASE specialty areas. Thirteen schools were certified in all eight of the ASE specialty areas. Two schools were certified in only three of eight specialty areas. ASE areas of specialization include automatic transmission/transaxle, brakes, electrical systems, engine performance, engine repair, heating and air conditioning, manual drive train and axles, and suspension and steering.
Findings
The study revealed that over an 11 -year period from 1980 to 1990, students enrolled in 26 selected automotive programs performed better on the OVEAT after the program obtained ASE certification than before the program obtained ASE certification. The null hypotheses for the automotive scores were rejected. The language and nonlanguage scores for the same students were compared before and after ASE certification to determine if there were any major changes in the academic 100
aptitude of the students. The study revealed no significant changes in the academic
aptitudes of students during the period of study; though changes that did occur were negative for both the language and nonlanguage scores. The null hypotheses for both language and nonlanguage scores were accepted.
Conclusions « ,
The study was a'quasi-experimental design that consisted of a total population of 26 vocational automotive programs that received ASE certification in 1985 through 1987. The results, therefore, are limited to the 26 automotive programs included in the study. The following conclusions are based upon the findings of the study.
1. There was no significant difference between the junior pre-ASE group and post-ASE group language scores. The language score is a part of the SFTAA that requires memory and verbal reasoning and gives an indication of the academic aptitude of students. The pre-ASE group mean scores were actually slightly higher than the post-ASE group, which would imply that academic aptitude was not a contributing factor in the study.
2. There was no significant difference between the senior pre-ASE group and post-ASE group language scores. The language score is a part of the SFTAA that requires memory and verbal reasoning. The pre-ASE group mean scores were actually slightly higher than the post-ASE group, which would imply that academic aptitude was not a contributing factor in the study. Although neither the junior level nor senior level mean score differences for pre- and post-ASE groups were significantly different, it is worthwhile to note that the tendency for both groups was a decline in mean scores.
3. There was no significant difference between the junior pre-ASE group and post-ASE group non-language scores. The nonlanguage score is a part of the SFTAA that uses sequence and analogy information to 101 give an indication of the academic aptitude of students. The pre-ASE group mean scores were actually slightly higher than the post-ASE group, which would imply that academic aptitude was not a contributing factor in the study.
There was no significant difference between the senior pre-ASE group and post-ASE group nonlanguage scores. The non-language score is part of the SFTAA that uses sequence and analogy information. The pre-ASE group mean scores were actually slightly higher than the post-ASE groups which would imply that academic aptitude was not a contributing factor in the study. Although neither the junior level nor senior level mean score differences for pre- and post-ASE groups were significantly different, it is worthwhile to note that the tendency for both groups was a decline in mean scores in the nonlanguage test.
The language and nonlanguage test sections of the SFTAA are used as indicators of academic aptitude. Neither the language nor the nonlanguage mean scores for junior and senior pre- and post-ASE groups changed significantly during the period of study. Therefore, academic aptitude cannot account for the increase in the post-ASE automotive mean scores for junior and senior level students.
Post-ASE junior students performed significantly better as a group on the OVEAT automotive test than pre-ASE junior students. Automotive scores are an indication of the technical competence of students. Therefore, the results support the hypothesis that there is a relationship between the junior level mean scores on this test and ASE certification.
Post-ASE senior students performed significantly better as a group on the OVEAT automotive test than pre-ASE senior students. Auto motive scores are an indication of the technical competence of students. Therefore, the results support the hypothesis that there is a relationship between the senior level mean scores on this test and ASE certification.
There was less variability, as indicated by the standard deviation of 5.17, in the junior level post-ASE automotive scores than in senior level scores, at 13.38 S.D. (See Figure 3.) That is, the mean scores for the junior groups were more homogeneous than the senior 102 groups in their technical ability as measured on the automotive section of the OVEAT. This may indicate that in the junior year, when basic concepts are being presented, students are more likely to be similar in their performance on achievement tests. By the senior year, however, where more highly technical concepts are presented, differences in skills are clearly evident, as measured on the automotive section of the OVEAT. It was noted that the increased variability was also present in the pre-ASE senior scores, as measured by the standard deviation of 8.40, compared to the junior pre-ASE standard deviation of 5.63.
9. Automotive scores began improving in both the junior and senior groups approximately 1 to 2 years prior to actual ASE certification. This early sign of improvement was most likely the result of changes being made in the facilities, equipment, and curricula, in preparation for the ASE on-site review.
10. Automotive scores continued to improve after ASE certification was obtained. Factors related to program certification that could encourage such improvements would be as follows: (a), equipment and instructional material grants from industry that follow program certification; (b). improved links to business and industry; (c). pride of accomplishment; and (d). the requirement that instructors must achieve ASE technician certification in the areas they are teaching.
Recommendations for Practice and Further Research
Practice
This study found that among the 26 automotive programs in the sample, there
was a positive relationship between ASE certification and scores on the automotive
section of the Ohio Vocational Education Achievement Test. The implication for
practice is that industry-based standards should be aggressively pursued bv state and local vocational educators and implemented in all vocational programs. Standards for some program areas already exist, such as ASE and PrintEd, and can be implemented quickly. Although nationally ASE is a voluntary program, Ohio is 1 of 11 states that 103
has mandated ASE standards as a condition for programs receiving federal vocational job training funds. All states should move to enact similar requirements.
Results of the study showed that academic aptitude of automotive students
did not change significantly during the 11-year study period. During the same period,
the level of skill required in the field of automotive repair increased dramatically.
According to automobile manufacturers, cars are becoming more complicated and
require a "higher caliber technician than ever before" (Quinn, 1993). Our students must be equal to the task. The implication for practice is that students must be selected for vocational programs based on their aptitudes, interests, and potential for success, not on other criteria such as income level, disadvantaged status, or other conditions that have no relation to the trade. An entrance-level standard should be developed for students' admission into high-tech vocational programs.
Although it was not one of the variables studied, the researcher noted that most of the programs that met the criteria for selection were located in joint vocational schools. In reflecting on the reasons for this self-selection and using personal experience as a basis, the researcher concluded that joint vocational schools generally have supervisory staff who are experienced in trades and are linked to business and industry through former job experience. Furthermore, most vocational areas (e.g., trade and industrial) have support staff who are able to help with the detailed paperwork part of the ASE application procedure. Therefore, a final recommendation for practice is to ensure there is a commitment from the superintendent, administrators, guidance counselors, clerical staff, and instructors to complete the ASE certification program. 104
Research
Results of the study have suggested ten items for future research that would
help substantiate or refute the findings of this study First, does the number of ASE
specialty areas held bv programs have any effect on the OVEAT scores? Second,
what effect, if any, does ASE certification have on graduates' earnings9 Third, does
ASE certification have any effect on student enrollment? Fourth, can similar results
be shown for programs that are ASE-certified in auto collision and truck repair?
Fifth, since most of the programs in this study were located in joint vocational
schools, what are the critical elements in a school environment that foster the pursuit
of voluntary industry-based standards? Sixth, do graduates from ASE-certified
programs tend to gain ASE Technician certification at higher percentages than
graduates from non-ASE certified programs? Seventh, can the results of this
experiment be corroborated by comparing test results from a control group (i.e..
students not enrolled in an ASE-certified program) to an experimental group (i.e.. from an ASE-certified program)? Eighth, do graduates from ASE-certified programs enjoy higher wages than their counterparts from non-ASE certified programs? Tenth, are graduates from ASE-certified programs more likely to gain promotions than their counterparts from non-ASE certified programs? Finally, how do the student scores on the automotive section of the OVEAT which are obtained after ASE recertification compare to those obtained during the initial ASE certification? BIBLIOGRAPHY
Anderson, L. (Ed). (1993, April). Work force solutions forum. Indianapolis, IN: CDS International.
Batt, R., & Osterman, P. (1993). A national policy for workplace training: Lessons from state and local experiments. Washington, DC: Economic Policy Institute.
Campbell, D.T., & Stanley, J.C. (1963). Experimental and quasi-experimental designs for research. Boston: Houghton Mifflin Co.
Center for Remediation Design. (1991, July). Developing industry-based skill standards. Unpublished report for the National Governors' Association.
Commission on Work, Family, & Citizenship. (1988). The forgotten half. Washington, DC. William T. Grant Foundation.
Commission on the Skills of the American Work Force. (1990) America's choice: High skills or low wages. Washington, DC. The National Center on Education and the Economy.
Council of Chief State School Officers. (1991). Connecting School and Employment. Washington, DC . Council of Chief State School Officers.
Federal Register. (1992, March 18). 57 (53), p. 9488.
Frankel, J R., & Wallen, N.E. (1990). How to design and evaluate research in education. New York: McGraw-Hill.
105 106
Hall, L. (Ed.) (1992, November). Framework update. Wellington. New Zealand Qualifications Authority.
Hatton, D. (1993, April). The need for national skill standards. In S erviceprofit profiles. Washington, DC. Electronics Industry Association.
Hildebrand, J. (1992, Fall). Toward national standards, local vs. national. In America's agenda: Schools for the tw enty-first century. New York: Scholastic, Inc.
Kearns, D.T., & Doyle, D P. (1991). Winning the brain race. San Francisco: ICS Press.
Kerlinger, F.N. (1979). Behavioral research: A conceptual approach. New York: Holt, Rinehart, and Winston.
Kolberg, W., & Smith, F. (1992, February 23). Our unsung Olympic hero. Washington Post, p. 6.
Kramer, R. (1990). Ed school follies. Washington, DC: Free Press.
Leo, J. (1993, June 21). Lowering the education bar. U.S. new s and world report, p. 19.
Magnet, M. (1993). The dream and the nightmare: The sixties'legacy to the underclass. New York: William Morrow.
Marshall, R., & Tucker, M. (1992). Thinking for a living: Education and the wealth o f nations. New York: Basic Books.
Marshall, W., & Schram, M. (1993).Mandate for change. New York. Berkeley Books.
McCracken, J. D. (1990, Autumn). Agriculture Education 886. Columbus: The Ohio State University, p. 36. 107
National Advisory Commission on Work-Based Learning. (1992, September). Policy research and technical assistance plan for skill standards.
National Alliance of Business. (1992, June). Rea! jobs for real people. Washington, DC: National Alliance of Business.
National Association of Trade and Industrial Education. (1992, March). Workforce 2020: Conference report. Leesburg, VA: National Association of Trade and Industrial Education
National Automotive Technicians Education Foundation. (Undated). ASE certification for automotive training programs. Herndon, VA: National Automotive Technicians Education Foundation.
National Coalition for Advanced Manufacturing . (1993). Preparing technical workers for the new industrial era. Washington, DC: National Coalition for Advanced Manufacturing.
National Commission on Excellence in Education. (1983). A nation at risk: The imperative for educational reform; A report to the nation and the Secretary of Education. Washington, DC: National Commission on Excellence in Education.
National Education Goals Panel. (1992, July 31). The goal 5 technical planning subgroup on international work force skills: Report to the National Education Goals Panel. Report 92-08. Washington, DC. National Education Goals Panel.
National Institute for Automotive Service Excellence. (Undated). Program description: Automotive and Auto Body Technician Training Program Certification. Herndon, VA: National Institute for Automotive Service Excellence.
Office of Technology Assessment. (1989, April). Performance standards for secondary education: Background paper. Washington, DC: Science, Education, and Transportation Program, Office of Technology Assessment, Congress of the United States. 108
Office of Work-Based Learning. (1992, September). Public dialogue on voluntary industry-based skill standards and certification: A final report presented to the U. S. Department o f Labor. Washington, DC Office of Work-Based Learning
Ohio Vocational Education Achievement Test Program. (1989). Columbus, OH: Ohio Division of Vocational and Career Education.
P Y 1992 strategic plan for developing voluntary industry-based skill standards and certification. (1992, November). Washington, DC: U.S. Departments of Labor and Education.
Quinn, C. (1993, April 7). [Unpublished testimony given at the annual Ohio Council on Vocational Education meeting in Columbus, Ohio],
Ravitch, D. (1992, Fall). A vision of excellence. In America's agenda: Schools for the twenty-first century. New York: Scholastic, Inc.
Secretary's Commission on Achieving Necessary Skills. (1991, July). What work requires o f schools. Washington, DC: Secretary's Commission on Achieving Necessary Skills.
Shoemaker, B. R. (1992). [Unpublished interview].
Skill standards and certification issues paper. (1992, April). Washington, DC: U.S. Departments of Education and Labor.
Sullivan, E.T., Clark, W.W., & Tiegs, E.W. (1970). Short form test o f academic aptitude: Examiner's manual, level 1. Monterey, CA: CTB/Macmillan-McGraw Hill.
Sullivan, E.T., Clark, W.W., & Tiegs, E.W. (1971). Short form test o f academic aptitude: Technical bulletin no. Monterey, 1. CA. CTB/Macmillan-McGraw Hill. 109
Texas Department of Commerce. (1993, February 10). Developing a system of skill standards and certification for the Texas work force. Austin. Texas Department of Commerce.
Thurow, L. (1992). Head to head: The coming economic battle among Japan, Europe, and America. New York: William Morrow.
Toth, T. (1993, January 11). The obstacles to innovation. U.S. news and world report, p. 61.
U.S. Congress. (1990, October). Carl D. Perkins Vocational and Applied Technology Education Act. Public Law 101-392. Washington, DC. U.S. Government Printing Office.
U.S. Department of Education. (1990, July). National Education Goals. Washington, DC: U.S. Department of Education.
U.S. Department of Education. (1991). America 2000: An Education Strategy. Washington, DC: U.S. Department of Education.
U.S. General Accounting Office. (1990, May). Training strategies: Preparing non college youth for employment in the U.S. and foreign countries. GAO/HRD 90-88. Washington, DC: U.S. General Accounting Office.
U.S. General Accounting Office. (1991, August). Transition from school to work: Linking education and worksite training. GAO/HRD 91-105. Washington, DC: U.S. General Accounting Office.
U.S. General Accounting Office. (1993, May). Occupational skill standards: Experience shows industry involvement to be key. GAO T-HRD 93-23. Washington, DC. U.S. General Accounting Office.
Utley, G. (1993). NBC Nightly News, [unpublished comments from a TV special on American education] 110
Van Erden, J.D. (1992, October 26). Education, Labor Departments announce grants for skill standards. Washington, DC: U.S. Departments of Labor and Education.
Voluntary industry-based skill standards and certification.P Y 1992 strategic plan. (1992, July 6). Washington, DC: U.S. Departments ofEducation and Labor.
Voluntary industry-based skill standards and certification: A final report to the U.S. Department of Labor, Employment and Training Administration, Office o f Work- based Learning. (1992, September). Washington, DC: U.S. Government Printing Office.
Washington, B.T. (1901). Up from slavery. New York: A.L. Burt.
Whitten, E. (1992, December). NTMA awarded grant to set national metalworking skill standards. Crossfeed. Ft. Washington, MD: National Tooling and Machining Association.
Williams, F. (1986). Reasoning with statistics. New York: Holt, Rinehart, and Winston.
Work force education reform. (1992, October). Vocational education journal, 67 (7), 30-31.