DOCUMENT RESUME Assessing Science Pathways

DOCUMENT RESUME

ED 426 843 SE 060 624

TITLE Assessing Science Pathways: Tracking Science Education and Careers from Precollege through Professional Levels. Undergraduate Program Directors Meeting (Chevy Chase, MD, September 30-October 2, 1996). INSTITUTION Howard Hughes Medical Inst., Chevy Chase, MD. Office of Grants and Special Programs. PUB DATE 1997-00-00 NOTE 93p. AVAILABLE FROM Howard Hughes Medical Institute, Office of Grants and Special Programs, 4000 Jones Bridge Road, Chevy Chase, MD 20815-6789. PUB TYPE Collected Works Proceedings (021) EDRS PRICE MF01/PC04 Plus Postage. DESCRIPTORS Biology; *Biomedicine; Career Choice; Elementary Secondary Education; Higher Education; Minority Groups; Program Evaluation; Science Careers; Science Curriculum; *Science Education; *Scientific Research; *Student Research; *Undergraduate Study

ABSTRACT This document presents the results of a meeting of undergraduate biology departments that receive funding from theHoward Hughes Medical Institute. The focus of this meeting was the assessmentof science pathways. Discussion focused on assessment, predictors of student successin science, the importance of undergraduate research, careerchoices, women and underrepresented minorities, encouraging interdisciplinarycollaborations, and teacher education and professional development programsfor preservice, inservice, and precollege teachers. Sections include:(1) program descriptions of the biomedical research program and the grants andspecial programs at the Howard Hughes MedicalInst:itute; (2) a preface and introduction;(3) program overview;(4) the keynote address;(5) summaries of panel and workshop discussions;(6) appendices with the agenda for the program directors meeting; and (7) a list ofattendees and awards. (DDR)

******************************************************************************** Reproductions supplied by EDRS are the best that can be made from the original document. ******************************************************************************** I 4 1

U S DEPARTMENT OF EDUCATION Office of Educational Research and Irnpmvement PERMISSION TO REPRODUCE AND DUCATIONAL RESOURCES INFORMATION DISSEMINATE THIS MATERIAL HAS CENTER (ERIC) BEEN GRANTED BY This document has been reproduced as ewed from the person or organization originating it 0 Minor changes have been made to i improve reproduction quality Points of view or opinions stated in this TO THE EDUCATtONAL RESOURCES document do not necessarily represent INFORMATION CENTER (ERIC) official OERI position or policy 1

. II . II . . o 1 . I

Ie IIel

. . I. .1. .1 a ' ill I 1: a 0 a 1 11 1.

I 11

2 Howard Hughes Medical Institute

Assessing Science Pathways Tracking Science Education and Careers From Precollege Through Professional Levels

Undergraduate Program Directors Meeting

September 30October 2, 1996

Undergraduate Biological Sciences Education Program Office of Grants and Special Programs 3 About This Report

In the fall of each year, the Howard Hughes Medical Institute convenes directors of the programs supported by grants from the Institute's Undergraduate Biological Sciences Education Program for a review of the year's activities. These meetings provide a forum for the program directors, program participants, and other members of the science education community to discuss educational challenges facing undergraduate science departments, and to learn how the community at large is addressing those challenges. The Institute sought to broaden the outlook at the 1996 meeting by including students and teachers who have participated in Institute-supported programs and promi- nent funders of science education. Each meeting is organized around a central theme, with plenary panel discussions and workshop demonstrations and discussions devoted to topics within that theme. The focus of the 1996 meeting was Assessing Science Pathways. The program directors examined issues related to assessing the impact of educational and professional experiences from the precollege years to the baccalaureate and beyond. Past meeting themes have included New Tools for Science Education ( 1995); Science Education: Expanding the Role of Science Departments ( 1994); Institutional Strategies for Enhancing Undergraduate Science Education ( 1993); Enriching the Undergraduate Laboratory Experience ( 1992); and Attracting Students to Science: Undergraduate and Precollege Programs (1991). This report contains the proceedings of the 1996 meeting.

The contributions of Judith Dickson, Eleanor Mayfield, Jeff Porro, and Robert Taylor to this reportare gratefully acknowledged. The photographs were taken by Barbara Ries.

The names of colleges and universities are listed as they appear in the 1997 Higher Education Directory.

Howard Hughes Medical Institute Progams Biomedical Research Program Grants and Special Programs Preface and Introduction vii Preface ix Introduction: Making Enlightened Choices for Science Education

Program Overview

1 Overview of the Undergraduate Biological Sciences Education Program The 1996 Program Directors Meeting Keynote Address

7 Help for Scientists and Non-Scientists from the Howard Hughes Medical Institute Daniel Koshland Panel and Workshop Discussions

17 Assessment

19 Predictors of Success in Science

22 Importance of Undergraduate Research

24 Career Choices for Science Graduates

26 Focusing on Women and Minorities Underrepresented in Science

29 Changing How Undergraduate Science Is Taught

33 Encouraging Interdisciplinary Collaborations

35 Research and Teaching: Finding a Balance

39 Outreach to Precollege Teachers

39 Programs for Inservice Teachers

41 Programs for Preservice Teachers Appendices

49 A: Agenda, Program Directors Meeting, September 30October 2, 1996

51 B: Attendees, Program Directors Meeting, September 30October 2, 1996

58 C: Undergraduate Biological Sciences Education Program-1988 Awards

59 D: Undergraduate Biological Sciences Education Program-1989 Awards

61 E: Undergraduate Biological Sciences Education Program-1991 Awards

62 F: Undergraduate Biological Sciences Education Program-1992 Awards

63 G: Undergraduate Biological Sciences Education Program-1993 Awards

Contents iii 64 H: Undergraduate Biological Sciences Education Program-1994 Awards

66 I: Undergraduate Biological Sciences Education Program-1996 Awards

67 J: Awardee Institutions by Carnegie Classification, 1992-1996

70 K: Awardee Minority Institutions, 1993-1996

71 L: Awardee Institutions by State, 1992-1996 Grants Publications 74 Grants Publications Howard Hughes Medical Institute 77 Trustees

78 Officers

79 Principal Staff Members

80 Office of Grants and Special Programs

iv Assessing Science Pathways 5A Howard Hughes Medical Institute Programs

The Howard Hughes Medical Institute was founded in 1953 by aviator- industrialist Howard R. Hughes. Its charter, in part, reads:

The primary purpose and objective of the Howard Hughes Medical Institute shall be the promotion of human knowledge within the field of the basic sciences (principally the field of medical research and medical education) and the effective application thereof for the benefit of mankind.

Biomedical Research Grants and Special Program Programs The Howard Hughes MedicalTo complement its research activi- Institute is a nonprofit medicalties, the Institute has a grants pro- research organization dedicated togram dedicated to strengthening basic biomedical research and edu-education in the biological and re- cation. Its principal objectives arelated sciences. Administered by the the advancement of fundamentalOffice of Grants and Special Pro- grams, the Institute grants are knowledge in biomedical science designed to enhance science educa- and the application of new scientific fion at the graduate, undergraduate, knowledge to the alleviation of dis-and precollege levels; to increase ease and the promotion of health. public understanding and apprecia- Through its program of directtion of science; and to support conduct of medical research in con-fundamental biomedical research junction with hospitals, it employsabroad and research resources in over 270 independent investigatorsU.S. medical schools. In addition, a at its laboratories in more than 60comprehensive assessment effort is leading academic medical centers,under way. The grants reach a wide universities, and hospitals through-range of institutions involved in for- out the United States. The Institutemal and informal science education, conducts research in five broadincluding colleges and universities, areas: cell biology, genetics, im-medical schools, research institutes, munology, neuroscience, and struc-elementary and secondary schools, tural biology. and museums. To aid these research efforts, the Since 1988 the Institute's grants Institute is involved in the training ofprogram has provided about $95 million in fellowship support to graduate and postgraduate students 1,400 students and physician scien- in its investigators' laboratories, hastists who have shown strong given substantial support to thepromise of becoming tomorrow's international genome mapping pro-leading biomedical researchers. gram, provides research training to The undergraduate program has medicalstudents throughtheawarded $335 million to strengthen Research Scholars Program (con-life sciences education at 220 public ducted jointly with the Nationaland private colleges and universi- Institutes of Health), and organizesties. These awards are intended to scientific conferences, workshops,enrich educational opportunities for and program reviews. science majors and enhance the

6 Howard Hughes Medical Institute Programs general scientific literacy of studentsawarded to 30 U.S. medical schools. who major in nonscience subjects. Annual payments of $550,000$1 mil- In addition to precollege activitieslion will be made over four years for in the undergraduate program, thejunior faculty start-up, core facilities, Institute has awarded $11 million topilot projects, emergency funds, and 51 museums, aquaria, botanical gar-other activities that will help the dens, and zoos to support innovative education programs and to interestschools sustain their commitment to youngsters in science. In 1994 theresearch. The research resources precollege program was extendedprogram also provides support to by awards totaling $10 million to 42research organizations serving the biomedical research institutions. biomedical community as unique The Institute's local science educa-resource laboratories and teaching tion initiatives provide opportunitiesfacilities. Through a grants initiative in the Washington, D.C., area for pre-launched in 1991, the Institute sup- college students at all levels to gainports the research of outstanding experience in the science classroombiomedical scientists abroad. Al- and laboratory. A holiday lecturetogether, more than $38 million in series on science for high school stu- five-year grants has been awarded to dents, held each December, is tele- cast via satellite throughout North143 international research scholars. America to more than 8,000 junior The Institute has a home page on and senior high schools. the World Wide Web, with direct A research resources competitionlinks to the grant sites. The univer- for U.S. medical schools was held insal resource locator (URL)is 1995. A total of $80 million was.

vi Assessing Science Pathways ks, Preface

Purnell W Choppin, M.D. El President

Jndergraduate enrollments infreshman and sophomore courses biology are rising sharply at collegesinbiology, chemistry, physics, and universities nationwideandand mathematics. Increasingly, col- the increase includes those wholege faculty are taking advantage of have been traditionally representedtechnological advances, especially in the sciences. This is an over-in the revolution in communication looked success story for a nation technologies. facing difficult choices about health care, the environment, and other As significant as these changes issues involving biology. are in the way college-level science We are learning that it is possibleis taught, they cannot be sustained if to improve science education forfaculty are not recognized for the large numbers of students. Under-value of their science teaching, graduate biology education has blos-whether for the amount of time they somed inpart because of thededicate to this noble goal or for the discipline's own vitality, but anothernovelty of their methods or curricu- reason is the important changes thatlum. We are learning that all of edu- have occurred in the classroom.cation is better served if students do Students are getting involved inresearch and researchers teach. In original research in addition toaddition, as we integrate recognition learning from lectures, textbooks,of teaching and reward for innova- and the scientific literature. Theytion into the culture of higher educa- are using powerful computers andtion, new tools will be required to state-of-the-art laboratory equip-determine the value of our efforts. ment and putting their new knowl- Through the Howard Hughes edge and skills to work in scientificMedical Institute's undergraduate settings and across society. program, we have seen that the This trend in higher education ispractice of science can go hand in unfolding at all types of campusesglove with the teaching of science. from big research universities toInstitutions have developed impres- small women's colleges and histori-sive programs to involve research- cally black institutions. The revolu-active faculty in science education, tion in undergraduate education isfrom the introductory through expanding the opportunities for stu-upper-division levels. HHMI grants dents, whether they eventually pur-have also supported programs that sue science as a career or just betterprovide opportunities for students to understand how science is changingbecome engaged in research in fac- their lives. ulty laboratories, to learn science by Specifically, colleges and universi-doing it, and interact with faculty sci- ties are changing the goals of teach-entists as colleagues and mentors. ing to include nurturing the interestThrough our assessment initiatives, of all students, especially womenwe have seen an increasing number and underrepresented minorities.of these students competing suc- Departments are changing how sci-cessfully in HHMI's graduate fellow- ence is taughttaking an interdisci-ship programs and in programs plinary approach and restructuringsupported by other organizations.

Preface vii 8 the biology department at the 4 University of Michigan, Ann Arbor, at the 1996 program directors meeting and in a recent Institute publication, Beyond Bio 101. Dr. Maddock eloquently described her efforts to develop her research program in cell biology while providing the undergraduates in her classes with excellent training in biology. Her efforts exemplify those of increasing numbers of science faculty members, who find they can be true to their science while remaining true to their students. HHMI is a proud partner in the r metamorphosis of biology education on our nation's campuses. More than 200 public and private colleges and universities have received several hundred million dollars in grants to expand research opportunities for students, update courses and facilities, attract talented women and underrepresented minorities to sci- Institute President The challenge remains for institu-ence, and reach out to science teach- Purnell W. Choppin tions to find the appropriate balanceers and schools in their communities. introduced Daniel between teaching and research, andThrough their actions these institu- Koshland, keynote speaker at the 1996 discern opportunities for facultytions are changing attitudes, percep- undergraduate members to merge their researchtions, and goals and, as a result, have program directors and teaching activities. This chal-begun to change how millions of meeting. lenge was underscored by Janineyoung people learn about a science Maddock, an assistant professor inthat is transforming their world.

viii Assessing Science Pathways 9 Introduction: Making Enlightened Choices for Science Education Joseph G. Perpich, M.D., J.D. I=1 Vice President for Grants and Special Programs

December 1996, Science maga-essential and sometimes frustrating zine cited advances in researchwork done by scientists all over the regarding the human immunodefi-world that preceded and provided ciencyvirus (HIV) asthethe basis for his work, acknowledg- "Breakthroughs of the Year," ining that he could not have achieved recognition of the potential power ofhis successes without them. accumulating knowledge about HIV As a society we love the science infectivity and chemokine receptors in the fight against AIDS. In hissuccess stories, but we sometimes opening editorial about the signifi-forget that science most often cance of these HW studies, Sciencemoves incrementally toward these editor-in-chief Floyd Bloom quotesbreakthroughs, relying on support Sir Walter Scott, who said "...Andfrom a scientifically literate public hope is brightest when it dawns(or portion of the public), and on the from fears." If, in fact, new knowl-work of many who may not receive edge about inhibitors of enzymesthe fanfare of our most notable sci- essential for viral replication pro-entists. This in no way diminishes vides hope for people infected withtheir work, or their hope of making HIV, then science has served itsan important discovery. Winston public well. But it is important to Churchill once remarked that "suc- remember that Science is a magazine for and by scientists, and thecess is nothing more than going public still needs to understand thefrom failure to failure with undimin- evolution of the science of HIV thatished enthusiasm." Somehow we has brought us this close to an effec- must promote this concept of sci- tive treatment strategy. ence just as fervently as we do the Fortuitously, one week after theheadline-grabbing successes. It is an Science announcement, Time maga-important, but essential lesson for zine named virologist David Da-Ithe budding scientist. It is a chal- Ho, M.D., "Man of the Year"lenging message for science educa- because of his leadership in thetors in a world where celebrity is an development of a therapeutic cock-increasingly important measure of tailof proteaseinhibitorsforachievement. patients infected with HIV. This Fast-movingscience,limited choice is enormously gratifying forresources, and too many distrac- all supporters of science, becausetions pose additional challenges to not only has a scientist been recog-our science education system, nized for his contributions to societyrequiring that we plan ever more but there has been an all-too-rarestrategically how to design, imple- convergence of the popular and sci-ment, and evaluate our efforts. entific press in a matter of significant public health importance. LOW aliMallitMiElillialialMELM It is important to remember, however, that the accomplishments ofEvaluating How Science Is science and scientists are not alwaysLearned as remarkable and ground breakingFor the past few decades all levels of as AIDS therapeutics. In the Timeeducation have been evaluating and article David Ho describes there-evaluating the way science is

Introduction ix 1 0 Joseph G. Perpich, Institute Vice President for Grants and Special Programs (left), with keynote speaker Daniel Koshland.

taught. Assessment has been a topicthrough graduate school. Standards of discussion at every HHMI under-developed by the National Research graduate program meeting sinceCouncil for precollege science edu- 1991, and it was the overarchingcation could suggest useful assess- theme of the 1996 meeting. At thatment strategies tocollege-level meeting program directors ex-science educators. pressed their desire for better evalu- Mary Allen of Wellesley College ation tools, some reported on thesummed up the fundamental issues progress of ongoing assessments,surrounding assessment as what, and others brought along evidencehow, and when to assess. Aaron of their successesenthusiastic stu-Ellison, Mount Holyoke College, dents and teachers who have beenadded a fourth: Why? Clearly, newly invigorated or reinvigoratedassessment can provide useful infor- Assessment has by their efforts. mation to educators, students, and been a topic of The 1993 program directors meet-the communities in which they inter- ing focused on institutional strate- discussion at act. However, assessment of science gies for effecting undergraduateeducation initiatives must take into every HHMI science education reform and theaccount a wide range of questions, undergraduate need to assess the degree to whichsuch as How can the effects of a spe- reform efforts have been successful.cific program be isolated from those program meeting Assessment is crucial to science edu-of other programs at a college or since 1991, and it cation reform, because it can provideuniversity? What is the long-term the essential data to determine what was the overarch- impact of such programs on stu- works and what does not, and whendents' decisions to remain in the sci- ing theme of the program interventions can have theences? How can quantitative and 1996 meeting. greatest impact in the science educa-qualitative data be used in conduct- tion continuum, from grade schooling assessments?

x Assessing Science Pathways Clearly defined objectives, well-under way at their institutions. planned methodologies, and mea-These include tracking the testMartha Peck surable outcomes are minimumscores and course choices of stu-said that her requirements for effective evalua-dents who have participated in sci- tion. Assessments can provideenceeducation programs andorganization, important answers in the aggregate.research experiences; evaluation ofBurroughs the career choices of women and For example, can they indicate what Wellcome Fund, types of interventions are receivedunderrepresented minorities; obser- positively and when they can best bevational studies of classrooms usingassesses its offered? innovative curricula and educationalprograms for a In 1996 representatives of founda-technologies; surveys to assess simple reason: to tions active in science and educationchanges in attitudes about science, attended the undergraduate pro-and evaluations of content and cur-see the value of gram directors meeting for the firstricular changes practiced by teach-its investment. time and added an important dimen-ers who participate in professional sion to the discussions about assess-development activities. Some exam- ment. Martha Peck said that herples of evaluation activities are organization, Burroughs Wellcomedescribed below. Fund, assesses its programs for a simple reason: to see the value of its investment. A major objective of the Fund's assessment is to determineSupporting What Works: the point in the education-careerStudent Research continuum (from elementary schoolExperiences through the postdoctoral and juniorSystematic research confirms what faculty levels) at which funding iswe know from anecdotal data, that most likely to contribute to an indi-the intense experience of student vidual's successful career path. research is an effective way to learn The Institute's Undergraduatescience. The Third International BiologicalSciencesEducationMathematics and Science Study, a Program aims to enrich educationalproject that evaluates science and opportunities for science majors andmathematics aptitude in more thanStudents who enhance the general scientific liter- half a million students worldwide,undergo the rigor acy of students who major in non-found that American students still science subjects. The program haslag behind their peers in some partsof in-depth awarded $335 million since 1988 toof Asia and Europe but that theexploration of a strengthen life sciences education atusual suspects, too much television 220 public and private colleges andand not enough studying, do notsubject...also universities. The goals of the pro-appear to be the cause of this deficit.provide an gram are addressed through four It is an understatement to say thatinvaluable lesson broad areas: student research andwe are not sure what works in sci- broadening access to science; equip-ence education. We suspect thatin what keynote ment and laboratories; science fac-many things help, like inquiry-basedspeaker Daniel ulty and curriculum; and precollegelearning,directexposureto Koshland calls outreach. research, the presence of mentors At recent undergraduate programand supportive family, and plain old"the sacredness of directors meetings, participantshard work. But what inspires one stu-the fact." have described evaluation activitiesdent may not ignite another. Human

;J2 Introduction xi beings are inconstant learners: some"spent an afternoon doing an experi- [Kristin] learn in a linear fashion, others in fitsment that was a failure because I for- Baldwin was an and starts, some by experience, andgot some fundamental cell biology." others by study. This variability undergraduate We are in the midst of a revolution in makes it difficult to design programsthe way undergraduate biology is economics major that will be broadly effective and thentaught and it will take some time to at Duke assess, measure, and predict whatsortouttheproperbalance. will make the difference in a stu- University [who I Assessment can help us in that dent's educational experience. process. More and more, research is was so captivated We also know from anecdotal evi- showing that what makes a differ- by her first real dence and some preliminary empiri- ence is how science and mathemat- brush with sci- ics material is taught. Studentscal studies that there is no substitute for the presence of a strong role ence that she exposed to too broad an array of content do not fare as well as stu-model or mentor in a student's deci- changed majors. dents who undergo the rigor of in-sion to stay in science. Among the depth exploration of a subject. Thiskey points to emerge from Wellesley type of experience also provides anCollege's Pathways for Women in the invaluable lesson in what keynoteSciences project is that students who speaker Daniel Koshland calls "thehave mentors and who participate in sacredness of the fact." undergraduate research are more The experience of Kristin Baldwinlikely to continue in science. This clearly makes the case for the valueconfirms our commitment to sup- of student research opportunities.porting faculty and teacher enhance- She also serves as an example of thement programs as well as initiatives danger of making science exclusive.that seek to provide science educa- Among the key Ms. Baldwin was an undergraduatetion opportunities through commu- points to emerge economics major at Duke Universitynity resources, such as churches, when she participated in a research from Wellesley communitycenters,andlocal fellowship sponsored by HHMI. Shegovernments. College's was so captivated by her first real Keith Amos is an example of a Pathways for brush with science that she changed"success story," an individual for majors. She now has an HHMI pre- whom we hope HHMI-funded pro- Women in the doctoral fellowship in molecular biol- grams made the difference in choos- Sciences project ogy in the laboratory of Dr. Mark Davis, an HHMI investigator ining a career in science. Under the is that students tutelage of Xavier University pro- immunology at Stanford University. who have men- Because so many undergraduategram director Deidre Labat, Mr. Amos participated in a summer tors and who institutions now offer laboratory- math and science academy as a 14- participate in based investigation as the mainstay of their curricula, they struggle withyear-old high school student. Amos undergraduate how to evaluate student progressis now a graduate of Xavier and a research are and how to strike a balance betweenstudent at Harvard Medical School. the research experience and funda-This past year as an HHMI-NIH more likely to mental course work. Tara Gupta, ascholar he conducted research at continue in sci- Colgate undergraduate who tookthe National Cancer Institute on ence. part in an off-campus researchpotential tumor rejection antigens in semester at NIH, reported that sheprostate cancer. xii Assessing Science Pathways 13 to practice it. The late Carl Sagan Making Science Education once said, 'We make our world sig-When receiving More Inclusive nificant by the courage of our ques-an honorary tions and by the depth of our There is something wonderfullyanswers. degree from democratic about the fact that we Once we make the decision thatRutgers really don't know who among ourwe will try to reach those who have University last children will rise to the highestnot already self-selected for science, ranks of scientific achievement. Itthen our strategies for educationyear, Yogi Berra means that it is in our best interest and evaluation must necessarilyopined, "In the to ensure that all students andchange. We must look more broadly teachers are able to feel, in theat those who seek careers inyears ahead, words of Jerry Pine of Caltech, theresearch and science teaching,when you reach a "rigors and joys" of science. make specialeffortstoenrollfork in the road, Students, more than anyone elsewomenandunderrepresented in our society, should be allowed theminorities, and find ways to deter-take it!" freedom to make real choices aboutmine the success of these efforts. where they wish to go. When receiving an honorary degree from Rutgers University last year, Yogi Berra opined, "In the years ahead,A Continual Search for when you reach a fork in the road,What Works take it!" New graduates are facingWhat is clear from the presentations alternative pathways to a scientificdescribed in this report is that all the career. The academic research labo-program participants have a good ratory might not be where they endsense of what does and does not What is clear up doing science in the near future.work. Feedback from participants, This changeinthe paradigmwhether students or teachers, is onefrom the presen- requires that we look differently at way of knowing. For example,tations described every aspect of science education,research experiences appear to have from how it is taught to whom wea palpable impact on a student's viewin this report is are targeting. Again, this requiresof science and an energizing effectthat all the pro- new tools for assessment. on teachers. Mary Co Ivard, a highgram partici- We often hear the argument thatschool biology teacher in upstate "the cream will rise to the top," thatNew York who has participated inpants have a those students who truly love sci-Cornell University's Institute ofgood sense of ence and harbor an inner drive toBiology Teachers, is a prime exam- pursue it will persist in scienceple. She describes the opportunitywhat does and despite all odds. That might be true,provided to teachers through thisdoes not work. but it is not an excuse for not provid- network as the basis of a "quiet revo-Feedback from ing better exposure to science for alllution" in which teachers are up- corners. Science should not be a for-dated scientifically and recognizedparticipants, bidding obstacle, in essence a boot-as professional educators. whether students camp experience intended to weed What we might never know is the or teachers, is out the weak from the strong. It is avalue of a single program, course, or way of thinking, a method of inquiry,experience in a student's decision toone way of that all should, at the very least,pursue a research career. Such an knowing. understand even if they choose notapproach is probably too simplistic

1 4 Introduction xiii when we consider the multiplicity ofteaching or course work. Our com- ...Grants for factors that influence learning andmitment of $335 million over the undergraduate career choices. past decade has supported a diverse Assessment, however, can provideset of programs to improve science science education important answers in the aggregate,education, not just at colleges and from 1,000 pri- that is, what types of interventionsuniversities, but also at the kinder- vate foundations are received positively and when isgarten through twelfth-grade levels. the best time in the educational expe-We are learning along with our and corporations rience to offer them. Martha Peck, ofgrantee institutions about what totaled $86 mil- the Burroughs Wellcome Fund, com-works in reaching these goals, and mented that, for funders, an impor- lion in 1994. The we applaud their efforts to enhance tantquestionis When isanthe art and science of educational Institute pro- interventionmostproductive?assessment. vided another Tracking students who go on to A survey we conducted this year careers in science may offer us an $29 million for revealed that grants for undergradu- opportunity to do retrospective analy-ate science education from a sample undergraduate sis about common experiences andof 1,000 private foundations and cor- perhaps draw inferences about their education in the porations totaled $86 million in 1994. effects in career selection. The Institute provided another $29 biological sci- Even with the most successfulmillion for undergraduate education ences alone. and proven programs however, wein the biological sciences alone.* In must continue to be diligent in our1993, the federal government in- efforts to provide lifelong safety netsvested nearly $500 million to sup- for those who choose science. Forport undergraduate science and example, Wellesley College has a math education, not counting more remarkable track record of retaining than $12 billion expended for stu- its undergraduate women in the sci- dent financial assistance, student ences: nearly one in four Wellesley graduates has a science degree.loans, and other support. Even Discouragingly, however, thesewith these relatively large sums, resources are increasingly con- women run the greatest risk of leaving the field within the first sixstrained, and funders of science edu- months of graduation. We need tocation must carefully target their Dr Varmus better understand the forces beyondsupport to the areas where it can have the greatest impact. reminded his the baccalaureate that discourage women from sticking with science. If we did not have so many excit- audience that ing options to pursue or support, "sciencelike no these decisions would not be so difficult. But the rapid growth of knowl- empire, no sect, A Sustained Conunitment edge about biology, combined with no starcan to Science Learning at All innovative classroom and laboratory Levels instruction, is creating a revolution eventually in undergraduate science, a phe- change the whole At HHMI we believe that the under-nomenon that, despite its excite- graduate educational experience face and state of ment, poses new challenges for opens up a world of opportunitiesfunders, institutions, faculties, and things through- for not only exposing new minds to out the world." science but reinvigorating people * Data compiled by the Foundation Center from who are already committed through Foundation Giving, 1966 edition.

xiv Assessing Science Pathways students. The rules are changing forgood science, the next best thing is everyone, making this a time of highto promote the learning of science,"The best thing demands and great hope. for its own value. for being sad is to This past year Harold Varmus, In The Once and Future King by M.D., Director of the NationalTerence H. White, the court magi-learn something

Institutes of Health, was nicknamedcian Merlin, who has been charged ...learn why the with educating King Arthur, urges "Dr. Who" by the editors of the world wags and Harvard Crimson when they learnedthe sad young man to study astrono- that he was to be the 1996 com-my. "The best thing for being sad iswhat wags it. mencement speaker. In his address,to learn something ... learn why theThat is the only Dr. Varmus said, "I speak for an ele-world wags and what wags it. That is ment of our culture at least as impor-the only thing which the mind canthing which the tant as politics or waran elementnever exhaust, never alienate, nevermind can never that has not been at this podiumbe tortured by, never fear or distrust,exhaust, never since Alexander Fleming, the dis-and never dream of regretting. coverer of penicillin, addressed theLearning is the thing for you."alienate, never be graduating class of 1945. That ele-Although we may all share the belieftortured by, never that learning is "the thing" for our ment is science." Science must grab fear or distrust, the bully pulpit, whenever possible,children, we are not likely to agree to remind its public that it is worthon how best to achieve that goal, par-and never dream the investment, the time, and theticularly with limited resources.of regretting. patience, and that a scientifically lit-Through careful planning and man- erate public is a wiser one. Dr.agement of science education pro-Learning is the Varmus reminded his audience thatgrams, we already know that we canthing for you." make a difference in how students "sciencelike no empire, no sect, The Once and no starcan eventually change thelearn. As we integrate new strategies, whole face and state of thingscurricula, and tools, we must continu-Future King, throughout the world." Good sci-ously find ways to measure ourT H. White ence is one end product of good sci-accomplishments and make changes ence education. But given that wethat will successfully carry our chil- can never truly know who will createdren into the next century of science.

Introduction xv 16 Overview of the Undergraduate Biological Sciences Education Program

Since 1988 theInstitute's Proposals are reviewed by an Undergraduate Biological Sciencesexternal panel of distinguished sci- Education Program has awardedentists and educators. The panel's $335.4 million in grant support toevaluations, in turn, are reviewed by 220 colleges and universities foran internal Institute committee that undergraduate science educationmakes recommendations to the (Figure 1). The goal of this programInstitute's Trustees, who authorize is to support efforts to strengthenfunding. Throughitsongoing the national quality of college-level assessment of the undergraduate education in the biological sciences and other scientific disciplines asprogram, the Institute has devel- they relate to biology. Anotheroped the program to respond to important objective is to supportnational needs in science education. outstanding programs that seek to In its first phase of operation attract and retain students in scien-(1988-1992), the undergraduate tific fields, including women andprogram supported programs of stu- members of minority groups. dent research and broadening Institutions have been invited toaccess to science; science faculty participate in the undergraduatedevelopment; curriculum, equip- competitions on the basis of theirment, and laboratory development; recent records of having graduatedand outreach programs linking sci- students who went on to medicalence departments with community school or to earn Ph.D.'s in biology,colleges, elementary and secondary chemistry, physics, or mathematics.schools, and other institutions. In its Data for these assessments were provided by the Association of American Medical Colleges, the National Research Council of the Figure 1 National Academy of Sciences, and 11111=111.11.1111=1.1 the U.S. Department of Education (Figure 2). Undergraduate Conipetitions (1988-1996) To identify institutions eligible for assessment, the Institute referred to 1988: 44 public and private comprehensive and liberal arts the classifications of higher-educa- colleges and universities were awarded $30.4 million in tion institutions by the Carnegie grant support for their science programs Foundation for the Advancement of 1989: 51 public and private research and doctorate-granting universities received $61 million to enhance undergrad- Teaching (1987 and 1994). These uate science education classifications are based on such fac- 1991: 44 public and private comprehensive and liberal arts tors as the level of degree offered, institutions wereawarded $31.5 million in grant support nature of the educational mission, 1992: 42 public and private research and doctorate-granting degree of specialization in particular universities were awarded grants totaling $52.5 million fields, and amount of annual federal 1993: 47 public and private comprehensive and liberal arts support for research and develop- institutions received grants totaling $28.5 million ment. The Institute also took into 1994: 62 public and private research and doctorate-granting account institutions' records of universities received grants totaling $86 million preparing students from minority 1996: 52 public and private baccalaureate and master's insti- groups underrepresented in the sci- tutions received grants totaling $45.4 million ences to pursue scientific careers.

Undergraduate Program Overview 1 7 Figure 2 second phase (1993-1994), the program continued its support of stu- 1MMIMMIIIIMM:Xdent research and broadening Assessment Criteria access activities and precollege and outreach programs, and included an To identify institutions for participation in the undergraduate biological sciences education program competitions, the increased emphasis on laboratory Institute conducted assessments covering the most recent 10- development through equipment year period for which data were available. Based on data on and renovation support. total baccalaureate degree production collected by the U.S. Department of Education, institutions were assessed on the The third program phase began in basis of the percentage and absolute number of graduates 1995, with invitations to 201 institu- from each institution who have tions classified as Baccalaureate Matriculated in Medical Schools Colleges I and II, Master's Colleges Data source: Association of American Medical Colleges and Universities I and II, and Schools of Engineering and Technology to Earned Doctorates in Biology compete for grants. In 1996 grants Data source: National Research Council of the National totaling $45.4 million were awarded Academy of Sciences to 52 baccalaureate and master's col- Earned Doctorates in Chemistry, Physics, or Mathematica leges and universities. Invitations will Data source: National Research Council of the National be issued in 1997 to institutions clas- Academy of Sciences sified as Research Universities I and II and Doctoral Universities I and II to compete for grants to be awarded in 1998. In this phase the Institute will continue its support for student research and broadening access to the sciences, equipment and laboratories, and precollege and other outreach programs. On the basis of Figure 3 informationcollectedthrough assessment activities, the program Awards to 220 Colleges and Universities includes support once again for fac- ($335.4 million) by Program Component, ulty and curriculum development in Phases IIII, 1988-1996 Competitions the Phase III competitions. Of the $335.4 million awarded, a total of $116 million supports stu- Faculty development Student research and $32 million (9%) broadening access dents in summer and academic-year $116 million (35%) laboratory experiences on and off campus, as well as prefreshman bridging programs, laboratory training, opportunities for students to present their research at scientific meetings, and other activities that promote a culture of science at the

Curriculum, undergraduate level. In the area of equipment, and laboratory Precollege and curriculum, equipment, and labora- development outreach programs tory development, a total of $114.4 $114.4 million $73 million (22%) (34%) million supports the enhancement 'of undergraduate science education

2 Assessing Science Pathways 1 8 through the acquisition of modernbers to broaden their knowledge teaching equipment, renovation ofbase or update their teaching skills. laboratories, and development of sci-A total of $73 million supports science curricula and educational tech-ence outreach programs for faculty nology. A total of $32 millionmembers, teachers, and students at supports the appointment of newtwo-year and four-year institutions science faculty members and oppor-and secondary and elementary tunities for existing faculty mem-schools (Figure 3).

9 Undergraduate Program Overview 3 The 1996 Program Directors Meeting Keynote Address Help for Scientists and Non-Scientists from the Howard Hughes Medical Institute Daniel Koshland, Ph.D. Professor, Department of Molecular and Cell Biology, University of CaliforniaBerkeley

am very impressed by all of theget grants and there are hundreds of imaginative HHMI programs. Underapplicants for every assistant profes- Dr.Choppin'sleadership,thesor position. Institute has progressed from being Have we overproduced scientists? a classic funder of research (whichA lot of people are suggesting that by itself is enough) to supportingwe should not train any more scien- undergraduate education, museums,tists. "We have too many people and and the whole spectrum of science. too little money," they say. In my What we all really ought to do isopinion, however, that approach sit back and say, "Gee, they're doing would turn a calamity into a disaster. such a marvelous job." However, I Benjamin Disraeli defined calam- know that the Institute is constantly ity and disaster by referring to his trying to improve its programs,competitor for prime minister of moving forward and thinking of newEngland, William Gladstone. He ideas. said: "If Mr. Gladstone fell into the Asking me here to discuss the Thames, that would be a calamity. If assessment of scientific careers andsomebody rescued him, it would be the other things that all of you whoa disaster." The solution that some are experts will be discussing in thepeople are suggesting to the prob- next couple of days is like carryinglem we have would actually be a coals to Newcastle. Since I am heredisaster. and very involved in this area, I The fact is that none of us who is would like to present a couple of myinvolved in training and assessing own ideas, some of which might bescientists can say to a young person, valuableand even if they are not"You can't go into science." In the valuable, they will infuriate you sodays before the National Institutes much that you will go out and workof Health and HHMI, people like twice as hard to get the rightMendel, Pasteur, and Darwin went answer. I am going to discuss theinto science. They often depended training of graduate students in sci-on personal friends and wealthy ence and then veer off a bit to talkdonors. Nobody today could say, about the training of nonscientists. 'We've got too many scientists. You can't go into it." I would like to see any of you (and I am an undergradu- .7.74W3, .7 *t1 ate adviser, too) say to a young man Too Many Scientists? named Albert Einstein, "We've got I think all of you know why this istoo many physicists" or say to a particularly important and contro-chemist named Pasteur, "We can't versial at the momentthat is, theuse you." shortage of funds for science. It is a We cannot turn off science or sci- different era from that in which Dr.entists. However, we also cannot Choppin and I started our researchexpect either to receive a lot more careers. Today it is very difficult tomoney for science. I hope there are

Keynote Address 7 21 New York Yankees, the newspaper reporters rushed up to him and said, "Mr. Ruth, how much are you getting paid?" He said, "You mean you get paid for playing baseball?" This is probably true of science, too, and there is no way to stop the flow of people going into science. Should we say that the whole thing is a fraud and that there are not enough problems to solve? Anybody who is familiar with biomedical research, which HHMI supports so well, knows that we have made a great deal of progress on infectious diseases, organic diseases, and inherited diseases. However, people are living longer, and we have not made as much progress on some of thementaldiseases,such as Alzheimer's and Huntington's, or on chronic diseases like arthritis that Daniel Koshland several more Howard Hugheses outmake old age painful. There are there who are willing to give a lot ofplenty of biomedical problems that money to science. Both Democratsstill need to be solved, to say nothing and Republicans in Congress haveof the problems of the environment been very protective of science, andand the toxic waste problem that it may be that funding can go up angets worse every year as our popula- incremental amount, but we cannottion increases. Clearly, there are expect it to increase in proportion toplenty of problems. the number of people who want to be scientists. Why is the number of scientists increasing exponentially? When youAssessing Scientists think about it, scientists are proba-What can we do about the oversup- bly the luckiest people in the world.ply of scientists relative to the num- We like to fiddle around and solveber of research positions and the important problems, and somebodyavailability of grant funding? In my is willing to put up capital to let usopinion, one of the most important play these games. The secret hasthings we can do is to more accu- gotten out that science is a veryrately assess how good people are. attractive profession. Even if we knew the exact num- I wrote recently that I have spentber of jobs in science, which nobody my career having fun and gettingcan define, we could not train one paid for it. I thought I was beingscientist for each job. Universities original until somebody told me awill never admit this, but they love story about Babe Ruth. When heto have a lot of applicants for a posi- was a young man, just hired by thetion. Companies like it, too. Even

8 Assessing Science Pathways 22 professors like to have more thanappeared in Science magazine dur- one applicant for a postdoctoral posi-ing my editorship. Two professors tion in their laboratory. Professorsof psychiatry became interested in also like to have more than one stu-how decisions were made about dent working with them. Over thewhom to release from prison on course of an active lifetime inparole. With the cooperation of research, a professor can produce aauthorities,theyobtainedthe large number of duplicates of him-records of prisoners who were self. This is one of many forces thatabout to be released on parole. They are producing a lot of scientists. knew what crimes the inmates had As a professor at Berkeley, I sit oncommitted, what their lives had committees that select students forbeen like beforetheirarrest, admission. We consider about 30 orwhether they had a history of 40 applicants for each student wearrests, and so on. They got assess- admit. How do we decide whom toments from a variety of sources admit? We look at grades, quantita-the predictions of the parole officer, tive and verbal SAT scores, and let-the prison chaplain, a nurse, a psy- ters from professors the student haschiatrist, and the arrest dossiers worked with. We add up all theseas to what was likely to happen things and we vote. when the inmate was released. One professor says grades are The researchers put all of these the most important factor. Anotherdata into a computer and correlated says, "He got a B in this course, butthem. What they found, somewhat his research professor said he did ato their astonishment, was that the great job in research." Somebodyassessments by the psychiatrist, the else says, "Her SAT was terrific."parole officer, or any of the others Another says, "The verbal SAT waswere not very good. The one thing great but the quantitative was terri-that correlated strongly with what ble." By combining all of thesehappened to the prisoner after things, we admit those who we thinkrelease was his previous record. are the best students. We publishedthis paperin However, we rarely go back to tryScience. I thought it was a great arti- to correlate those assessments withcle. However, it caused an enormous how the students actually did. Now,stir, and a lot of psychiatrists were if a company like General Motors ordeeply offended. But it emphasized Goodyear Tire ran its business with-the fact that conditions of complex out assessing how good its initialevents may be between little under- selections were, it would be out ofstood factors and not just a compila- business. It is a minor scandal thattion of general wisdom. scientists have been so careless A similar type of assessment about assessing how good a selec-should be applied to science.I tion job they do. This suggestion willwould like to see a program that not be regarded as anything new bykept records on individuals, starting HHMI. It's something that theywith high school and continuing and many of youare already start-throughcollegeandgraduate ing to do. school: records of grades including I would like to remind you of asuch items as SAT scores, as well as study that caused quite a stir when itassessments of how various people

23 Keynote Address 9 thought the person would do in sci-level. This student's ambition is to ence, for example, the committeego to graduate school. In the current that admitted her to graduate schoolclimate I would say to him, "I think and the professors who assessedthat's probably not the best idea. her when she passed her first quali-There are a lot of other things that fying exam. You would also obtain ayou can do with your life. Your train- rough assessment of how good aing in science will be helpful to you. researcher she is 5 or 10 years afterBut for you to compete in the mod- she got her degree. Then, using aern world of grant-funded research computer, you would try to correlateis going to be very tough." these data. If we could do this, we Of course, there are also students would be able to do a much betterwho get much better grades than job of resolving the ratio of appli-the student mentioned above as cants to jobs. One could say, "Yourundergraduates and who get into SAT score is 'x.' We find that thatgraduate school but who are just not score almost invariably leads to athat good as researchers. We are meaningful career in science." now starting to advise these students to go into different careers.

FT.-777r; If we had the basic data for this kind of assessment, we would be Steering Students to doing a good thing for science in Alternative Careers general. We will get better teachers, Some of the people who cry thatbetter museum directors, and better there is a terrible crisis in science dopeople with scientific training in not mention the fact that scientists'other careers. I would like to appeal technological unemployment rate isto the Institute to consider funding a less than one percentone of thepilot project of this nature. Some lowest in the world. Technologicalgraduate students in the social sci- unemployment is defined as notences could get their Ph.D.'s by col- doing work for which you arelecting this information. trained. It does not mean that if your Some people have said to me that ambition was to be a professor atthey are concerned that assessment Wisconsin and you did not achievemight steer minorities away from that ambition, you are technologi-science. To them I would say that cally unemployed. If you ended upyou are doing a favor to a minority as a high school science teacher orindividual as much as to a majority took a position at an institutionindividual if you are candid with him less distinguished than those repre-or her. The criteria for minorities sented at this meeting, that wouldshould be exactly the same as for not be considered technologicalthe majority. You want to help the unemployment. student find the profession in which My point is that we have to learnshe will be the most useful and will how to help students choose differ-enjoy herself the most. ent careers. Let us say that I have an I would guess that M.D.'s may undergraduate student who has got-have had higher average grades ten a B in elementary chemistrythan researchers as undergradu- and a B in organic chemistry, andates, but they have a different kind his grades continue at about thatof mind. They will do a very good

lo Assessing Science Pathways 2 4 job as an M.D. and probably notstudents who arenot already such a good job as a researcher.inclined toward science come into That is the kind of issue I would likescience at that stage. It is important to have more information about. to maintain them and train them, but it is in kindergarten and primary school where the selection takes place. Schools and Learning I heard a story, which is probably The New YorkTimes recentlyrelevant, about a father saying to his reviewed a book about some of theson, "Son, you've got to go to school. arguments regarding the relevanceIt's 10:00 in the morning. It's your of school curricula. I read it and Iduty to go to school." The son says, thought, "You know, that's probably"Dad, I hate school. The students part of my problem, because I reallyhate me, the professors hate me, the enjoyed school and any kind of prob-principal hates me. Why do I have to lem." For example, I always enjoyedgo to school?" The father says, math. I read an article about efforts"Well, in the first place, you're 35 to make math more meaningful toyears oldand you're going to have students by relating it to the budgetto go to school. In the second place of the United States or some otheryou are headmaster and it's your problem. I did not care about theduty." budget of the United States when I Schools are like football teams was an undergraduate, but I lovedthese days. Everybody knows how the problem in which you had to fig-to make them better. Everyone is ure out how fast you were moving insecond-guessing headmasters, so a canoe, rowing at three miles anprobably a lot of headmasters hate hour, when the stream was movingto go to school. But they are the at five miles an hour. That was aones in the firing line, helping ordi- great problem, and who cared aboutnary students, gifted students, and the budget of the United States? people of every race and from every I asked some of my graduate stu-walk of life. In a democracy we have dents about this, and almost everyto keep our schools doing well. one of them had the same experi- We must have new young people ence that I had: They were verybecoming scientists, and we cannot good at math as K-12 students.create a situation in which the pub- Many of them are no longer inter-lic's image is that science is a terri- ested in working on mathematical orble field to go into. We cannot go to computer problems, but they werethe legislators and say, 'We're train- very good at it in elementary schooling these students. You've got to and high school. What happens isgive them jobs." that when you are very good at Nobody guarantees an architec- math, you enjoy school and do notture student that there is going to be need to be told about relevance. a building boom when he gets his You in this audience who are indegree. Nobody guarantees mem- undergraduate education tend tobers of other professions that they feel that you must keep the goodare going to have a job or that they students and train them well. It isare going to have all the support probable, however, that very fewthey need to do the job. The justifi-

Keynote Address ii cation for scientists is that we arequantum mechanical atom, because solving problems that are of greatwhen we were able to measure relevance to society. It is our job toatomic spectra with a high degree of send a message to which Congressresolution, it became clear that the is really sympathetic. Bohr atom was a deficient model. I heard a story that illustrates this issue. An admiral was commanding a battleship in the South Seas on a Training Nonscientists stormy night when the crew could Another subject that is of concern tobarely see what was ahead of them. me is the training of nonscientists.Suddenly thefirstmate said, This subject is important for three"Admiral, sir, there's a light ahead, reasons. First, some people who doright on a collision course with us, not think that they are scientistsand you've got to stop it! Something may become interested in it andterrible is going to happen!" So the shift, which would be good. Second,admiral got on the two-way radio we really ought to have people mak-and said, "Ship ahoy, ship ahoy, on ing budgets who understand thecollision course. Turn your vessel benefits of science and its impact on40 south by southeast." The reply society. Third, *scientific training iscame back, "I'm sorry. You'd better going to be very important in tryingchange your direction 40 north by to resolve some of the terrible prob-northwest." The admiral said, "I lems we face in our society. don't think you understand. This is Scientists are particularly good atAdmiral Vanderbilt and I'm on a bat- solving problems. I have noticedtleship. I think you'd better change that on the interdisciplinary depart-your direction 40 south by south- mental committees that we have ateast." A little voice came back Berkeley, the chairman is usually asaying, "I'm sorry. You don't under- scientist. An English professor usu-stand. This is Midshipman Jones, ally writes the minutes because theand I'm on a lighthouse." scientists are illiterate, but the scien- There are some facts that really tists can keep things on track. do convince you to change your What we need to do is to try tocourse. convey some of the ways in which scientists have solved problems. I One of the things that is very important is what scientists call theEnvironment or Heredity sacredness of a fact. I sometimesAs editor of Science, I once wrote an hear people who really do not under-editorial about spousal abuse in stand science say, "The exceptionwhich I mentioned how effective the proves the rule." Well, of course, theorganizationMothersAgainst exception does not prove the rule inDrunk Driving had been and said science; it breaks it. that it was time to act more concert- In science, if you obtain new dataedly on spousal abuse. In the that do not fit your existing theory,process of writing that editorial I you must get a new theory. That islearned some statistics: 50 percent how the Thompson atom becameof the women who are killed in the the Bohr atom and eventually theUnited States are killed by a hus-

12 Assessing Science Pathways 26 band, lover, or would-be lover. Thatknow what tests you would have to is a staggering number. Of men whodo to investigate that question. I was kill their wives, about 80 percent arestunned to find that these social orga- the sons of men who were alsonizations had never thought to do involved in spousal abuse. this and did not know how to do it. It struck me that a statistic of this I am appalled when I read in the magnitude might indicate that anewspapers about welfare reform hereditary factor was at work. As weand policy makers saying that all we know from experience with otherhave to do is train these people and hereditary factors, if we can find thethey are going to have jobs. Fifty gene, we may be able to develop apercent of the homeless are men- therapy. So I called some of thetally ill. You are not going to find a agencies involved in spousal abuse,lot of new jobs for people with prob- some of which had written to tell melems like that, no matter how you that they liked the editorial I hadtrain them. written, and I asked them whether No doubt there are tough deci- they had considered the possibilitysions to be made when setting of a hereditary factor. budgets, but getting the basic facts Almost of them said, "Oh, no.on which you base a decision is This is a learning factor. These menimportant. think they can get away with battering their wives because they've seen their father battering their mother." I said that it did not seem to me thatValue of the Scientific wife-battering was an instinctiveApproach thing that all of us would do ifAnother important thing that scien- we learned that we could get awaytists can do is what I call the differ- with it. Besides, there is some-entiation of a project. As scientists, thing called the Oedipus complexwe know that big, complicated prob- the notion that boys identify morelems are not solved by intellectualiz- with their mothers than with theiring. A lot of people think that the fathers. beginning of modern science was "Have you not investigated thethe famous Buchner experiment, in hereditary factor?" I asked. Theywhich the Buchners broke up yeast, responded, "How would we dogradually isolated the enzymes, and that?" I mentioned the very famouswere able to reconstruct the metab- studies led by Seymour Kety andolism of sugars by breaking the others that identified schizophrenia,problem up into small pieces. manic depression, and other mental We are not trying to cure cancer illnesses as hereditary. The socialby constantly discussing cancer as a scientists were not familiar withwhole. We are facing the fact that these studies. I mentioned studies ofcancer has many causesinheri- adopted children, which correlatedtance in some cases, carcinogens in them with their adoptive family orothers, viruses in still othersand their biological family. we tackle each of these causative Now, I do not know whethermechanisms individually. spousal abuse is caused by environ- Too often when I discuss social ment or heredity, but I certainlyscience issues with some of my

Keynote Address 13 2 7 friends, they say, "It's no use havingtant that some of that knowledge be a deterrent for murder, becausetransmitted, not only to try to instill murder is a crime of passion andan appreciation of science but also you can't stop it." Well, that is notbecause it helps people to under- true. Some murders are crimes ofstand the issues they are going to be passion. Other murderers are peo-faced with. ple who want to eliminate witnesses The 0. J. Simpson trial is proba- or take revenge, or they are young-bly a perfect example of the misuse sters who hold up a mom-and-popand misunderstanding of science. store and then panic. There are allPsychological testing in which one sorts of murders, and you have tois able to predict what a juror is differentiate them in the same waygoing to decide before the trial even that you differentiate cancers. starts, regardless of the evidence, If scientists were still studying theand using that information for population genetics problem bywealthy clients is an example of the looking at the total population, wemisuse of science to subvert the would not have gotten very far.jury system. People talk about the Through being able to sequencemisuse of genetic engineering, but DNA, however, the progress hasthey should start talking about the been enormous. The ability of sci-misuse of social science. ence to break problems into parts is I heard a story about a young very important. man who lived down the street from I would like to advocate gettingCharles Darwin. This young man, the really great talents in science towho may have been a lot like me, spend some time teaching nonscien-thought, "I'm going to fool this tists. Such teaching may be a moreDarwin. He's supposed to know a prodigious problem than evaluatinglot." He got the body of a beetle and scientists, because all the scientists Iskillfully glued on the legs of a spi- know were very good in elementaryder and the wings of a butterfly. He school mathematics. Logical proce-took this creature to Darwin and dure is something that appealed toasked, "What kind of an animal is them. Most of the people who arethis?" Darwin looked at it and said to not scientists, however, hated math.the young man, "By the way, does So we may have to think differentlythis bug hum?" The young man had when we teach nonscientists. not thought of that, but he recov- Many liberal arts courses in sci-ered quickly. "Oh, yes, it hums." At ence at universities are what I wouldwhich point Darwin said, "Then I call tourist education: You're flyingthink it's a humbug." over a country at 30,000 feet and I do not know whether that is the someone says, "Here's physics" orconclusion of my talk. I know I am "Here's chemistry." The coursetalking about areas in which the becomes a very generalized com-Institute is very involved, not only in ment instead of getting down andtraining nonscientists but also in learning the standard scientific pro-supporting museums and so forth. tocols as to how you distinguishMuch as I hate to see any money between one decision and another. Itdiverted from the support of basic is not going to be easy to changeresearch, I think that the Institute's that. On the other hand, it is impor-broad viewfocusing on the impor-

14 Assessing Science Pathways 28 tance of science in societyis the A member of the audience con- right one. In addition to support fortrasted attitudes toward career the assessment of science pro-choices among students and profes- grams, there is a need to supportsors in science with those in the fine programs for people who will notarts, who "have no illusions about themselves become researchers buttheir chances of becoming Picasso. will go into other professions andThey take the competition for grant- help to improve the general under-ed." By contrast, she said, scientists standing of science in society. assume that there will be a job for every graduate.

Teaching Science to Nonscience Discussion Majors. Faculty who teach science to Career Choices for Science Graduates.nonscience majors must demon- Many in the audience took issuestrate a lot of creativity and innova- with Dr. Koshland's suggestion thattion, yet generally they are not grades might be a basis for identify-rewarded for these efforts, one par- ing those students most likely to suc-ticipant commented. Dr. Koshland ceed in science. Such a strategynoted that although much depends risks diminishing the diversity ofon the personality of the individual approaches that enriches science,faculty member and on whether said one listener. Dr. Koshlandinstitutions consider the teaching of responded that although this was ascience to nonscience majors to be concern, "a more rational selectionimportant, some great scientists process would produce a better situ-have always enjoyed teaching fresh- ation than the present one, in whichmen and nonscience majors. so many people are disappointed and the feedback to young people about Nature and Nurture. A listener a career in science is negative." chided Dr. Koshland for jumping to Another listener said that somethe conclusion that domestic vio- ongoing studies suggest that gradeslence has a genetic basis. Dr. are a poor predictor of career suc-Koshland responded that scientists cess andthat involvementin should not have preconceived ideas research might be a much betterbut should not be afraid to seek data one. Dr. Koshland said that it wason such controversial issues. He important to establish whether orsaid that he sometimes feels that not grades are a predictor of successsocial scientists "don't want to hear and that such information would bethat anything is genetic." helpful to professors attempting to advise students on career choices. In response to those who said that they felt unqualified to advise stu-Biographical Sketch dents about alternative careers, heDaniel Koshland said that students will ultimatelyDr. Koshland was born in New York make their own career decisions,City on March 30, 1920. He earned but it would be helpful if more objec-his B.S. degree at the University of tive data were available to guideCaliforniaBerkeley in 1941 and his those decisions. Ph.D. from the University of Chicago

Keynote Address 15 in 1949. After two postdoctoral years Dr. Koshland has contributed at Harvard, he joined the staff ofmore than 400 articles to the scien- Brookhaven National Laboratorytificliterature.His early work and later had joint appointments atfocused on enzyme mechanisms Rockefeller University and Brook-and protein chemistry, leading to haven until 1965. At that time hethe concept of single and double dis- joined the faculty of the University ofplacement reactions, the develop- CaliforniaBerkeley in the Depart-ment of reagents of carboxyl groups ment of Biochemistry. and tryptophan, and the analysis of Dr. Koshland is currently a pro-factors that explain the high cata- fessor in the graduate school in thelytic power of enzymes. This work Department of Molecular and Cellled to his concept of the induced fit BiologyattheUniversityoftheory, the general work on cooper- CaliforniaBerkeley and was editorativity mechanisms, and the discov- of Science magazine from 1985 toery of negative cooperativity. During 1995. He isa member of thethis period the concept of reporter National Academy of Sciences, thegroups was developed and demon- American Academy of Arts andstrated experimentally. Sciences, the American Chemical His recent work has focused on Society, and the American Society of mechanisms of behavior with bacteri- Biological Chemists. Among his al chemotaxis as a model system. honors are the National Medal of Science, the Edgar Fahs SmithBacteria were shown to have a mem- Award of the American Chemicalory that is essential for their survival. Society, the Pau ling Award of theThe receptors and many of the gene American Chemical Society, theproducts of the processing system Rosenstiel Award of Brandeishave been isolated, and the biochem- University, and the T Duckett Jonesistry of their sensory and adaptive Award of the Helen Hay Whitneyproperties has been delineated. Foundation. He was a Guggenheim Dr. Koshland's research has pro- Fellow and a Visiting Fellow of Allduced major advances in our under- Souls College at Oxford University.standing of enzymes and protein He has been elected an Honorarychemistry. He is an international Foreign Member of the Japaneseleader in research on short- and Biochemical Society and the Royallong-term memory. As editor of Swedish Academy of Sciences. Science, the largest-circulation sci- He was elected president of theentific journal publishing primary American Society of Biologicalresearch across the entire spectrum Chemists. He helped establish andof the sciences, he had an active role was chairman of the Academynot onlyinselecting primary Forum, a committee of the Nationalresearch on the frontiers of science Academy of Sciences, which isbut also in influencing and acting as charged with helping to develop pol-a spokesman for science policy. He icy on issues that pose dilemmas atwas instrumental in improving the the interface between science andreporting of scientific discoveries so societal problems. He has beenthat they are understandable to lay- elected to the council of the Nationalmen and to scientists in different Academy of Sciences. disciplines

16 Assessing Science Pathways 3 0 Panel and Workshop Discussions

evaluationLichteremphasized Assessment that the science and higher education communities must find ways to A ssessment was the overarching theme of the 1996 undergradu-do assessment more effectively. ate program directors meeting, and Raymond Bacchetti, William and this topic was woven into almost allFlora Hewlett Foundation, observed of the discussions that took placethat as a result of changed public during the meeting. Although meet-attitudes, higher education is now ing participants had many questionsperceived as an interest group. and concerns about assessment,Higher education needs to defend they had no doubt that it is anitself more effectively with regard to increasingly important issue in sci-results, he said. Without assess- ence education. ment, there is no.basis for making Mary Allen, Wellesley College,the case that higher education is summed up the fundamental issuesimportant and effective. surrounding assessment as what, Higher-educationaccrediting how, and when to assess. Aaronagencies are beginning to require Ellison, Mount Holyoke College,institutions to have assessment suggested that a further importantmechanisms in place, said John question for scientists is, What is theMickus, Benedictine University. hypothesis? When his institution went through a Representatives of foundationsreaccreditation process recently, it making grants in science and educa-was asked to demonstrate that mea- tion, attending the undergraduatesurable objectives had been set for program directors meeting for thenew courses. first time, offered their perspectives Determining what toassess on the importance of assessment.involves setting clear goals and Martha Peck, Burroughs Wellcomeobjectives for programs. Many Fund, said that her organizationspeakers emphasized the impor- assesses its programs for a simpletance of thinking about assessment reason: to see the value of its invest-criteria up front when designing a ment. The fund would like to know,program. "Goals are more clearly for example, at what point in thedefined if at the outset one takes into education-career continuum (fromaccount how theymightbe elementary school through the post-assessed," said Frank Vellaccio, doctoral and junior faculty level) College of the Holy Cross. funding is most likely to contribute Clearly defined objectives, well- to an individual's successful careerplanned methodologies, and mea- path. surable outcomes are necessary but Robert Lichter,Camille andnot sufficient conditions for good Henry Dreyfus Foundation, definedassessment, according to Michael assessment as a process or set ofGaines, University of Miami. Gaines activities designed to answer theadded that he was dissatisfied with question, Does something matter?what he called the bean-counting Although noting that the concept ofapproach to assessment, expressing assessment is ambiguousthere isa widespread view among meeting disagreement, for example, onparticipants that many benefits of whether assessment is the same asscience education programs are

Panel and Workshop Discussions 17 31 ;11

Several panelists intangible. Alan Gubanich, Uni-competition for research positions conducted a plenary versity of NevadaReno, spoke forhas never been more intense and session on tracking many when he said, "We have tostudents are being urged to consid- science graduates' think about how to define success." career pathways. er alternative careers. Allen com- From left to right: In a diverse student population,mented that it is as important to Martha Peck, students may have very differentproduce scientifically literate legisla- Burroughs Wellcome starting lines. Deidre Labat, Xaviertors as more researchers. Fund; Mary Allen and Christine Birchwood, University of Louisiana, said that Discussions of how to assess fre- Wellesley College; she measures success by the extentquently focused on the difficulty of Keith Amos, Harvard of systemic change or by seeingestablishing appropriate control University Medical School; and Kristin improvements in the performancegroups. Lichter said that although Baldwin, Stanford of the best students trickle down toscientists tend to favor controlled University. that of the average students. experiments, it is not clear from the Defining success by the classicalevaluation literature that controlled criteria, which focus on numbers ofexperiments are always necessary. students who pursue careers in sci-Helene Slessarev, Wheaton College, entific research, numbers of publica-said that results similar to those of tions, numbers of research grants,controlled experiments can be and so on, may be both inadequateobtained by simpler methodologies and inappropriate at a time whensuch as the use of focus groups.

18 Assessing Science Pathways 3 2 Slessarev echoed the commentsbe appropriate to an institutional set- of many other meeting attendeesting (one size does not fit all). when she urged program directors Several speakers drew a distinc- to build linkages with their col-tion between assessment and track- leagues in the cognitive and socialing. Trackinglong-term follow-up sciences who have experience inof program participantsis a neces- program assessment. Fredericksary precursor for assessment. It Eiserling, University of Californiawas noted that institutions' alumni Los Angeles, described the positiveoffices can be a useful resource in results of one such collaboration attracking graduates. For some atten- his institution: After his colleaguedees, however, tracking raises pri- Orville Chapman worked with a fac-vacy concerns. Karen Olmstead, ulty member in the psychologyUniversity of South Dakota, urged department, an expert in percep-that institutions seek students' per- tion, to reorganize an organic chem-mission for tracking and allow them istry course, test scores increasedto choose not to be tracked. by about 50 percent. Keith Amos, a graduate of Xavier However, test scores are just oneUniversity who is now a student at way to measure a program's impact.Harvard Medical School (and a for- Another equally important variablemer participant in the HHMI might be student attitudes towardNational Institutes of Health [NM] science, the measurement of whichResearch Scholars program), said would require a different kind ofthat one of the benefits of tracking assessment tool. As for when tofrom the perspective of program assess, participants generally feltparticipants is that it enables them that the most valuable data are likelyto keep in touch and build valuable to come from long-term studies.peer networks. It also provides a Allen noted that assessment of insti-corps of program alumni who may tutional change is also important. be willing to serve as mentors to Another important question isfuture participants. Thus, tracking How much data is necessary tohas benefits unrelated to the provi- prove something? For example,sion of assessment data. must every institution assess the value of student participation in research, or are a few representative studiesenough?SarahElgin,Presi citors of Success Washington University, suggestedha Science that a few large, sophisticated stud-Mary Allen presented an overview ies might be more valuable thanof the results to date of the Pathways many small, unsophisticated ones.for Women in the Sciences project, a This approach would also conservelongitudinal study of career choices resources. andattitudechangesamong

,Some attendees expressed con-Wellesley College graduates, funded cern that devoting resources toby the Alfred P. Sloan Foundation assessment meansthatfewerAmong the key points to emerge resources are available for otherfrom the study so far are that program activities. Others cautionedstudents who have mentors and that assessment measures need towho participate in undergraduate

Panel and Workshop Discussions 19 3 3 Wellesley College: Pathways for Women in the Sciences

Women remain underrepresented in scientific careers, particularly at the senior levels of the science professions. In an effort to better understand the factors influencing women's choice of and persistence in a science career path, Wellesley College, with funding from the Alfred P. Sloan Foundation, initiated the Pathways for Women in the Sciences project in 1991.

The project's first phase comprised a longitudinal study of the Wellesley class of 1995 and a retrospective study of alumnae who graduated with majors in science or mathematics from 1983 to 1991. It is one of the few carefully controlled longitudinal analyses of postgraduation activities. Key findings of The Wellesley Report, Part I, published in 1993, include the following:

DIE Interest in pursuing science or mathematics as a career is probably developed before college. Science majors are more likely than nonscience majors to score highly on mathematics SATs. Science majors and premed students are more likely to have parents in science, mathematics, and health-related careers. Students in science are more likely than other students to have received encouragement from multiple sources to pursue science. Science majors are more likely than nonscience majors to see society at large as supportive of women in science. Students in the physical sciences are more likely than those in the life sciences to cite college teachers as a source of support, encouragement, and influence on their choice of major.

Part II of the report, currently in preparation, focuses on about 800 women from the classes of 1968-1991 who graduated from Wellesley in science, mathematics, or computer science and who did not go into medicine. (The 342 women-30 percent of science graduateswho went into medicine were studied separately.) The following data are excerpted from the prepublication report with the permission of Wellesley College.

More than 40 percent of the study cohort (345 women) had left science by the time of data collection. The study found that about a quarter of science graduates leave the field at graduation, and a further 2-3 percent leave each year from year 1 to year 11 after graduation.

Ell Those who remain in science at year 11 after graduation represent just over half of those who graduated with a mathematics, computer science, or science degree. Few women leave after 11 years in the field.

Ell Having a mentor while an undergraduate is associated with retention in science. By the end of year 11, 62 percent of women who had a mentor are retained, compared with 52 percent who did not have a mentor. More than half of all women in science careers said that they would not be where they are today without their mentor. No differences were found between women who had a male mentor and women who had a female mentor. IN Participation in undergraduate research is also associated with retention in science. At the end of year 11, 60 percent of women who engaged in undergraduate research are retained, compared with 53 percent of women who did not engage in such research.

These data are significant for the Institute's undergraduate biological sciences education program, which is currently providing research experience with a mentor for 20,000 undergraduates.

20 Assessing Science Pathways 34 COPY AVMLABF Michael Gaines (left) and Beatriz Blanco, University of Miami, and Richard Storey, Colorado College, took part in a workshop discussion on bridging programs.

ht..;

AY '

research are more likely to pursuewere no laboratories at his high graduate education and careers inschool in a small Louisiana town. science. A mentor's most importantWhen, at 14, he expressed interest functions are providing support andin the summer science program at encouragement; providing careerXavier University of Louisiana, he adviceissecondary.Parentalwas astonished to receive a personal encouragement is also crucial. letter from the program director, An anecdote related by KeithDeidre Labat. "I still have that letter Amos illustrated the importance of atoday," said Amos. mentor's taking a personal interest Another student participant in the in a student. He said that he becamemeeting, Beatriz Blanco, seemed to interested in science in junior highbe living proof of the accuracy of the school and was determined toWellesley study's findings. Blanco become a doctor. However, theretransferred to the University of

Panel and Workshop Discussions 21 Miami from Miami-Dade Com-research in off-campus laboratories. munity College after participating inTara is doing research at NIH. an HHMI-supported bridging program. She said parental encouragement had been the biggest factor in her success, followed by good highImportance of school teachers, a dedicated mentorUndergraduate Research (Michael Gaines), research experi-All of the student participants at the ence, and financial support frommeeting spoke emphatically of HHMI. the importance of undergraduate Scott Santos, a graduate of theresearch experience in motivating University of Hawaii at Manoa whothem to pursue careers in science. is now a graduate student at theKristin Baldwin, a graduate of Duke State University of New York atUniversity and now a graduate stu- Buffalo, said that the opportunity todent at Stanford University, changed work at Waikiki Aquarium's blue-her major from economics to bio- water marine lab during his seniorlogy as a result of her participation year of high school had been the cat-in an HHMI-supported undergradu- alyst for his decision to pursue aate research fellowship program. career in science. Before that expe-She is now working in the laborato- rience he had not even planned tory of Dr. Mark Davis, an HHMI go to college. investigator, and is the recipient of Santos participated in the HHMIan HHMI predoctoral fellowship. program at the University of Hawaii Nancy Pruitt described Colgate at Manoa, where his mentor wasUniversity's off-campus study pro- Robert Kinzie of the Hawaii Institutegram that enables selected students of Marine Biology. "He taught meto spend the fall semester of their that research is fun. Someone is pay-senior year doing intensive research ing you to have fun." at NIH. Students spend at least 30 Christine Birchwood, a recenthours each week working on a labo- graduate of Wellesley, said that theratory research project under the personal relationship between adirection of an NIH scientist and mentor and a student may be thealso take one to three courses most important factor in determin-required for the undergraduate sci- ing whether the student goes on to aence major. The students receive career in science. credit for two science courses. High school teachers can be men- One unresolved problem with off- tors, too. Tara Gupta, a student atcampus research programs, several Colgate University, told of becomingattendees observed, is how to enable interested in biology because of astudents to follow through on their high school teacher who told herresearch experience after they return that there were still many unknownsto campus. Pruitt said that as a result in the field. "That was the most excit-of feedback from students indicating ing part of biology for methatthat one semester at NIH was not there was so much empty space andenough, Colgate has decided to room for me to fill it." Her mentor atexpand the program by allowing stu- Colgate was Nancy Pruitt, whodents to begin their laboratory expe- advises students planning to conductrience the summer before the fall

22 Assessing Science Pathways 36 semester. In addition, in 1997 Colgatealso enjoyed the professionalism and A workshop discussion will open the program to a small num-the sense of community evident in on preparing students ber of students from other colleges. the laboratories where they worked. for research careers featured (left to right) Tara Gupta, who took part in Both students and program direc- Nancy Pruitt and Tara the off-campus program, describedtors noted that research experience Gupta, Colgate the difference between classroomcan also be counted a success if stu- University; Christopher Womersley, University laboratories and actual researchdents learn that they do not like lab- of Hawaii at Manoa; experience as being "like the differ-oratory work and therefore decide Scott Santos, State ence between taking driver's educa-to pursue other career options. University of New York at Buffalo; and Ted tion classes and actually driving Research experience, however Greenwood, Alfred P. a car." valuable, does not give students a Sloan Foundation. Several of the students said thatbroad background in fundamental independence, although often thescientific knowledge. A number of most challengingaspectofaspeakers stressed the need for a bal- research experience, was also theance between research experience most valuable because it made themand fundamental course work. Tara think and forced them to be self-Gupta confessed that by concentrat- reliant. In a classroom laboratory,ing on research as an undergradu- Christine Birchwood said, "you areate she had not learned some of the told what to think, what to do, andbasics. "I spent an afternoon doing what to expect." Doing research hadan experiment that was a failure given her confidence in her ownbecause I forgot some fundamental ideas and abilities, she said. Studentscell biology."

3 7 Panel and Workshop Discussions 23 Anotherstudentparticipant, Kathryn Goettge, a graduate ofCareer Choices for Lehigh University who is now a grad-Science Graduates uate student at the University of As competition for academic and North CarolinaChapel Hill, pointed research positions has stiffened, the out a possibly unintentional conse- traditional view of a successful sci- quence of the spread of programs to ence career pathgraduate school, provide undergraduates with re-postdoctoral fellowship, tenure-track search experience: Such experience,academic appointment, and finally she said, is now virtually a prerequi-the attainment of independent inves- site for admission to graduate school.tigator statushas been increas- Attendees expressed a range ofingly called into question. Many views on the value of publishing themeeting participants said that the results of undergraduate researchtime has come to adopt a broader projects. Robert Lichter said thatview of a successful science pathway. presenting results for peer review Too many graduate departments and publication is a critical aspect ofare continuing to ignore the realities research. Christopher Womersley,of the job market for Ph.D.'s, said University of Hawaii at Manoa, saidTed Greenwood, Alfred P. Sloan that it is unrealistic to expect allFoundation. In many research fields, undergraduates to get papers pub-the supply of Ph.D.'s vastly exceeds lished and expressed concern thatthe demand. "We must stop produc- emphasizing the publication ofing research-oriented clones of fac- results would put too much pressureulty who are unprepared for the on students. Aaron Ellison noted thatkinds of jobs in which many, if not new journals have sprung up thatmost, of them will end up," he said. are dedicated to publishing under- Student participants made it clear graduate research, but he was skep-that they are not naive about their tical of the value of such journals. prospects for pursuing careers in aca- Several attendees described theirdemic research. "None of us is expecting to come out of a postdoc institutions' programstotrain and have a tenure-track position at a undergraduates in laboratory safety. major research university waiting for Many institutions require students us," said Kathryn Goettge. She added to attend a seminar on safety before that pharmaceutical and biotechnol- they enter a laboratory. At Carnegie ogy companies will likely be the Mellon, students spend the firstmajor employers of her generation of week of their summer programbiological sciences graduates. learning about all aspects of labora- Kristin Baldwin said that her col- tory safety and must pass an exami-leagues at Stanford are considering nation on safety issues beforemany career options, such as work- beginning laboratory work. At theing in industry or investment bank- University of Hawaii at Manoa, stu-ing. According to Keith Amos, dents receive a two-week orientationHarvard medical students are con- that covers safety, laboratory tech-sidering working for regulatory niques, stress management, andagencies or managed-care compa- other topics. nies, starting their own biotechnol-

24 Assessing Science Pathways ogy businesses, or going into ruralfessors (including Sands himself) Careers in primary care. have developed research collabora- biotechnology Several attendees said that col-tionswithindustryscientists. research were leges and universities need to de-Lehigh has initiated a master of sci- discussed by (left to right) velop more extensive links withence program in biotechnology that Shuang Ruy Huang, industry and opportunities for stu-brings together science and engi- Merck Company dents to obtain research experienceneering education. Foundation; Lucybeth Nieves and Lillian Bird, in an industrial setting. Jeffrey Shuang Ruy Huang, Merck University of Puerto Sands said that Lehigh UniversityCompany Foundation, described Rico Rio Piedras has capitalized on its location in eastsome of the educational outreach Campus; Kathryn Goettge, University centralPennsylvaniaclosetoprograms established by Merck, of North Carolina at many pharmaceutical and biotech-which include an undergraduate Chapel Hill; and nology companiesto create a vari-internship program for juniors and Jeffrey Sands, Lehigh ety of interactions with industry. seniors who spend the summer University. As a result of these interactions,working in a Merck laboratory; an HHMI-supported undergraduatesambassador programin which can obtain summer research experi-Merck scientists maintain relation- ences in pharmaceutical companyships with the universities where laboratories, scientists working forthey received their education, visit- pharmaceutical and biotechnologying regularly to lecture and work companies can work toward ad-with faculty members at those insti- vanced degrees with mentoringtutions; and a new mentor program from professors at Lehigh, and pro-in which Merck scientists serve as

39 Panel and Workshop Discussions 25 at local elementary and middle schools; it is hoped that some of these undergraduates will decide to pursue teaching careers. Some institutions are devising programs to surmount students' need for financial support for a postgraduate year of teaching training. Donald GoldbergsaidthatOccidental College participates in a National Science Foundation collaborative teacher preparation program that provides students with scholarships. Several attendees told anecdotes Ken Harewood, career counselors to students atto support their contention that a sci- Florida A&M nearby universities. ence education can be valuable in University, moderated Ken Harewood described Florida a resource group unpredictable ways for people who meeting for directors A & M University's program to placedo not pursue careers in science. At of HHMI-supported students in industry laboratories forone institution a survey of biology undergraduate internships lasting 8-10 weeks. programs at master's and biochemistry graduates' career institutions. Students, faculty, and industry rep-choices found that many students resentatives all benefit from the rela-commented favorably on the value of tionship, he said. Kathleen Parson,their science education even though Macalester College, said that shethey had not pursued careers in sci- had spent a one-year industrial sab-ence. One graduate said, "My under- batical at 3M in Minneapolis andgraduate degree taught me to think, had maintained her relationshipto organize, and to set goals." with the company since that time, Norman Hackerman said that the sending students there on intern-Robert A. Welch Foundation consid- ships and bringing 3M scientists toers the development of scientifically her college as guest lecturers. educated lay persons to be its most Goettge noted that in industry important goal. "Those who don't go scientists tend to work in teams, into research become supporters of whereas in academia they are more research," he said. likelytoworkindependently. Research experience in industry gives students valuable exposure to collaborative science, she said. Focusing on Women Several attendees commented thatand Minorities few biology graduates now enter precollege teaching and wondered howUnderrepresented in more students could be influenced tothe Sciences consider this career option. ThomasImplicit in the notion that all stu- Settlemire, Bowdoin College, anddents should have access to a sci- Alice Grier Lee, Washington andence education is the recognition Jefferson College, described pro-that strategies are needed to recruit grams at their institutions in whichand retain women and members of undergraduates work with studentsminority groups that have tradition-

26 Assessing Science Pathways 40 ally been underrepresented in theand after participating in the four- sciences. week program, and scores on the Ted Greenwood described a pro-post-program test go up significantly, gram, funded by the Alfred P. Sloansaid program director Richard Storey. Foundation forsix years,thatTo date 95 percent of program partic- increased the recruitment and reten-ipants have gone on to college. tion of women in science, mathemat- Colorado College offers a second ics, and engineering. The programresidential summer program for supported model graduate andincoming minority freshmen who undergraduate programs at institu-have indicated an intention to major tions committed to altering their cul-in science. The program covers writ- ture, climate, and (where necessary)ing, mathematics, and computer curriculum to enable women to suc-skills; note-taking and study skills; ceed as readily as men. and test-taking strategies. The earli- Characteristics of funded pro-est participants in this program are grams included vigorous recruit-now in their junior year, said Storey; ment of women students, improvedone measure of the program's suc- mentoring and advising, and effortscess will be how many of them to eliminate sexism in the classroomgraduate. (intentional or unintentional); under- Michael Gaines described a bridg- graduate research opportunities;ing program to encourage talented and curriculum changes that empha-students from Miami-Dade Com- sized collaborative learning, hands-munity College to transfer to the on approaches, and reliance onUniversity of Miami. Features of the meaningful real-world examples. program include a high-school-to- Another Sloan program is in-college transition course that empha- tended to increase the number ofsizes basic skills in chemistry, mathe- minority students receiving Ph.D.'smatics, and writing; twice-weekly in science and engineering. Themathematics and chemistry work- foundation's approach is to seek outshops during the academic year; a faculty members with a provensummer course in biotechnology lab- record of success in producingoratory techniques; and a monthly minority Ph.D.'s and offer themresearch colloquium. additional resources for each addi- Quantitative assessment of the tional minority Ph.D. they produce. program's impact is difficult to do, Several program directorssaid Gaines, because students enter described HHMI-supported pro-the program from unequal starting grams that focus on the recruitmentpoints. "You can take the sure-fire and retention of minorities underrep-people that you know are going to resented in the sciences. Coloradosucceed, but to me that's not as chal- College offers a residential summerlenging as taking the people who bridging program for high school stu-have not had an opportunity because dents that aims to increase partici-of where they went to high school." pants' scientific knowledge, teach the This point was reinforced by principles of field study, and encour-Charles Putman, who said that the age participants to continue their edu-most important lesson he had cation. Students take a writing andlearned from Duke University's analytical skills inventory test beforebroad-spectrum program to attract

Panel and Workshop Discussions 27 41 Shuang Ruy Huang, Merck Company Foundation, Velma Richardson, Tuskegee University, and Verna Case, Davidson College, chatted during a break.

women and minorities into scienceWomersley, is that students choose was "how critical it is for institutionstheir own research projects and to take risks." Students who wouldselect their own mentor by inter- not normally have met Duke'sviewing faculty members. admission requirements had thrived Karen Olmstead described the in the HHMI-supported summerUniversity of South Dakota's out- research program, which had alsoreach program to American Indian served as a stimulus for reform oftribal colleges, which focuses on Duke's biology curriculum. developing research infrastructure The University of Hawaii atfor faculty and students at these often Manoa's program includes two-yeargeographically remote institutions. hands-on internships and travelHHMI support has provided a molec- awards that enable students toular biology facility at one tribal col- obtain research experience on thelege; workshops, release time, and United States mainland and presenttravel funds for faculty development; their findings at scientific meetings.and laptop computers for faculty A key feature of the program, saidmembers who need to travel long dis- programdirectorChristophertances between college campuses.

28 Assessing Science Pathways 412 Several program directors com-theresearchquestionstobe mented on the fact that women great-answered, work collaboratively, and ly outnumber men in undergraduatepresent their results to their peers research programs targetedatfor critical comment. underrepresented minorities. It was Programdirectorspresented suggested that the stipends offeredexamples of how the principles of dis- by such programs may not be highcovery-based science teaching are enough to attract minority menbeing introduced on their campuses. and that minority men may be lessAt the University of CaliforniaLos likely to view biology as a viableAngeles, an inquiry-based approach career option. Ray Gavin, Cityis being incorporated into introduc- University of New York Brooklyntory organic chemistry, and students College, proposed that programs beare encouraged to publish their targeted at underrepresented minori-research on the Internet. Guy ty men just as they are now targetedBradley said that Eckerd College at women. offers a special one-month course in hands-on genetic engineering. The students who learn the most in this course "are those whose experi- Changing How ments don't work the first time." Undergraduate Science The University of Maryland Is Taught College Park has developed three Burgeoning enrollments in under-upper-level undergraduate science graduate biology present collegescourses in which students work in and universities with a dilemma:teams. Teams are selected so that There are simply too many studentsthey include students of both sexes, for all to participate in resource-inten- of different races, and with a range sive undergraduate research pro-of grade point averages. Each team grams. However, research is thewrites a research proposal and each characteristic activity of science andstudent makes an oral presentation. as such, attendees argued, is a pivotalStudents also critique other teams' component of any science curricu-research proposals. The perfor- lum. Christopher Womersley saidmance of average students in these that traditional lecture-based meth-courses has improved, and bright ods of undergraduate science educa-students have learned that they can tion have been comparable withhelp others to perform well. training carpenters in only the theory City University of New York City and not the practice of their trade. College has introduced an under- Julio de Paula, Haverford College,graduate course in the techniques of advocated creating undergraduatemolecular biology. Working in research methods courses and dis-groups, students learn how to run covery-based laboratory coursesconcurrent experiments, budget that provide a research-like environ-their time, and troubleshoot prob- ment while offering more guidance lems that arise. This course also pro- than an independent research expe-videsvaluableexperiencefor rience. Distinguishing features ofstudents who go on to jobs as labo- discovery-based (or inquiry-based)ratory technicians in industry, said courses are that students originateprogram director Sharon Cosloy.

4 3 Panel and Workshop Discussions 29 At the University of Chicago, Nancy Devino saidthat the HHMI support for undergraduateNational Research Council's national programs provided the impetus forscience education standards may, by reinvigoration of the institution'schanging the way precollege science undergraduate biological sciencesis taught, provide an impetus for the program, said program director Joséadoption of the same kinds of Quintans. The university has theapproaches at the undergraduate new Biological Sciences Learninglevel. Students who have learned sci- Center in which all biology educa-ence through hands-on, discovery- tion (for undergraduates, graduatebased approaches in elementary students, and medical students) nowand secondary school will not toler- takes place. Quintans characterizedate didactic lectures when they get the new center as a one-roomto college, she said. schoolhouse where undergraduates Devino and Greenwood recom- mingle not only with graduate stu-mended a report titled Talking About dents and medical students but alsoLeaving,* an inquiry into the reasons with faculty researchers whose labo-for high attrition rates among under- ratories are housed in the researchgraduate science, mathematics, and wing of the building. engineering majors. Among the At some institutions, the impetusreport's findings, said Greenwood, for curriculum reform came fromare that students who drop out of sci- outside in the form of criticism byence are no less capable than those external evaluators. Ray Gavin, Citywho remain and that many students University of New York Brooklyn(both those who leave and those College, said that such criticism hadwho persist) are unhappy with the been a catalyst for wholesale cur-quality of teaching and advising they riculum redevelopment, includingreceive as undergraduates. the introduction of inquiry-based Several program directors de- approaches in biology laboratories. scribed their institutions' experi- Science faculty often fear that,ences with peer-tutoring programs. because inquiry-based teachingAtWashingtonandJefferson approaches are time-intensive, stu-College, freshmen and upper-class dents will lack the content knowl-students work together to design and edge necessary to do well onconduct experiments in a biotechnol- standardized tests such as theogy workshop. At California State Graduate Record Examination andUniversity-Northridge, student tutors the Medical College Admissionprovide supplemental instruction in Test. Kathleen Blits,St. John'sthe more difficult science courses. College, said that such fears may beStudent participation is booming, misplaced; her institution's entiresaidprogramdirectorJoyce science program is predicated on Maxwell. The tutors benefit as much the principle that less is more, and

students do very well on standard- * Seymour, E. and N.M. Hewitt, 1994. Talking ized tests. Some program directors, About Leaving: Factors Contributing to High however, were skeptical that stu- Attrition Rates Among Science, Mathematics, and Engineering Undergraduate Majors. Final dents retain information any better Report to the Alfred P. Sloan Foundation on an in inquiry-based courses than in tra- Ethnographic Inquiry at Seven Institutions, ditional lecture-based courses. Boulder, CO: University of Colorado.

30 Assessing Science Pathways 4 4 Darrel Falk, Point Loma Nazarene College (left), and Anna Tan-Wilson, State University of New York at Binghamton, moderated a program directors' forum.

as the tutees because "there is notutors.Programdirector Alan better way to learn than to teach." Gubanich said that this approach is At the State University of Newvery successful, although the defini- York at Binghamton, upper-leveltion of success is different for each undergraduates participate in an on-student. For example, a student who campus summer program to createobtains a C in mathematics, meeting undergraduate laboratory courses.her personal goal and enabling her During the following semester,to graduate, is considered a success these students work as mentors one as much as the student who discov- on one with other undergraduates. ers new self-confidence in math and Program director Anna Tan-Wilson said that although the goal of thegoes on to medical school. summer program was to provide At many institutions information upper-level undergraduates withtechnology is a catalyst for class- teaching experience, an unexpectedroom innovation.* Electronic corn- additional benefit was that the experience of preparing laboratories got *The role of electronic communication and other computer technologies in the classroom was students excited about research. examined at the 1995 HHMI undergraduate pro- The University of Nevada-Reno gram directors meeting. (New 7bols for Science developed a program in which stu- Education, 1996, Howard Hughes Medical Institute, Office of Grants and Special dents who have difficulty with math- Programs.) Also see the HHMI World Wide Web ematics are trained to be math site at http://www.hhmi.org.

Panel and Workshop Discussions 31 4 5 Scott Santos, University of Buffalo; Robert Cluss, Middlebury College; William Marks, Vilanova University; and Kristin Baldwin, Stanford University, examined the offerings at a materials exchange that ran concurrently NorthCarolinaStateUniver with the meeting.

fm,ix

munication can help to foster a sensetechnology into the classroom will of community in a class, said Earlput students who cannot afford com- Fleck, Whitman College. He hasputers at a disadvantage. He noted used listservs for two years to pro-that many Brooklyn College students vide course material to students, andlive in households without tele- he also encourages students to postphones. Other program directors said questions. "Other students will oftenthat their institutions were respond- answer the questions before I do.ing to this concern by developing on- Sometimes a student who doesn'tcampuscomputerlaboratories. say a word in class will participate inUnless such laboratories are open 24 the electronic network." hours each day, however, many stu- Gavin, however, worried that thedents who have full-lime jobs may still widespread integration of informationlack adequate access to computers.

32 Assessing Science Pathways 46 The responsibility of colleges anddevelopment of interdisciplinary Interdisciplinary universities to prepare students forcourses can be an important incen- collaborations were the workplace by equipping themtive to faculty to overcome tradi- the topic at a workshop session featuring with skills in writing and data analy-tional disciplinary barriers and (left to right) Robert sis, as well as in the use of comput-begin to work together. Julio de Lichter, Camille and ers, was an issue that concernedPaula said that an HHMI-funded Henry Dreyfus both program directors and student Foundation; Frederick effort to develop new introductoryEiserling, University of participants. Keith Amos said that itcourses in biology, chemistry, math, CaliforniaLos Angeles; is crucial for students to learn goodand physics at Haverford College and Dale Benham, writing skills as well as science. Nebraska Wesleyan had facilitated the development of University. Some institutions have implementedrelationships among faculty in those workshops in writing and statisticsdepartments. and increased their report-writing Dale Benham reportedthat requirements to ensure that stu-Nebraska Wesleyan University is dents develop adequate skills ininvesting some of its HHMI funding these areas. in the development of an interdisciplinary, laboratory-based environmental science course for freshman nonscience majors. The course will Encouraging be team-taught by faculty from the Interit'sciplinary geology, chemistry, mathematics, Collaborations and physics departments. Several program directors noted Environmental science was that the availability of funds for theselected because it involves issues-

Panel and Workshop Discussions 33 47 some controversialthat are not only University of CaliforniaLos Angeles: What relevant to Nebraska but also impor- to Assess? tant globally. One of the measures of its impact will be a longitudinal com- UCLA's introductory chemistry and biology courses parison of the career outcomes of have been redesigned using interactive video learn- course alumni with a control group of ing and student teamwork. Assessment strategies students who do not take the course. that have been used include The University of CaliforniaLos strategies for developing and validating assess- Angeles has developed a four- ments: model-based versus ideographic assess- course, self-paced interdisciplinary ment approaches and their trade-offs; life sciences curriculum and a bat- tery of measures to assess student mapping assessment to key characteristics of performance in the new curriculum, laboratory science: content understanding, said program director Frederick problem solving, collaboration or teamwork, Eiserling. Assessment tools include and metacognition (learning to learn); a diagnostic test on course entry to measure how much students already development of quality criteria to judge perfor- know, an electronic portfolio that mance; and students maintain as they progress successful faculty collaboration processes. through the sequence of courses, and a self-assessment test that students complete when they decide they are ready to evaluate their mas-

Mary Lynn Grayeski, Research Corporation (left); Mark Jacobs, Swarthmore College; José Quintans, University of Chicago; and Karen Olmstead, University of South Dakota, discussed fostering faculty development at a workshop session.

34 Assessing Science Pathways 4 8 tery of both theoretical and practical At predominantly undergraduate course material. institutions, expectations are some- Karen Olmstead noted that inter-what different. Teaching is a pri- disciplinary collaborations may bemary criterion for tenure, and junior easier to initiate at small institutionsfaculty members who want to do because faculty members are moreresearch often struggle to find the likely to be acquainted with theirtime and resources to do so, accord- counterparts in other departments. ing to Julio de Paula. Collaborativeinitiativesmay How to achieve a better balance extend beyond other science depart-between teaching and researchon ments and may have unexpectedthe one hand, increasing the value spinoffs,noted James Collins,placed on teaching at research- Arizona State University His institu-focused institutions and, on the tion's invitation to its school of archi-other hand, providing improved tecture to design new scienceopportunities to pursue research at laboratories led architecture facultypredominantly undergraduate insti- and graduate students to investigatetutionswas a recurring theme at how to create space that helps sci-the meeting. ence teaching. As a result, Collins Theinstitutionalcultureof said, the school of architecture hasresearch universities needsto made the design of science laborato-change so that teaching excellence ries a major initiative. is valued as highly as research in the tenure and promotion process, many attendees said. Equally, faculty at predominantly undergradu- Research and Teaching: ateinstitutions needabetter Finding a Balance research infrastructure, including As competition stiffens for bothnot only laboratory space and equip- availableresearchdollarsandment but also sabbatical leaves, trav- tenured faculty positions, junior fac-el funds, and release time to attend ulty members feel the pressurescientific meetings so that they most directly. With the tenure clockremain current in their fields. ticking, they must balance the often Julio de Paula and Mark Jacobs, competing demands of research andSwarthmore College, emphasized teaching while contributing theirthe importance of frequent sabbati- share of service, the third traditionalcal leaves as a faculty development element of an academic career. tool. At Haverford College, tenure- At research-intensive institutions,track faculty can take a sabbatical research accomplishments are per-after three years and another after ceived as paramount, creating atenure is obtained. Swarthmore dilemma for young faculty membersoffers its faculty a sabbatical every who wish to excel at teaching as wellfourth year. as in research. Janine Maddock, Many participants expressed con- University of Michigan-Ann Arbor,cern that, although the teacher- told of being advised by her meritscholar who is actively involved in review committee to focus on herboth teaching and research remains research and put less effort into heran academic ideal, in reality the high- teaching. est status is frequently awarded to

48 Panel and Workshop Discussions 35 Janine Maddock, University of MichiganAnn ArborExcited About Education As a tenure-track faculty member at a major research university, Janine Maddock is under enormous pressure to winin a highly competitive environmentresearch grant support for her genetics laboratory and to publish research results that will strengthen hercase for tenure. Maddock, an assistant professor of biology at the University of MichiganAnn Arbor, is also deeply committed to undergraduate education. She encourages undergraduates to work in her laboratory, welcoming the opportunity to demonstrate the fun as wellas the challenge of science. Her classroom presentations to undergraduates are laced with references to cutting-edge research discoveries. With deadlines looming on two crucial grant applications, she takes two hours out of her crowded schedule to talk about her scientific career to members of the university's undergraduate biology club. Maddock exemplifies the dilemma faced by university science faculty members whoare committed to teaching well but whose professional survival depends on obtaining research support. Balancing the demands of teaching and researchnot to mention administrative responsibilities and family commitmentsis a constant challenge. The focus of Maddock's research is cell fate and the mechanisms of cell cycle control, using the bacterium Caulobacter crescentus as a model. She arrived at the University of Michigan after completing a postdoctoral fellowship with developmental biologist Lucille Shapiro at Stanford University School of Medicine. Like most faculty members who work with undergraduates, Maddock finds that teaching extends beyond the lecture hall. "A lot more goes into the job of teaching than most people ever know," she says. 'We counsel. We mentor. We advise. It's a lot more than just delivering a lecture." Then there are departmental commitments, which for Maddock have included serving on the graduate admissions committee and on the microbiology and cell and molecular biology curriculum committees. The mother of a 10-year-old girl and a 5-year-old boy, Maddock believes that excitement about science is instilled in childhood. She regularly visits a local elementary school to give science presentations to fourth graders. Her contributions to teaching should be weighed equally with her research productivity when she is evaluated for tenure, says Maddock. "I'm excited about education. I want to keep that level of excitement and get tenure." (Adapted from Beyond Bio 101, Howard Hughes Medical Institute, Office of Communications, 1996.)

those who do not teach, and the sta-nontenured lecturer career track for tus of those who focus solely onfaculty who focus on teaching and teaching is devalued. Some research-are not involved in research. intensive institutions, including the In the biological sciences, lectur- University of Chicago and Carnegieers generally work full-time on labor- Mellon University, have introduced aintensive biology laboratory courses.

36 Assessing Science Pathways 50 Swarthmore College: What to Assess?

To track science faculty members as they advance in their careers and to assess the effect of external funding on faculty development, the following categories of each faculty member's teaching and research are periodically examined: I new laboratories teaching new techniques within already existing science courses; new courses offered on previously unexamined topics at the frontier of a field of science; 3 number and area of publications, to assess the importance of the support both for general productivity and for easing access to a new field;

rapublications with student coauthorsresearch articles and abstractsto assess fruitful interaction with college and high school students; and research and educational grants obtained after faculty-development support.

Susan Henry, noting that the propor-that surveys of alumni several years tion of women in lecturer positions atafter graduation might produce more Carnegie Mellon is higher than thethoughtful, and as a result more valu- proportion in tenured or tenure-trackable, assessment data than the typi- positions, wondered whether the lec-cal end-of-course student evaluations. turer track would become a de factoHenry said that Carnegie Mellon is academic ghetto for women. Otherexperimenting with peer evaluation attendees worried that the insti-ofteaching.Severalspeakers tutionalization of a nontenured lectur-observed that although the elements er track exacerbates the separationof good teaching may be difficult to between research and teaching. define precisely, most faculty mem- Several attendees suggested thatbers know who the good teachers are funding agencies could encouragein their own departments. institutions to value teaching more Broadening the definition of by supporting postdoctoral fellow-scholarship to include contributions shiPs that include a teaching compo-to teaching, including developing nent. At present, it was noted, theinnovative courses and bringing best graduate students and postdocseducational technology into the usually have research fellowshipsclassroom,isanother way to that exempt them from teaching. increase the value of teaching. The need to develop better criteriaJournals published by a number of to define good teaching was anothereducational professional associa- recurring theme. It was suggestedtions publish articles about innova-

51 Panel and Workshop Discussions 37 tions in teaching. However, at someulty who are under pressure to pro- institutions, publication in such jour-duce publishable results may fear nals may be valued less highly thanthat having undergraduates in the publication in research journals. laboratorywill impedetheir Elizabeth Duffy described a pro-progress. Although research grants gram that the Andrew W Mellonoften include funds for faculty Foundation is funding at selected lib-salaries and overhead expenses, eral arts colleges to support the useagencies that support undergradu- of educational technology in teach-ate education programs usually do ing. She said that the foundation isnot pay for these costs. using its funding to encourage insti- Some institutions are finding tutions to recognize the efforts ofways to provide incentives. Henry faculty, especially junior faculty, whosaid that Carnegie Mellon is allocat- devote time to this kind of curricu-ing part of its HHMI grant to pro- lum development. "We have beenvide financial support to junior very explicit about telling institu-faculty who work with undergradu- tions that we don't want to fund aates in their laboratories. Bradley project that is going to jeopardize theMcPherson said that Centenary assistant professors who work on it."College of Louisiana offers summer Several attendees commentedresearch support to faculty mem- that faculty members become morebers who apply for but do not excited about teaching when theyreceive external research grants. are also engaged in research. Abolition of the mandatory retire- However, many participants felt thatment age for faculty means that better incentives are needed forinstitutions must reconsider the role junior faculty to become involved inof tenure, said Henry. She warned undergraduate research. Junior fac-that this issue will be decided by

Tracking faculty -r-- careers was the topic of a plenary session featuring (from left to right) Susan Henry, Carnegie Mellon University; Janine Maddock, University of MichiganAnn Arbor; Julio de Paula, Haverford College; and Elizabeth Duffy, Andrew W. Mellon Foundation.

38 Assessing Science Pathways 52 society and politicians unless acade- Mary Colvard, a high school biol- mia takes some action. ogy teacher in upstate New York Henry, who is dean of sciences atand a participant in the Cornell Carnegie Mellon University, andInstituteofBiology Teachers Frank Vellaccio, who is vice provost(CIBT) network, said that profes- of the College of the Holy Cross, sional isolation is a primary cause of stressed that facultynot adminis-teacher attrition. Bacchetti said tratorsmake the rules that governthere is evidence that attrition the tenure and promotion process.diminishessignificantlywhen "If the faculty decide that they wantschool districts make a commitment to change the reward structure, it'sto in-service education. The Hewlett as good as done," said Henry. Foundation supports programs that encourage good candidates to enter and remain in teaching as well as programs that improve teacher training and promote systemic Outreach to Precollege change in the education system. Teachers CIBT overcomes teachers' isola- Another finding of the Pathways fortion by linking them to a network of Women in the Sciences study ofcolleagues and Cornell University Wellesley graduates is that interestfaculty, said Colvard. She and Peter in pursuing science or mathematicsBruns, Cornell program director, as a career is probably developedsaid that the program is effective for before college. Precollege sciencetwo reasons: ongoing assessment education lays the foundation thatand a strong partnership between enables a student to succeed in sci-the teachers and the faculty partici- ence as an undergraduate. Meetingpants. For example, new classroom participants heard from severallaboratory materials are developed HHMI grantees who have developedjointly by teachers and faculty work- innovative collaborative programs to ing in teams. The materials are then enhance the preparation and profes-tested in classrooms for a year. sional development of precollege sci- Components that do not prove use- ence teachers. ful are eliminated. Other forms of ongoing assess- Programs for Inservice Teachers Teaching is often a lonely career,ment include extensive use of ques- said Raymond Bacchetti, Williamtionnaires and teacher interviews as and Flora Hewlett Foundation. Hewell as having teachers serve as quoted Judith Warren-Little of theinstructors at the annual summer University of CaliforniaBerkeley:institutes that provide an opportunity "Schools are collections of smallfor teachers to familiarize themselves businesses called classrooms." In awith recent research advances. multimedia presentation by FrankBruns advised other program direc- Vellaccio, a public school teacher intors involved in teacher development Worcester, Massachusetts, Dermotprograms to form partnerships with Shea, described a typical school daytheir institutions' schools of educa- as consisting of "cells and bellsthetion to develop effective assessment bell rings and you go into your cell."strategies.

53 Panel and Workshop Discussions 39 During a plenary session on assessing science partnerships

(from left), Norman __J Hackerman, Welch Foundation; Sandra Mims, Pasadena Unified School District; Jerome Pine, California Institute of Technology; Mary Co Ivard, Cobleskill- Richmondville High School; and Peter / Brtms, Cornell University, discuss science outreach for teachers.

Outreach activities in the Cornellperformance assessment, and use of program include return-to-campusstudent portfolios. lectures and regional meetings at Another program benefit that is which Cornell faculty members aregreatly appreciated by teachers, said invited to lecture. Program exten-Colvard, is the opportunity to obtain sion associates travel around thesecond-hand equipment when uni- state to conduct workshops forversity laboratories upgrade their teachers; each workshop participantfacilities. "Your trash is our trea- is charged with presenting the work-sure," she said. shop twice to other groups of teach- Vellaccio and Barbara Swidler, an ers, thus disseminating the materialselementary' school mathematics and information more widely. teacher in Worcester, Massachu- The CIBTnet communicationssetts, described the partnership pro- network facilitates ongoing contactgram developed by the College of among teachers and between teach-the Holy Cross and the Worcester ers and Cornell faculty. Such ongo-public schools. The key program ing contact is essential for achievingelements are a summer institute for systemic, sustainable change, saidteachers and one-year sabbaticals Colvard.Tangibleresultsarefor selected teachers. already evident: New York teachers The Holy Cross and Cornell pro- have developed a new hands-on,grams are both based on the constructivist, and inquiry-basedassumption that teachers can best sciencecurriculumandhavejudge the kind of professional devel- designed an examination blueprintopment training they need. The ulti- that incorporates essay questions,mate goals of both are to increase

40 Assessing Science Pathways students' science literacy and enthu-concept well, or is it necessary to siasm for science and to encouragecover everything?" she asked. Other more students to seek further edu-speakers noted that precollege teach- cationorcareersinscience.ers need a broad, but not necessarily However,these outcomesaredeep, understanding of scientific con- extremely difficult to measure at thecepts and that college-level science local level, said Vellaccio, for rea-courses geared to their needs are dif- sons that include student mobility,ficult to find. the need for a long time frame, and Mark Jacobs, Swarthmore Col- the influence of other uncontrolledlege, pointed out a need to provide variables on student learning. financial incentives for faculty to Teacher outcomessuch as sub-become involved in summer insti- ject matter knowledge, self confi-tutes for precollege teachers. High dence, and theability to useschool teachers are paid to attend a hands-on approaches, collaborativesummer institute at Swarthmore, he learning techniques, and educa-said, because the college success- tional technologyare measured byfully argued that teachers need to a variety of means, including inter-earn money during the summer to views and monitoring of partici-support their families. However, the pants'classroomteachingtoSwarthmore professors who work observe the degree to which theywith the teachers are unpaid. have integrated the newly learned approaches into their instructionalPrograms for Preservice Teachers style. Classroom time spent teach-Studies have shown that elementary ing science can also be measured.school teachers tend to avoid sci- 'We are confident that if we get theence, spending little classroom time desired teacher outcomes, they willon the subject, and that students lead to the desired student out-begin to lose interest in science by comes," said Vellaccio. the fourth grade. Such findings Teachers on sabbatical werepromptedtheUniversity of replaced in the classroom by recentNebraska-Lincoln to focus its precol- Holy Cross science graduates wholege outreach efforts on revising sci- were not certified teachers. Thisence instruction for preservice approach would not have been per-elementary education majors. mitted in many other states, which George Veomett, program direc- require a certified teacher to be intor, and Cindy Mahler, a student the classroom at all times. Onemajoring in elementary education at unexpected benefit of the program,the University of NebraskaLincoln, said Vellaccio, was that some of thedescribed the characteristics of the uncertified student teachers werefour hands-on science courses (in inspired to obtain their certificationbiology, chemistry, physics, and and pursue teaching careers. earth science) developed so far: an Sarah Elgin commented that intro-emphasis on both process and con- ducing teachers to a hands-on,tent; a classroom environment that inquiry-based approach to scienceemphasizes success; use of coopera- teaching can conflict with the impera-tive learning groups; and multiple tive to cover a large amount of mater-bases for student assessment, ial. "Is it better to teach one scientificincluding test scores, self-assess-

Panel and Workshop Discussions 41 Frank Vellaccio (left), and Barbara Swidler, College of the Holy Cross; Raymond Bacchetti, William and Flora Hewlett Foundation; and Cindy Mahler and George Veomett, University of NebraskaLincoln, participated in a workshop discussion on assessing science teacher training.

ments, and reports from student-Claremont Graduate School in Los teacher supervisors. Angeles. Assessment data are available for The course, which was based on the hands-on biology course, whichcooperative learning techniques and was developed first and has beenclass discussion and included no lec- taken by the greatest number of stu-tures, was team-taught by a master dents. In addition to assessing con-elementary school teacher and a tent knowledge, the program hasmember of Caltech's science faculty. also endeavored to assess student-Students were graded on the basis teachers' comfort level and enthusi-of projects, homework, science note- asm about teaching science. Thebooks, and class participation. Two data suggest that the course hastrial runs of the course at Caltech positively affected preservice teach-enabled refinements to be made ers' attitudes toward science andbefore the course was more widely confidence in their ability to teachdisseminated. science, said Veomett. CaltechemployedInverness Jerome Pine, California InstituteResearch Associates, a firm of proof Technology(Caltech),andfessional evaluators, to assess the Sandra Mims, an elementary schoolcourse. The evaluators conducted teacher in the Pasadena Unifiedprecourse interviews with course School District, described a preser-developers and students, organized vice teacher-training program inclassroom observations and focus biology and physics that was devel-groups, interviewed students during oped collaboratively by teachers,the course, and conducted follow-up Caltech scientists, and faculty of theinterviews with both students and

42 Assessing Science Pathways 56 California Institute of Technology: What to Assess?

The two topics in the preservice science coursein physics (sound) and biology (plants)are taught in a laboratory through experimen- tation and discussion. Among the issues assessed are Ithe effectiveness of the broad-based collaborative effort, the response of preservice elementary teachers to thecourse, the subject matter knowledge gained by students, the acceptance of the course for trials by other universities, the success of transfer to other faculty members, and the effect of the course on teachers after graduation.

faculty. A simple, inexpensive ques-sibility by turning over science tion to ask in program assessment,teacher education to the schools of Norman Hackerman, said Pine, is What did I want toeducation. That connection needs to Robert A. Welch Foundation, spoke at a achieve with this program that Ibe re-established, said Lytle. workshop session on didn't? Caltech is planning to do a Norman Hackerman noted thatbridging programs. follow-up study of 12 students whoTexas had abolished the bachelor's took the preservice course and 12 who did not. The hands-on,collaborative- learning approaches being used in these programs, both in-service and preservice, are in line with the recommendations of the National Research Council's national science education standards, noted Nancy Devino.CharlesLytle,North Carolina State University, said that the adoption of such approaches leads to profound changes in student attitudes and class participation. He said that the most dramatic improvements are often seen in students who previously seemed the least interested in science. Where responsibility lies for the r- education of precollege science teachers was a recurring topic of .21411a discussion. Some attendees con- 4." tendedthat universityscience departments have abdicated respon-

57 Panel and Workshop Discussions 43 Joseph Perpich (left) moderated a plenary review session featuring Susan Henry, Carnegie Mellon University; Mary Allen, Wellesley College; and Peter Bruns, Cornell University.

degree in education in 1989 in anclearly define their goals and objec- effort to reassert the primacy of sub-tives and think about how they ject-matter degrees. As a result, themight measure the achievement of University of Texas at Austin hasthose intentions. recentlyreintroducedteaching Science educators are still strug- degrees in chemistry, physics, biolo-gling to devise appropriate assess- gy, and geology. ment toolstomeasuretheir programs' achievements. The most valuable information may come from long-term assessment, but Summary such assessment is a challenge Assessment is an important man-becauseoftheresourcesit agement tool, not only to documentrequires. Program managers can success but also to enable ongoingbenefit from collaborations with program adjustment and improve-cognitive and social scientists, sta- ment. The strongest programs aretisticians, and education specialists those that continually assess andwho have expertise in assessment adjust. To be most effective, assess-methodologies. ment must be built in when pro- A consequence of science educa- grams are designed. In this waytion reform is the recognition that program managers are required totraditional methods of evaluating

44 Assessing Science Pathways 58 the performance of students, teach-you know that what you are doing ers, and faculty members are nomakes a difference? The program longer adequate. Because the goals,directors meeting spotlighted the objectives, and target audiences of science education programs aremany creative ways in which HHMI- extremely diverse, no one-size-fits-supported institutions are respond- all technique exists for definitivelying to change and endeavoring to answering the question How doassess the results of their efforts.

Panel and Workshop Discussions 45 59 Appendices

I Appendices 49A: Agenda, Program Directors Meeting, September 30October 2, 1996

51B: Attendees, Program Directors Meeting, September 30October 2, 1996

58C: Undergraduate Biological Sciences Education Program-1988 Awards

59D: Undergraduate Biological Sciences Education Program-1989 Awards

61E: Undergraduate Biological Sciences Education Program-1991 Awards

62F: Undergraduate Biological Sciences Education Program-1992 Awards

63G: Undergraduate Biological Sciences Education Program-1993 Awards

64H: Undergraduate Biological Sciences Education Program-1994 Awards

66I: Undergraduate Biological Sciences Education Program-1996 Awards

67J: Awardee Institutions by Carnegie Classification, 1989-1996

70K: Awardee Minority Institutions, 1989-1996

71 L: Awardee Institutions by State, 1989-1996

74Grants Publications

_6 1 Appendix A

Agenda, Program Directors Meeting, September 30October 2, 1996

Monday, September 30, 1996

Welcoming Remarks HHMI Overview and Introduction of Keynote Speaker Purnell Choppin Keynote Address Daniel Koshland

Tuesday, October 1, 1996

Session I

Plenary Panel Discussion: Baccalaureate and Beyond: Tracking the Science Pathway Burroughs Wellcome FundMartha Peck Duke University/Stanford UniversityKristin Baldwin Wellesley CollegeMary Allen, Moderator, and Christine Birchwood Xavier University of Louisiana/Harvard University School of MedicineKeith Amos

Workshop Demonstrations and Discussions Group 1Taking a Different Route: Careers in Biotechnology Research Lehigh UniversityJeffrey Sands and Kathryn Goettge Merck Company FoundationShuang Ruy Huang University of Puerto Rico Rio Piedras CampusLillian Bird, Moderator, and Lucybeth Nieves Group 2Preparing Students for Research Careers Colgate UniversityNancy Pruitt and Tara Gupta Alfred P. Sloan FoundationTed Greenwood University of HawaiiChristopher Womersley, Moderator, and Scott Santos Group 3Bridging Programs Colorado CollegeRichard Storey University of MiamiMichael Gaines, Moderator, and Beatriz Blanco Robert A. Welch FoundationNorman Hackerman

Session II

Plenary Panel Discussion: Assessing Science Partnerships California Institute of TechnologyJerome Pine and Sandra Mims Cornell UniversityPeter Bruns, Moderator, and Mary Co Ivard Robert A. Welch FoundationNorman Hackerman

Resource Group Meetings Master's Colleges and Universities Florida A&M UniversityKen Harewood, Moderator California State UniversityNorthridgeJoyce Maxwell, Co-Moderator Baccalaureate Colleges Kenyon CollegeJoan Slonczewski, Moderator Occidental CollegeDonald Goldberg, Co-Moderator Research and Doctoral Universities University of Texas at AustinMary Ann Rankin, Moderator Tufts UniversityRoss Feldberg, Co-Moderator

62 Agenda, Program Directors Meeting 49 Program Directors Forum Point Loma Nazarene CollegeDarrel Falk, Moderator SUNY at BinghamtonAnna Tan-Wilson, Co-Moderator

Wednesday, October 2, 1996

Session III

Plenary Panel Discussion: Tracking Faculty Careers Carnegie Mellon UniversitySusan Henry, Moderator Haverford CollegeJulio de Paula Andrew W. Mellon FoundationElizabeth Duffy University of MichiganAnn ArborJanine Maddock Workshop Demonstrations and Discussions Group 1Fostering Faculty Development Research CorporationMary Grayeski Swarthmore CollegeMark Jacobs University of ChicagoJosé Quintans, Moderator University of South DakotaKaren Olmstead Group 2Assessing Laboratory Development/Interdisciplinary Collaborations Duke UniversityCharles Putman Camille and Henry Dreyfus FoundationRobert Lichter Nebraska Wesleyan UniversityDale Benham University of CaliforniaLos AngelesFrederick Eiserling, Moderator Group 3Assessing Science Teacher Training College of the Holy CrossFrank Vellaccio, Moderator, and Barbara Swidler William and Flora Hewlett FoundationRaymond Bacchetti University of NebraskaLincolnGeorge Veomett and Cindy Mahler

Plenary Review Summary of Key Issues Mary Allen, Peter Bruns, Susan Henry, and Joseph G. Perpich

50 Appendix A 6 3 Appendix B

Attendees, Program Directors Meeting, September 30October 2, 1996

Georgiana F. Aboko-Cole, Ph.D. Christine Birchwood Director Student Center for Pre-Professional Education Department of Biological Sciences Howard University Wellesley College

Mary Mennes Allen, Ph.D. Lillian Bird, Ph.D. Director of Biological Chemistry and Chairman and Professor Jean Glasscock Professor Department of Chemistry Wellesley College University of Puerto Rico Rio Piedras Campus G. Samuel Alspach, Jr., Ph.D. Chair and Professor Beatriz Blanco Department of Biology Student Western Maryland College Department of Biology University of Miami Keith Amos Student Kathleen C. Blits, Ph.D. Harvard University Medical School Tutor St. John's College Raymond Bacchetti, Ph.D. Program Officer Edward M. Bonder, Ph.D. Department of Education Associate Professor of Cell Biology William and Flora Hewlett Foundation Department of Biological Sciences Rutgers the State University of New Claudia F. Bailey, Ph.D. Jersey Newark Campus Vice Chair and Associate Professor of Biological Sciences Guy Bradley, Ph.D. Department of Biological Sciences Assistant Professor of Molecular University of Arkansas Physiology Department of Biology Kristin Baldwin Student Eckerd College Department of Immunology Stanford University Joan B. Broderick, Ph.D. Assistant Professor Ted Bartlett, Ph.D. Department of Chemistry Professor Amherst College Department of Chemistry Fort Lewis College Peter J. Bruns, Ph.D. Director John Benditt Division of Biological Sciences Editor, Science's Next Wave Cornell University Science Magazine David K. Burnham, Ph.D. Dale Benham, Ph.D. Associate Professor of Microbiology Associate Professor of Biology and Department of Microbiology and Chair, Department of Biology Molecular Genetics Nebraska Wesleyan University Oklahoma State University

Attendees, Program Directors Meeting 51 6 4 Verna M. Case, Ph.D. Julio C. de Paula, Ph.D. Professor of Biology and Chair Associate Professor of Chemistry Biology Department Chemistry Department Davidson College Haverford College

John Chant, Ph.D. Nancy Devino, Ph.D. Assistant Professor Study Director Molecular and Cellular Biology Committee on Undergraduate Science Department Education Harvard University National Research Council

Mildred Chaparro, Ph.D. Eneida Diaz-Pérez, Ph.D. Professor Associate Professor Department of Biology Biology Department University of Puerto Rico Mayaguez University of Puerto Rico Campus Cayey University College

José Miguel Cimadevilla, Ph.D. Jacqueline Dorrance Professor Foundation Administrator Department of Biological Sciences Arnold and Mabel Beckman Foundation St. Mary's University Elizabeth A. Duffy, M.B.A., M.A. Robert G. Cluss, Ph.D. Program and Research Associate Associate Professor Administrative and Financial Officer Department of Chemistry and The Andrew W. Mellon Foundation Biochemistry Middlebury College Frederick A. Eiserling, Ph.D. Dean, Life Sciences James P. Collins, Ph.D. College of Letters and Science Professor and Chair University of CaliforniaLos Angeles Department of Zoology Arizona State University Sarah C. R. Elgin, Ph.D. Mary P. Co lvard, M.S.Ed. Professor Science Teacher Department of Biology Biology Department Washington University Cobleskill-Richmondville High School Aaron M. Ellison, Ph.D. Sharon Cosloy, Ph.D. Marjorie Fisher Associate Professor of Professor and Chair Environmental Studies Department of Biology Department of Biological Sciences City University of New York City College Mount Holyoke College

Jeffrey D. Cross, Ph.D. Kay Etheridge, Ph.D. Professor of Psychology Associate Professor of Biology Department of Psychology Department of Biology Allegheny College Gettysburg College

Benedict T. DeCicco, Ph.D. Paul F. Fahey, Ph.D. Professor Dean, College of Arts and Sciences Department of Biology Professor of Physics Catholic University of America University of Scranton

52 Appendix B 6 5 Darrel R. Falk, Ph.D. Donald Y. Goldberg, Ph.D., J.D. Professor and Chair Associate Dean of Faculty for the Department of Biology Sciences Point Loma Nazarene College Occidental College

Ross S. Feldberg, Ph.D. Julie Graf, M.Ed. Associate Professor of Biology Hughes Initiative Program Director Biology Department Office of Academic Affairs Tufts University University of Colorado at Boulder

Earl W Fleck, Ph.D. Mary Lynn Grayeski, Ph.D. Professor and Chair Program Officer Biology Department Science Advancement Program Whitman College Research Corporation

Gary R. Francois, Ph.D. Ted Greenwood, Ph.D. Program Officer Professor of Psychology Alfred P. Sloan Foundation Department of Psychology Knox College Sandra Gregerman, M.S. Director William H. F'uchsman, Ph.D. Undergraduate Research Opportunity Professor of Chemistry Program Department of Chemistry University of MichiganAnn Arbor Oberlin College

Alan A. Gubanich, Ph.D. Michael S. Gaines, Ph.D. Associate Professor Professor and Chair Department of Biology Department of Biology University of Nevada, Reno University of Miami Tara H. Gupta Raymond E. Garcia, Ph.D. Student / Intern Professor of Biochemistry National Institutes of Health Department of Chemistry and Biochemistry Norman Hackerman, Ph.D. California State UniversityLos Angeles Chairman Scientific Advisory Board Ray H. Gavin, Ph.D. Robert A. Welch Foundation Professor and Chairman Department of Biology Prudence J. Hall, Ph.D. City University of New YorkBrooklyn Associate Professor of Biology and College Department Chair Department of Biology Kathryn L. Goettge Hiram College Graduate Student Department of Microbiology and Ken R. Harewood, Ph.D. Immunology Professor of Biology University of North Carolina at Chapel Department of Biology Hill Florida A&M University

66 Attendees, Program Directors Meeting 53 William H. Harvey, Ph.D. Herb Levitan, Ph.D. Professor and Chair Section Head Biology Department Course and Curriculum Development Earlham College Program Division of Undergraduate Education Carla Ann Hass, Ph.D. National Science Foundation Instructor Department of Biology Robert L. Lichter, Ph.D. Pennsylvania State University Executive Director Camille and Henry Dreyfus Foundation Susan A. Henry, Ph.D. Professor, Biological Sciences, and Thomas S. Litwin, Ph.D. Dean, Mellon College of Science Director of the Clark Science Center Carnegie Mellon University and Adjunct Associate Professor of Shuang R. Huang, Ph.D. Biological Sciences Vice President Smith College The Merck Company Foundation Charles F. Lytle, Ph.D. Mark Jacobs, Ph.D. Coordinator of Biology Outreach Associate Provost and Professor of Biological Sciences Department Biology North Carolina State University Department of Biology Swarthmore College Janine Maddock, Ph.D. Assistant Professor Ivan Johnson, Ph.D. Department of Biology Professor of Biology University of MichiganAnn Arbor Department of Biology Concordia CollegeMoorhead Cindy Mahler Student Larry Hudson Jones, Ph.D. Department of Biological Sciences Associate Dean of the College and University of NebraskaLincoln Professor of Biology Department of Biology Shyamal K. Majumdar, Ph.D. University of the South Professor and Department Head Department of Biology Rebecca Jones Lafayette College Program Officer Abell Foundation R. William Marks, Ph.D. Associate Professor Daniel E. Koshland Jr., Ph.D. Department of Biology Professor Emeritus Villanova University Department of Molecular and Cell Biology University of CaliforniaBerkeley Karl R. Mattox, Ph.D. Dean, College of Arts and Science Deidre D. Labat, Ph.D. Professor of Botany Dean, College of Arts and Sciences Miami University Xavier University of Louisiana Joyce B. Maxwell, Ph.D. Alice Grier Lee, Ph.D. Professor of Biology Associate Professor of Biology School of Science and Mathematics Washington & Jefferson College California State UniversityNorthridge

54 Appendix B 6 7 A. Bradley McPherson, Ph.D. Kathleen A. Parson, Ph.D. Mary Warters Professor of Biology Academic Dean and Professor of Biology Centenary College of Louisiana and Chemistry Macalester College Bain D. Mehrotra, Ph.D. Professor of Chemistry Martha Peck, M.Sc. Department of Chemistry Vice President, Programs Tougaloo College Burroughs Wellcome Fund

John C. Mickus, Ph.D. William J. Perreault, Ph.D. Professor of Biology Department of Biological Sciences Professor of Biology Benedictine University Department of Biology Lawrence University Sandra Mims, M.A. Science Resource Teacher Jerome Pine, Ph.D. Pasadena Science Program Professor of Biophysics Pasadena Unified School District Department of Physics California Institute of Technology Virinder K. Moudgil, Ph.D. Professor and Chair Clifton Poodry, Ph.D. Department of Biological Sciences Director Oakland University Division of Minority Opportunities in Research Lucybeth Nieves National Institute of General Medical Student Sciences Biology Department National Institutes of Health University of Puerto Rico Rio Piedras Campus Nancy L. Pruitt, Ph.D. Anthony J. Olek, Ph.D. Associate Professor of Biology Director of the Undergraduate Program Colgate University in Biology and Medicine Department of Biology Charles E. Putman, M.D. College of Arts and Sciences Executive Vice President for Research University of Rochester Administration and Policy James B. Duke Professor of Radiology Karen L. Olmstead, Ph.D. and Professor of Medicine Associate Professor and Associate Chair Duke University Department of Biology University of South Dakota José Quintans, M.D., Ph.D. Master and Associate Dean Judith Owen, Ph.D. Biological Sciences Collegiate Division Professor and Chairperson University of Chicago Department of Biology Haverford College Mary Ann Rankin, Ph.D. Paul J. Paolini, Ph.D. Dean, College of Natural Sciences and Associate Dean and Professor of Biology Professor, Department of Zoology College of Science Division of Natural Sciences San Diego State University University of Texas at Austin

Attendees, Program Directors Meeting 55 Velma B. Richardson, Ph.D. Peter F. Small, Ph.D. Associate Professor Associate Dean of the College Biology Department Ursinus College Tuskegee University Gerhard Sonnert, Ph.D. Exyie C. Ryder, Ph.D. Associate Professor Professor of Biology Department of Physics Southern University Agricultural and Harvard University Mechanical College at Baton Rouge John G. Stevens, Ph.D. Jeffrey A. Sands, Ph.D. National Executive Officer Professor and Chairperson of Molecular Council on Undergraduate Research Biology Department of Biological Sciences Richard D. Storey, Ph.D. Lehigh University Professor and Chair of Biology Department of Biology Scott R. Santos Colorado College Student Department of Biological Sciences Barbara Swidler, M.Ed. State University of New York at Buffalo Math Instructor Department of Mathematics Roger H. Sawyer, Ph.D. Worcester Public Schools Professor and Chair Department of Biological Science Anna Tan-Wilson, Ph.D. University of South CarolinaColumbia Professor Department of Biological Sciences C. Thomas Settlemire, Ph.D. State University of New York at Associate Professor Binghamton Department of Biology Bowdoin College Bruce R. Telzer, Ph.D. Professor of Biology Peter R. Shank, Ph.D. Pomona College Associate Dean of Medicine and Biological Sciences (Research) Michael F. Tibbetts, Ph.D. Brown University Assistant Professor Department of Biology Harold Silverman, Ph.D. Bard College Associate Dean, College of Basic Sciences, and John J. Tudor, Ph.D. Professor of Zoology and Physiology Professor and Chair Louisiana State University and A&M Department of Biology College Saint Joseph's University

Helene Slessarev, Ph.D. Frank Vellaccio, Ph.D. Director Provost Urban Studies Department Office of the Provost Wheaton College College of the Holy Cross

Joan L. Slonczewski, Ph.D. George E. Veomett, Ph.D. Associate Professor of Biology Associate Professor Biology Department School of Biological Sciences Kenyon College University of NebraskaLincoln

56 Appendix B 6 9 Graham C. Walker, Ph.D. Christopher Z. Womersley, Ph.D. Professor Professor of Zoology Department of Biology Department of Zoology Massachusetts Institute of Technology University of Hawaii at Manoa

John E. Walsh, Sc.D. Terry Woodin, Ph.D. Associate Dean of the Faculty for the Program Director Sciences and Francis and Mildred Division of Undergraduate Education Sears Professor of Physics National Science Foundation Department of Physics and Astronomy Dartmouth College Robert J. Wyman, Ph.D. Professor of Biology Gail L. Waring, Ph.D. Biology Department Professor of Biology Yale University Department of Biology Marquette University Jill Yager, Ph.D Chair, Biology Department John D. Weete, Ph.D. Professor and Associate Dean Associate Professor of Environmental College of Science and Mathematics Science Auburn University Antioch University

Mary E. McKelvey Welch, Ph.D. Robert Yuan, Ph.D. Professor and Chair Professor Department of Biology Department of Microbiology Fisk University University of Maryland

Attendees, Program Directors Meeting 57 7 0 Appendix C

Undergraduate Biological Sciences EducationProgram-1988 Awards

Allegheny College Millsaps College Meadville, Pennsylvania $900,000 Jackson, Mississippi 600,000 Amherst College Morehouse College Amherst, Massachusetts 500,000 Atlanta, Georgia 400,000 Bryn Mawr College Morgan State University Bryn Mawr, Pennsylvania 800,000 Baltimore, Maryland 400,000 Bucknell University Mount Holyoke College Lewisburg, Pennsylvania 700,000 South Hadley, Massachusetts 400,000 Carleton College Oakwood College Northfield, Minnesota 800,000 Huntsville, Alabama 400,000 Clark Atlanta University Oberlin College Atlanta, Georgia 700,000 Oberlin, Ohio 900,000 College of Wooster Occidental College Wooster, Ohio 1,000,000 Los Angeles, California 500,000 Colorado College Pomona College Colorado Springs, Colorado 800,000 Claremont, California 500,000 Davidson College Rhodes College Davidson, North Carolina 700,000 Memphis, Tennessee 700,000 DePauw University Saint Olaf College Greencastle, Indiana 600,000 Northfield, Minnesota Dillard University 600,000 Smith College New Orleans, Louisiana 400,000 Northampton, Massachusetts Earlham College 400,000 Swarthmore College Richmond, Indiana 750,000 Swarthmore, Pennsylvania Franklin & Marshall College 900,000 Tuskegee University Lancaster, Pennsylvania 400,000 Tuskegee, Alabama Grinnell College 900,000 Union College Grinnell, Iowa 400,000 Schenectady, New York Hampshire College 900,000 Vassar College Amherst, Massachusetts 400,000 Poughkeepsie, New York Hampton University 700,000 Wabash College Hampton, Virginia 600,000 Haverford College Crawfordsville, Indiana 900,000 Haverford, Pennsylvania 1,200,000 Wake Forest University Howard University Winston-Salem, North Carolina 800,000 Washington, D.0 900,000 Wellesley College Kenyon College Wellesley, Massachusetts 750,000 Gambier, Ohio 500,000 Wesleyan University Lincoln University Middletown, Connecticut 1,200,000 Lincoln University, Pennsylvania 700,000 Whitman College Macalester College Walla Walla, Washington 400,000 Saint Paul, Minnesota 800,000 Xavier University of Louisiana Marlboro College New Orleans, Louisiana 1,800,000 Marlboro, Vermont 400,000 Middlebury College Middlebury, Vermont 400,000 Total $30,400,000

58 Appendix C El Undergraduate Program-1988 Awards 71 Appendix D

Undergraduate Biological Sciences Education Program-1989 Awards

Brown University Rensselaer Polytechnic Institute Providence, Rhode Island $1,000,000 Troy, New York 1,000,000 Carnegie Mellon University Rice University Pittsburgh, Pennsylvania 1,200,000 Houston, Texas 1,000,000 Case Western Reserve University Stanford University Cleveland, Ohio 1,200,000 Stanford, California 1,000,000 City University of New York City College Stevens Institute of Technology New York, New York 1,400,000 Hoboken, New Jersey 1,000,000 College of William and Mary Tufts University Williamsburg, Virginia 1,000,000 Medford, Massachusetts 1,500,000 Colorado State University University of Arizona Fort Collins, Colorado 1,500,000 Tucson, Arizona 1,500,000 Columbia University University of CaliforniaDavis New York, New York 1,000,000 Davis, California 1,200,000 Cornell University University of CaliforniaIrvine Ithaca, New York 2,000,000 Irvine, California 1,200,000 Dartmouth College University of CaliforniaSan Diego Hanover, New Hampshire 1,000,000 San Diego, California 1,200,000 Duke University University of CaliforniaSanta Cruz Durham, North Carolina 1,500,000 Santa Cruz, California 1,000,000 Emory University University of Chicago Atlanta, Georgia 1,000,000 Chicago, Illinois 2,000,000 Indiana University at Bloomington University of Colorado Bloomington, Indiana 2,000,000 Colorado Springs, Colorado 2,000,000 Johns Hopkins University University of Illinois at Chicago Baltimore, Maryland 1,000,000 Chicago, Illinois 1,000,000 Lehigh University University of Illinois at UrbanaChampaign Bethlehem, Pennsylvania 1,000,000 Urbana, Illinois 1,500,000 Louisiana State University and A&M College University of Kansas Main Campus Baton Rouge, Louisiana 1,000,000 Lawrence, Kansas 1,000,000 Massachusetts Institute of Technology University of MinnesotaTwin Cities Cambridge, Massachusetts 1,000,000 Minneapolis, Minnesota 1,000,000 Miami University University of MissouriColumbia Oxford, Ohio 1,000,000 Columbia, Missouri 1,000,000 New York University University of North Carolina at Chapel Hill New York, New York 1,200,000 Chapel Hill, North Carolina 1,500,000 Ohio State University University of Pennsylvania Columbus, Ohio 1,200,000 Philadelphia, Pennsylvania 1,200,000 Pennsylvania State University Main Campus University of Puerto Rico Rio Piedras Campus University Park, Pennsylvania 1,200,000 Rio Piedras, Puerto Rico 1,200,000 Princeton University University of Southern California Princeton, New Jersey 1,000,000 Los Angeles, California 1,000,000 Purdue University Main Campus University of Texas at Austin West Lafayette, Indiana 1,000,000 Austin, Texas 1,000,000

Arena( D n Undergraduate Program-1989 Awards 59 University of Utah University of WisconsinMadison Salt Lake City, Utah 1,200,000 Madison, Wisconsin 1,200,000 University of Vermont Wayne State University Burlington, Vermont 1,000,000 Detroit, Michigan 1,200,000 University of Virginia Yale University Charlottesville, Virginia 1,000,000 New Haven, Connecticut 1,000,000 University of Washington Seattle, Washington 1,000,000 Total $61,000,000

60 Appendix D II Undergraduate Program-1989 Awards Appendix E

Undergraduate Biological Sciences Education Program-1991 Awards

Antioch University Hope College Yellow Springs, Ohio $600,000 Holland, Michigan 750,000 Barnard College Illinois Benedictine College New York, New York 900,000 Lisle, Illinois 500,000 Bates College Jackson State University Lewiston, Maine 900,000 Jackson, Mississippi 700,000 Beloit College Juniata College Beloit, Wisconsin 650,000 Huntingdon, Pennsylvania 900,000 California State University-Long Beach King College Long Beach, California 750,000 Bristol, Tennessee 500,000 California State University-Los Angeles Knox College 800,000 Los Angeles, California Galesburg, Illinois 500,000 Calvin College Lafayette College Grand Rapids, Michigan 500,000 Easton, Pennsylvania 600,000 Canisius College Lawrence University Buffalo, New York 700,000 Appleton, Wisconsin 700,000 Centre College Danville, Kentucky 500,000 Manhattan College Riverdale, New York 500,000 City University of New York Brooklyn College Brooklyn, New York 1,200,000 Nebraska Wesleyan University Lincoln, Nebraska 500,000 City University of New York Queens College Flushing, New York 1,100,000 Ohio Wesleyan University Delaware, Ohio 700,000 Colby College Waterville, Maine 1,000,000 Reed College Colgate University Portland, Oregon 1,000,000 Hamilton, New York 900,000 San Diego State University College of the Holy Cross San Diego, California 1,200,000 Worcester, Massachusetts 600,000 Spelman College Cooper Union Atlanta, Georgia 900,000 New York, New York 750,000 University of Puerto Rico Mayaguez Campus Eckerd College Mayaguez, Puerto Rico 500,000 St. Petersburg, Florida 550,000 University of Scranton Fisk University Scranton, Pennsylvania 700,000 700,000 Nashville, Tennessee University of Texas at San Antonio Fort Lewis College San Antonio, Texas 650,000 Durango, Colorado 800,000 University of the South Gettysburg College Sewanee, Tennessee 500,000 Gettysburg, Pennsylvania 850,000 Ursinus College Goucher College Collegeville, Pennsylvania 500,000 Towson, Maryland 500,000 Villanova University Hamilton College Villanova, Pennsylvania 650,000 Clinton, New York 700,000 Williams College Harvey Mudd College Williamstown, Massachusetts 900,000 Claremont, California 500,000 Hiram College Hiram, Ohio 700,000 Total $31,500,000

Appendix E 0 Undergraduate Program-I991 Awards 61 \? 4 Appendix F

Undergraduate Biological Sciences Education Program-1992 Awards

Arizona State University University of Delaware Tempe, Arizona $1,500,000 Newark, Delaware 1,000,000 Auburn University University of Georgia Auburn University, Alabama 1,000,000 Athens, Georgia 1,400,000 Boston University University of Hawaii at Manoa Boston, Massachusetts 1,000,000 Manoa, Hawaii 1,000,000 Brandeis University University of Iowa Waltham, Massachusetts 1,400,000 Iowa City, Iowa 1,100,000 California Institute of Technolov University of Kentucky Pasadena, California 2,000,000 Lexington, Kentucky 1,000,000 Georgetown University University of Maryland College Park Washington, D.0 1,000,000 College Park, Maryland 1,300,000 Harvard University Cambridge, Massachusetts 1,100,000 University of Massachusetts at Amherst Amherst, Massachusetts Illinois Institute of Technology 1,200,000 Chicago, Illinois 1,000,000 University of MichiganAnn Arbor Iowa State University Ann Arbor, Michigan 1,400,000 Ames, Iowa 1,300,000 University of NebraskaLincoln Kansas State University Lincoln, Nebraska 1,000,000 Manhattan, Kansas 1,200,000 University of Nevada, Reno Marquette University Reno, Nevada 1,100,000 Milwaukee, Wisconsin 1,200,000 University of New Mexico Michigan State University Albuquerque, New Mexico 1,000,000 East Lansing, Michigan 1,200,000 University of Notre Dame North Carolina State University Notre Dame, Indiana 1,500,000 Raleigh, North Carolina 1,000,000 University of Oregon Oklahoma State University Eugene, Oregon 1,000,000 Stillwater, Oklahoma 2,000,000 University of Pittsburgh, Pittsburgh Campus Rutgers the State University of New Jersey Pittsburgh, Pennsylvania 1,700,000 Newark Campus, Newark, New Jersey 1,200,000 University of Rochester State University of New York at Albany Rochester, New York 1,000,000 Albany, New York 1,200,000 University of South CarolinaColumbia State University of New York at Binghamton Columbia, South Carolina 1,000,000 Binghamton, New York 1,500,000 Vanderbilt University Texas Tech University Nashville, Tennessee 1,200,000 Lubbock, Texas 1,300,000 University of CaliforniaBerkeley Washington State University Pullman, Washington Berkeley, California 2,000,000 1,500,000 University of CaliforniaLos Angeles Washington University Los Angeles, California 1,300,000 Saint Louis, Missouri 1,700,000 University of CaliforniaSanta Barbara West Virginia University Santa Barbara, California 1,000,000 Morgantown, West Virginia 1,000,000 University of Cincinnati Cincinnati, Ohio 1,000,000 Total $52,500,000

62 Appendix F 0 Undergraduate Program-1992 Awards c'l7 5 Appendix G

Undergraduate Biological Sciences Education Program-1993 Awards

Amherst College Ohio Wesleyan University Amherst, Massachusetts $500,000 Delaware, Ohio 500,000 Bates College Saint Joseph's University Lewiston, Maine 500,000 Philadelphia, Pennsylvania 500,000 Bowdoin College St. Mary's University Brunswick, Maine 550,000 San Antonio, Texas 650,000 Bryn Mawr College Saint Olaf College Bryn Mawr, Pennsylvania 600,000 Northfield, Minnesota 500,000 California State University-Los Angeles San Diego State University Los Angeles, California 650,000 San Diego, California 500,000 California State University-Northridge Smith College Northridge, California 700,000 Northampton, Massachusetts 600,000 Carleton College Southern University and A&M College at Baton Rouge Northfield, Minnesota 850,000 Baton Rouge, Louisiana 700,000 City University of New York City College Swarthmore College New York, New York 650,000 Swarthmore, Pennsylvania 650,000 City University of New York Herbert H. Lehman College Tougaloo College Bronx, New York 500,000 Tougaloo, Mississippi 500,000 City University of New York Hunter College Tuskegee University New York, New York 650,000 Tuskegee, Alabama 500,000 Clark Atlanta University Union College Atlanta, Georgia 550,000 Schenectady, New York 500,000 College of the Holy Cross University of Puerto Rico Cayey University College Worcester, Massachusetts 550,000 Cayey, Puerto Rico 600,000 Colorado College University of Puerto Rico Mayaguez Campus Colorado Springs, Colorado 650,000 Mayaguez, Puerto Rico 500,000 Concordia College-Moorhead University of Puerto Rico Rio Piedras Campus Moorhead, Minnesota 550,000 Rio Piedras, Puerto Rico 500,000 Fisk University University of Texas at El Paso Nashville, Tennessee 550,000 El Paso, Texas 850,000 Fort Lewis College Ursinus College Durango, Colorado 500,000 Collegeville, Pennsylvania 500,000 Gettysburg College Wellesley College Gettysburg, Pennsylvania 500,000 Wellesley, Massachusetts 850,000 Hampton University Wesleyan University Hampton, Virginia 500,000 Middletown, Connecticut 500,000 Haverford College Western Maryland College Haverford, Pennsylvania 600,000 Westminster, Maryland 500,000 Hobart and William Smith Colleges Wheaton College Geneva, New York 550,000 Wheaton, Illinois 550,000 Humboldt State University Williams College Arcata, California 800,000 Williamstown, Massachusetts 500,000 Knox College Wofford College Galesburg, Illinois 500,000 Spartanburg, South Carolina 700,000 Morehouse College Xavier University of Louisiana Atlanta, Georgia 1,000,000 New Orleans, Louisiana 1,400,000 Oberlin College Oberlin, Ohio 500,000 Total $28,500,000

Appendix G D Undergraduate Program-1993 Awards 63 t7 6 Appendix H

, Undergraduate Biological Sciences Education Program-1994 Awards

Arizona State University Oregon State University Tempe, Arizona $1,000,000 Corvallis, Oregon 2,000,000 Auburn University Pennsylvania State University Main Campus Auburn University, Alabama 1,000,000 University Park, Pennsylvania 1,000,000 Boston University Princeton University Boston, Massachusetts 1,200,000 Princeton, New Jersey 1,300,000 Brown University Rice University Providence, Rhode Island 1,500,000 Houston, Texas 1,600,000 California Institute of Technology Rutgers the State University of New Jersey Pasadena, California 1,200,000 New Brunswick Campus, New Brunswick, Carnegie Mellon University New Jersey 1,400,000 Pittsburgh, Pennsylvania 1,500,000 Stanford University Case Western Reserve University Stanford, California 1,800,000 Cleveland, Ohio 2,000,000 State University of New York at Binghamton Binghamton, New York Catholic University of America 1,000,000 Washington, D.0 1,200,000 State University of New York at Buffalo Buffalo, New York Cornell University 1,500,000 Ithaca, New York 2,000,000 State University of New York at Stony Brook Stony Brook, New York Dartmouth College 1,000,000 Hanover, New Hampshire 1,800,000 Temple University Philadelphia, Pennsylvania 1,200,000 Duke University Texas A&M University Durham, North Carolina 1,800,000 College Station, Texas 1,700,000 Emory University Texas Tech University Atlanta, Georgia 1,200,000 Lubbock, Texas 1,000,000 Georgia Institute of Technology Tufts University Atlanta, Georgia 1,000,000 Medford, Massachusetts 1,800,000 Harvard University University of Arizona Massachusetts 1,000,000 Tucson, Arizona 2,000,000 Howard University University of Arkansas Main Campus Washington, D.0 1,800,000 Fayetteville, Arkansas 1,000,000 Johns Hopkins University University of CaliforniaDavis Baltimore, Maryland 1,800,000 Davis, California 2,000,000 Lehigh University University of CaliforniaSan Diego Bethlehem, Pennsylvania 1,500,000 San Diego, California 1,100,000 Louisiana State University and A&M College University of Chicago Baton Rouge, Louisiana 1,200,000 Chicago, Illinois 1,800,000 Massachusetts Institute of Technology University of Colorado at Boulder Cambridge, Massachusetts 1,600,000 Boulder, Colorado 1,800,000 Miami University University of Georgia Oxford, Ohio 1,000,000 Athens, Georgia 1,000,000 Mississippi College University of Illinois at UrbanaChampaign Clinton, Mississippi 1,000,000 Urbana, Illinois 1,200,000 New York University University of Iowa New York, New York 1,300,000 Iowa City, Iowa 1,000,000 Oklahoma State University Main Campus University of Massachusetts at Amherst Stillwater, Oklahoma 1,000,000 Amherst, Massachusetts 1,000,000

64 Appendix H 0 Undergraduate Program-1994 Awards University of Miami University of Washington Coral Gables, Florida 2,000,000 Seattle, Washington 1,800,000 University of MissouriColumbia University of WisconsinMadison 1,300,000 Columbia, Missouri Madison, Wisconsin 1,000,000 University of NebraskaLincoln Utah State University Lincoln, Nebraska 1,000,000 Logan, Utah 1,400,000 University of New Orleans New Orleans, Louisiana 1,500,000 Vanderbilt University University of North Carolina at Chapel Hill Nashville, Tennessee 1,000,000 Chapel Hill, North Carolina 2,000,000 Washington University University of North Dakota Main Campus Saint Louis, Missouri 1,400,000 1,600,000 Grand Forks, North Dakota Wayne State University University of South Dakota Detroit, Michigan 1,100,000 Vermillion, South Dakota 1,100,000 Yale University University of Tennessee New Haven, Connecticut 1,600,000 Knoxville, Tennessee 1,400,000 University of Texas at Austin Austin, Texas 1,000,000 Total $86,000,000

Appendix H 0 Undergraduate Program-1994 Awards 65 Appendix I

Undergraduate Biological Sciences Education Program-1996 Awards

Allegheny College Middlebury College Meadville, Pennsylvania $600,000 Middlebury, Vermont 650,000 Bard College Mount Holyoke College Annandale-on-Hudson, New York 600,000 South Hadley, Massachusetts 900,000 Barnard College Nebraska Wesleyan University New York, New York 1,100,000 Lincoln, Nebraska 1,200,000 Bates College Oakland University Lewiston, Maine 600,000 Rochester, Michigan 600,000 Beloit College Oberlin College Beloit, Wisconsin 1,200,000 Oberlin, Ohio 600,000 Benedictine University Occidental College Lisle, Illinois 600,000 Los Angeles, California 800,000 Bryn Mawr College Ohio Wesleyan University Delaware, Ohio Bryn Mawr, Pennsylvania 1,000,000 650,000 Canisius College Point Loma Nazarene College San Diego, California 750,000 Buffalo, New York 650,000 Pomona College Carleton College Claremont, California 900,000 Northfield, Minnesota 650,000 Reed College Centenary College of Louisiana Portland, Oregon 700,000 Shreveport, Louisiana 600,000 St. John's College City University of New York Brooklyn College Annapolis, Maryland 1,000,000 Brooklyn, New York 1,200,000 St. Mary's University City University of New York Queens College San Antonio, Texas 600,000 Flushing, New York 600,000 Saint Olaf College Colby College Northfield, Minnesota 1,300,000 Waterville, Maine 1,000,000 Smith College Colgate University Northampton, Massachusetts 1,600,000 Hamilton, New York 650,000 Spelman College College of the Holy Cross Atlanta, Georgia 800,000 Worcester, Massachusetts 1,100,000 Swarthmore College Colorado College Swarthmore, Pennsylvania 1,200,000 Colorado Springs, Colorado 750,000 Tuskegee University Davidson College Thskegee, Alabama 600,000 Davidson, North Carolina 650,000 University of Texas at San Antonio Earlham College San Antonio, Texas 750,000 Richmond, Indiana 600,000 Villanova University Florida A&M University Villanova, Pennsylvania 1,600,000 Tallahassee, Florida 700,000 Washington and Jefferson College Hampshire College Washington, Pennsylvania 600,000 Amherst, Massachusetts 1,300,000 Wellesley College Haverford College Wellesley, Massachusetts 1,500,000 Haverford, Pennsylvania 750,000 Wesleyan University Hope College Middletown, Connecticut 750,000 Holland, Michigan 700,000 Western Maryland College Humboldt State University Westminster, Maryland 700,000 Arcata, California 650,000 Whitman College Kenyon College Walla Walla, Washington 600,000 Gambier, Ohio 1,500,000 Williams College, Lawrence University Williamstown, Massachusetts 900,000 Appleton, Wisconsin 800,000 Xavier University of Louisiana Macalester College New Orleans, Louisiana 1,600,000 Saint Paul, Minnesota 1,000,000 Total $45,400,000

66 Appendix ILi Undergraduate Program-1996 Awards 79 Appendix J

Undergraduate Biological Sciences Education Program Awardee Institutions by Carnegie Classification, 1992-1996*

Doctoral Universities II or Doctorate- Granting Universities II (1992-1994 Carnegie Classifications competitions): These institutions offer a The Carnegie Foundationforthefull range of baccalaureate programs and Advancement of Teaching classifies col-are committed to graduate education leges and universities on the basis ofthrough the doctorate. They award annu- such factors as the range of the bac-ally at least 10 doctoral degreesin calaureate program, number of Ph.D.three or more disciplinesor 20 or degrees awarded annually, and amount more doctoral degrees in one or more of annual federal support for research disciplines. and development, as appropriate. TheMaster's Colleges and Universities I Institute's assessments of institutions for(1996 competition) or Comprehensive the 1992-1994 competitions were basedUniversities and Colleges I (1993 com- on the 1987 Carnegie Foundation classi- petition): These institutions offer a full fications and, for the 1996 competition, range of baccalaureate programs and are on the 1994 classifications.* The follow- committedtograduateeducation ing classifications and categorical defini-through the master's degree. They award tions were used for the 1992-199640 or more master's degrees annually in competitions, which included both pub-three or more disciplines. lic and private institutions: Master's Colleges and Universities II Research Universities I (1992-1994 (1996 competition) or Comprehensive competitions): These institutions offer aUniversities and Colleges II (1993 com- full range of baccalaureate programs, are petition): These institutions offer a full committedtograduateeducation range of baccalaureate programs and are through the doctorate, and give high pri- committedtograduateeducation ority to research. They award 50 or more through the master's degree. They award doctoral degrees each year. In addition, 20 or more master's degrees annually in they receive annually $40 million orone or more disciplines. more in federal support. Baccalaureate Colleges I (1996 compe- Research Universities II (1992-1994tition) or Liberal Arts Colleges I (1993 competitions): These institutions offer acompetition): These institutions are pri- full range of baccalaureate programs, are marily undergraduate colleges with committedtograduateeducationmajor emphasis on baccalaureate degree through the doctorate, and give high pri- programs. They award 40 percent or ority to research. They award 50 or more more of their baccalaureate degrees in doctoral degrees each year. In addition,liberal arts fields and are restrictive in they receive annually between $15.5 mil- admissions. lion and $40 million in federal support. Baccalaureate Colleges II (1996 com- Doctoral Universities I or Doctorate-petition) or Liberal Arts Colleges II Granting Universities I ( 1992-1994 (1993 competition): These institutions competitions): These institutions offer aare primarily undergraduate colleges full range of baccalaureate programs and with major emphasis on baccalaureate are committed to graduate educationdegree programs. They award less than through the doctorate. They award at40 percent of their baccalaureate least 40 doctoral degrees annually in fivedegrees in liberal arts fields or are less or more disciplines. restrictive in admissions.

*Further information may be found in Classification of Institutions of Higher Education, Carnegie Foundation for the Advancement of Thaching, Princeton, N.J., 1994

8 appendixJ El Awardee Institutions by Carnegie Classification 67 University of Maryland College Park Research Universities I University of Massachusetts at Amherst Arizona State University University of Miami Boston University University of MichiganAnn Arbor Brown University University of MissouriColumbia California Institute of Technology University of NebraskaLincoln Carnegie Mellon University University of New Mexico Main Campus Case Western Reserve University University of North Carolina at Chapel Cornell University Hill Duke University University of Pittsburgh, Pittsburgh Emory University Campus Georgia Institute of Technology University of Rochester Georgetown University University of Tennessee, Knoxville Harvard University University of Texas at Austin Howard University University of Washington Iowa State University University of WisconsinMadison Johns Hopkins University Utah State University Louisiana State University and A&M Vanderbilt University College Washington University Massachusetts Institute of Technology Wayne State University Michigan State University West Virginia University New York University Yale University North Carolina State University Oregon State University Research Universities II Pennsylvania State University Main Auburn University Campus Brandeis University Princeton University Kansas State University Rutgers the State University of New Lehigh University Jersey New Brunswick Campus Oklahoma State University Main Campus Stanford University Rice University State University of New York at Albany State University of New York at Albany State University of New York at Stony University of Arkansas Main Campus Brook University of Delaware Temple University University of Notre Dame Texas A&M University University of Oregon Tufts University University of South CarolinaColumbia University of Arizona Washington State University University of CaliforniaBerkeley University of CaliforniaDavis Doctoral Universities I University of CaliforniaLos Angeles Catholic University of America University of CaliforniaSan Diego Clark Atlanta University University of CaliforniaSanta Barbara Illinois Institute of Technology University of Chicago Marquette University University of Cincinnati Main Campus Miami University University of Colorado at Boulder State University of New York at University of Georgia Binghamton University of Hawaii at Manoa University of Illinois at Urbana Doctoral Universities II Champaign Dartmouth College University of Iowa San Diego State University University of Kansas Main Campus Rutgers the State University of University of Kentucky New Jersey Newark Campus

68 Appendix J Awardee Institutions by Carnegie Classification University of Nevada, Reno Beloit College University of New Orleans Bowdoin College University of North Dakota Main Campus Bryn Mawr College University of Puerto Rico Rio Piedras Carleton College Campus Colby College University of South Dakota Colgate University College of the Holy Cross Master's Universities and Colleges I Colorado College Benedictine University Davidson College California State UniversityLos Angeles Earlham College California State UniversityNorthridge Gettysburg College Canisius College Hampshire College Centenary College of Louisiana Haverford College Hobart and William Smith Colleges City University of New York Brooklyn Hope College College Kenyon College City University of New York City College Knox College City University of New York Herbert H. Lawrence University Lehman College Macalester College City University of New York Hunter Middlebury College College Morehouse College City University of New York Queens Mount Holyoke College College Nebraska Wesleyan University Concordia CollegeMoorhead Oberlin College Florida A&M University Occidental College Hampton University Ohio Wesleyan University Humboldt State University Pomona College Mississippi College Reed College Oakland University St. John's College Saint Joseph's University Saint Olaf College St. Mary's University Smith College Southern University and A&M College at Spelman College Baton Rouge Swarthmore College Tuskegee University Union College University of Puerto Rico Mayaguez University of Puerto Rico Campus Cayey University College University of Texas at El Paso Ursinus College University of Texas at San Antonio Washington and Jefferson College Villanova University Wellesley College Xavier University of Louisiana Wesleyan University Western Maryland College Master's Universities and Colleges II Wheaton College Point Loma Nazarene College Whitman College Williams College Baccalaureate Colleges I Wofford College Allegheny College Amherst College Baccalaureate Colleges II Bard College Fisk University Barnard College Fort Lewis College Bates College Tougaloo College

Appendix J 0 Awardee Institutions by Carnegie Classification 69 Appendix K

Undergraduate Biological Sciences Education Program Awardee Minority Institutions, 1993-1996

In the assessment of institutions for the 1993 and 1996 undergraduate grantsHistorically Black Institutions competitions, the Institute has takenClark Atlanta University into account the institutions' records of Fisk University graduating in the sciences students from Florida A&M University minority groups underrepresented in sci- Hampton University entific fields. Information for these Howard University assessments has been provided by the Morehouse College following sources: Southern University and A&M College at The Minority Access to Research Baton Rouge Careers ProgramoftheNational Spelman College Institutes of Health. (This program was Tougaloo College created in 1977 by the National Institute Tuskegee University of General Medical Sciences to increase Xavier University of Louisiana the number of biomedical scientists from minority groups.) The Minority Biomedical ResearchInstitutions with Significant Support ProgramoftheNationalUnderrepresented Minority Student Institutes of Health. (This program was Presence in the Sciences established in 1972 by the NIH Division California State UniversityLos Angeles of Research Resources to develop minor- City University of New York Brooklyn ity student, faculty, and institutional College involvement in biomedical research.) City University of New York City College The National Association for Equal City University of New York Herbert H. Opportunity in Higher Education. (This Lehman College organization, founded in 1969, repre- City University of New York Hunter sents and serves some 117 historically College and predominantly black colleges and Fort Lewis College universities.) St. Mary's University The Office of Civil Rights of the U.S. University of Puerto Rico Department of Education. (This federal agency is responsible for analyzing and Cayey University College disseminating data on minority students University of Puerto Rico Mayaguez at the nation's colleges and universities, Campus including the number of degrees con- University of Puerto Rico Rio Piedras ferred,assubmitted through the Campus Integrated Post-Secondary Education University of Texas at El Paso System and required of all institutions.) University of Texas at San Antonio

70 Appendix K 0 Awardee Minority Institutions 83 Appendix L

Undergraduate Biological Sciences Education Program Awardee Institutions by State, 1992-1996

Alabama Georgia Auburn University, Auburn University Clark Atlanta University, Atlanta Tuskegee University, Tuskegee Emory University, Atlanta Arizona Georgia Institute of Technology, Athens Arizona State University, Tempe Morehouse College, Atlanta University of Arizona, Tucson Spelman College, Atlanta University of Georgia, Athens Arkansas University of Arkansas Main Campus, Fayetteville Hawaii University of Hawaii at Manoa, Honolulu California California Institute of Technology, fflinois Pasadena California State UniversityLos Angeles Benedictine University, Lisle California State UniversityNorthridge Illinois Institute of Technology, Chicago Humboldt State University, Arcata Knox College, Galesburg Occidental College, Los Angeles Point Loma Nazarene, San Diego University of Chicago, Chicago Pomona College, Claremont University of Illinois at Urbana- San Diego State University, San Diego Champaign, Urbana Stanford University, Stanford University of CaliforniaBerkeley Wheaton College, Wheaton University of CaliforniaDavis University of CaliforniaLos Angeles Indiana University of CaliforniaSan Diego, La Jolla Earlham College University of CaliforniaSanta Barbara University of Notre Dame, Notre Dame

Colorado Iowa Colorado College, Colorado Springs Iowa State University, Ames Fort Lewis College, Durango University of Colorado at Boulder University of Iowa, Iowa City

Connecticut Kansas Wesleyan University, Middletown Kansas State University, Manhattan Yale University, New Haven

Delaware Kentucky University of Delaware, Newark University of Kentucky, Lexington

District of Columbia Louisiana Catholic University of America, Washington, D.C. Centenary College of Louisiana, Georgetown University, Washington, D.C. Shreveport Howard University, Washington, D.C. Louisiana State University and A&M College, Baton Rouge Florida Florida A&M University, Tallahassee Southern University and A&M College at University of Miami, Coral Gables Baton Rouge

.e84 Appendix L El Awardee Institutions by State 71 University of New Orleans Xavier University of Louisiana, New Nevada Orleans University of Nevada, Reno

Maine New Hampshire Bates College, Lewiston Dartmouth College, Hanover Bowdoin College, Brunswick Colby College, Waterville New Jersey Princeton University, Princeton Maryland Rutgers the State University of New Johns Hopkins University, Baltimore Jersey Newark Campus St. John's College, Annapolis Rutgers the State University of New University of Maryland, College Park Jersey New Brunswick Campus Western Maryland College, Westminster New Mexico Massachusetts University of New Mexico Main Campus, Amherst College, Amherst Albuquerque Boston University, Boston Brandeis University, Waltham New York College of the Holy Cross, Worcester Bard College, Annandale-on-Hudson Hampshire College, Amherst Harvard University, Cambridge Barnard College, New York City Massachusetts Institute of Technology, Canisius College, Buffalo Cambridge City University of New York Brooklyn Mount Holyoke College, South Hadley College Smith College, Northampton City University of New York City College Tufts University, Medford City University of New York Herbert H. University of Massachusetts at Amherst Lehman College Wellesley College, Wellesley City University of New York Hunter Williams College, Williamstown College City University of New York Queens Michigan College Hope College, Holland Colgate University, Hamilton Michigan State University, East Lansing Cornell University, Ithaca Oakland University, Rochester Hobart and William Smith Colleges, University of MichiganAnn Arbor Geneva Wayne State University, Detroit New York University, New York City State University of New York at Albany Minnesota State University of New York at Carleton College, Northfield Binghamton Concordia CollegeMoorhead State University of New York at Buffalo Macalester College, St. Paul State University of New York at Stony Saint Olaf College, Northfield Brook Union College, Schenectady Mississippi University of Rochester, Rochester Mississippi College, Clinton Tougaloo College, Tougaloo North Carolina Davidson College, Davidson Missouri Duke University, Durham University of MissouriColumbia North Carolina State University, Raleigh Washington University, Saint Louis University of North Carolina at Chapel Hill Nebraska Nebraska Wesleyan University, Lincoln North Dakota University of NebraskaLincoln University of North Dakota, Grand Forks

72 Appendix L El Awardee Institutions by State 85 Ohio Tennessee Case Western Reserve University, Fisk University, Nashville Cleveland University of Tennessee, Knoxville Kenyon College, Gambier Vanderbilt University, Nashville Miami University, Oxford Oberlin College, Oberlin Texas Ohio Wesleyan University, Delaware University of Cincinnati Main Campus, Rice University, Houston Cincinnati St. Mary's University, San Antonio Texas A&M University, College Station Texas Tech University, Lubbock Oklahoma University of Texas at Austin Oklahoma State University Main University of Texas at El Paso Campus, Stillwater University of Texas at San Antonio

Oregon Oregon State University, Corvallis Utah Reed College, Portland Utah State University, Logan University of Oregon, Eugene Vermont Pennsylvania Middlebury College, Middlebury Allegheny College, Meadville Bryn Mawr College, Bryn Mawr Virginia Carnegie Mellon University, Pittsburgh Hampton University, Hampton Gettysburg College, Gettysburg Haverford College, Haverford Lehigh University, Bethlehem Washington Pennsylvania State University Main University of Washington, Seattle Campus, University Park Washington State University, Pullman Saint Joseph's University, Philadelphia Whitman College, Walla Walla Swarthmore College, Swarthmore Temple University, Philadelphia West Virginia University of Pittsburgh, Pittsburgh West Virginia University, Morgantown Campus Ursinus College, Collegeville Villanova University, Villanova Wisconsin Washington and Jefferson College, Beloit College, Beloit Washington Lawrence University, Appleton Marquette University, Milwaukee Rhode Island University of WisconsinMadison Brown University, Providence Puerto Rico South Carolina University of Puerto Rico Cayey University of South CarolinaColumbia University College, Cayey Wofford College, Spartanburg University of Puerto Rico Mayaguez Campus South Dakota University of Puerto Rico Rio Piedras University of South Dakota, Vermillion Campus

Appendix L L Awardee Institutions by State 73 e* 86 Grants Publications

Information for Predoctoral Fellows and General Publications Meeting the Challenges of Science Fellowship Institutions (annual) Education Reform, 1996 Grants for Science Education (annual) Information for Physician Postdoctoral International Program Community Partnerships in Science Fellows and Fellowship Institutions Scienffic Meeting of International Education, Washington, D.C., (annual) Research Scholars from the Baltics, Metropolitan Area Precollege Science Undergraduate Biological Sciences Central Europe, and the Former Soviet Education Initiatives (annual) Education Program Union, Program, Abstracts, and Fact Sheet on HHMI Grants Programs Information for Colleges and Directory 1996 (program and technological orienta- Universities Awarded Undergraduate Local tions) (annual) Grants (annual) Student and Teacher Internship Program Holiday Lectures Brochures (annual) Precollege and Public Science Education Meeting of the Montgomery County Da Vinci and Darwin in the Molecules of Program Public Schools Student and Teacher Life, Stephen K. Burley, M.D., D.Phil., Information for Science Museums and Interns at the National Institutes of and John Kuriyan, Ph.D., 1993 Biomedical Research Institutions Health (brochure, annual) Genes, Gender, and Genetic Disorders, Awarded Precollege Grants (biennial) Shirley M. Tilghman, Ph.D., and Robert Research Resources for Medical Schools Program Directories L. Nussbaum, M.D., 1994 Program Graduate Science Education Program The Double Life of RNA, Thomas R. Cech, Research Resources for Medical Schools Directory of Predoctoral Fellows (pending) Ph.D., 1995 Program (biennial) Directory of Medical Student Fellows The Immune SystemFriend and Foe, International Program (pending) John W. Kappler, Ph.D., and Philippa Information for International Research Directory of Physician Postdoctoral Marrack, Ph.D., 1996 Scholars and Grantee Institutions Fellows (pending) (biennial) Undergraduate Biological Sciences Program Announcements Education Program Graduate Science Education Program Meetings of Grantees Undergraduate Program Directory Predoctoral Fellowships in Biological Graduate Education Fellows Meetings (periodic) Sciences (annual) Meeting of Medical Student Fellows, Precollege Science Education Program Research Training Fellowships for Program and Abstracts (annual) Precollege Science Education Program Medical Students (annual) Meeting of Predoctoral and Physician Directory (periodic) Postdoctoral Research Fellowships for Postdoctoral Fellows, Program and Research Resources for Medical Schools Physicians (annual) Abstracts (annual) Program Undergraduate Biological Sciences Undergraduate Program Directors Research Resources Program Directory Education Program Meetings (periodic) Undergraduate Biological Sciences Attracting Students to Science: International Program Education Program (biennial) Undergraduate and Precollege International Research Scholars Program Precollege and Public Science Education Programs, 1992 Directory (periodic) (See Meetings of Program Enriching the Undergraduate Grantees, International Program) Precollege Science Education Program Laboratory Experience, 1993 Many of these publications have been (biennial) Institutional Strategies for Enhancing redesigned and edited for use on the Research Resources for Medical Schools Undergraduate Science Education, Grants Web pages. The Institute's home Program 1993 page can be accessed at Research Resources for Medical Schools Science Education: Expanding the Role http://www.hhmi.org Program of Science Departments, 1994 International Program New Mots for Science Education, 1995 Publications on the HHMI Grants International Program (biennial) Precollege Science Education Program Program may be requested from Precollege Program Directors Meetings Howard Hughes Medical Institute Information Booklets Science Museums: Creating Partnerships Office of Grants and Special Programs (available only to grantees) in Science Education, 1994 4000 Jones Bridge Road Graduate Science Education Program Science Museums: Enlisting Communities Chevy Chase, MD 20815-6789 Information for Medical Student Fellows in Science Education Partnerships, (fax) 301-215-8888 and Fellowship Institutions (annual) 1995 (e-mail) [email protected]

74 Grants Publications 8 7 Howard Hughes Medical Institute

8. e James A. Baker, 111, Esq. Garnett L. Keith Trustees Senior Partner Chairman Baker & Botts Sea Bridge Investment Advisors Former Vice President and Chief Alexander G. Hearn, M.D. Investment Officer Adjunct Professor Prudential Insurance Company The Rockefeller University Professor Emeritus of Medicine William R. Lummis, Esq. Cornell University Medical College Former Chairman of the Board of Former Senior Vice President Directors and Chief Executive Officer Merck Sharp & Dohme, International The Howard Hughes Corporation

Frank William Gay Irving S. Shapiro, Esq., Chairman Former President and Chief Executive Of Counsel Officer Skadden, Arps, Slate, Meagher & Flom SUMMA Corporation Former Chairman and Chief Executive Officer James H. Gilliam Jr., Esq. E.I. du Pont de Nemours and Company Executive Vice President Beneficial Corporation George W Thorn, M.D., Chairman Emeritus Hanna H. Gray, Ph.D. Professor Emeritus President Emeritus and Professor of the Harvard Medical School Department of History and the College The University of Chicago

89 Howard Hughes Medical Institute 77 Purnell W. Choppin, M.D. Officers President

W Maxwell Cowan, M.D., Ph.D. Vice President and Chief Scientific Officer

Joan S. Leonard, Esq. Vice President and General Counsel

Joseph G. Perpich, M.D., J.D. Vice President for Grants and Special Programs

C. F. Wolfe Vice President and Chief Investment Officer

78 Howard Hughes Medical Institute 9 0 Craig A. Alexander, Esq. Dennis WC. Liu, Ph.D. Principal Staff Deputy General Counsel Grants Program Officer Members Stephen A. Barkanic Margaret Feczko May, Esq. Grants Program Officer Associate General Counsel

W Emmett Barkley, Ph.D. Robert H. McGhee Director of Laboratory Safety Director of Research Facilities Planning

Lillian H. Blucher Alan E. Mowbray Managing DirectorInvestments Director of Management Services

David A. Clayton, Ph.D. Robert C. Mullins Senior Scientific Officer Director of Internal Audit

Winfred J. Clingenpeel Edward J. Palmerino Director of Purchasing Assistant Controller

Jill G. Conley, Ph.D. Susan S. Plotnick Grants Program Officer Assistant Treasurer

Barbara Filner, Ph.D. Robert A. Potter Grants Program Officer Director of Communications

James R. Gavin III, M.D., Ph.D. Ellen B. Safir Senior Scientific Officer Managing DirectorInvestments

Donald H. Harter, M.D. Mark W Smith Senior Scientific Officer and Controller Director, HHMI-NIH Research Scholars Program Lauren Talner Spiliotes, Esq. Associate General Counsel Heidi E. Henning, Esq. Associate General Counsel Daniel M. Szente Managing DirectorEquity Edward H. Klees, Esq. Associate General Counsel Claire H. Wmestock, Ph.D. Senior Scientific Officer Robert B. Kuhn III Director of Human Resources

Howard Hughes Medical Institute 79 91 Joseph G. Perpich, M.D., J.D. Mary McCormick, Ph.D. Office of Vice President for Grants and Special Program Analyst Grants and Programs Graduate Science Education Program Special Stephen A. Barkanic Jean E. Schroeder, Ph.D. Programs Program Officer Program Analyst Undergraduate Science Education Precollege Science Education Program Program International Program

Jill G. Conley, Ph.D. Patricia Soochan Program Officer Program Analyst Precollege Science Education Program Undergraduate Science Education International Program Program

Barbara Filner, Ph.D. Tony Tse Program Officer Program Analyst Graduate Science Education Program Precollege Science Education Program International Program Dennis WC. Liu, Ph.D. Program Officer Rose A. Napper Public Science Education Program Executive Assistant Research Resources Program and Information Patricia A. Anderson Senior Secretary Yvonne P. Forsbergh Program Analyst Jamie H. Gresty Precollege Science Education Program Senior Secretary International Program Monica O'Day Maryrose Franko, Ph.D. Senior Secretary Program Analyst Graduate Science Education Program

Gertrude B. Kelly Moscow Office Editor Laura M. Kennedy Grants Publications Program Coordinator Baltics, Central Europe, and Former Maria Koszalka Soviet Union Initiative Program Analyst Undergraduate Science Education Program

80 Howard Hughes Medical Institute 9 2 Howard Hughes Medical Institute Office of Grants and Special Programs 4000 Jones Bridge Road Chevy Chase, MD 20815-6789 http://www.hhmi.org http://www.hhmi.org/undergraduate/

05-97-006-6K 10/27/199813:57 6142920263 ERIC CSMEE

.2(s)ocila4- U.S. Department of Education Office of Educational Research and Improvement (OERI) National.Librery of Education (NLE) ERIC Educational Resources Information Center (ERIC) REPRODUCTION RELEASE (Specific Document) I. DOCUMENT IDENTIFICATION: Title: ASSEsspIG SC1E/JOEP=4-7-/-11V6y.5: -TRACK.iNG- ScIEAJCEEDUcA-7i6A1 AND CAREERS FRomPRE-COL LE-G-Eri4Roc./H PRor.&.---Ss/ on/AL 1-6-052-..S

Author(s): Corporate Source: Publication Date: /-16wAtzb H1.3q-HIES InEZVCAL /AISTMITE 5/9 7

II. REPRODUCTION RELEASE:

In order to disseminate as widely as possible timely and significant materials of Interest to the educational community, documents announced In the monthly abstract Journal of the ERIC system, Resoumes In Education (RIE), are usually made available tO (MY% in microfiche, reproduced paper copy, and electronic media, and sold through the ERIC Document Reproduction Service (EORS). Credit is given to the source of each document, and, if reproduction release iS granted, one of the following notices is affixed to the document.

If permission is granted to reproduce and disseminate the identified document please CHECK ONE of the following three options and sign at the bottom of the page.

The sample sticker shovm below will be The sample eddied shown below will be The sample sacker shown below MO be affixed to aii Level I documents mixed to ell Level 2A documents affixed te ON Level 20 documents PERMISSION TO REPRODUCE AND PERMISSION TO REPRODUCE AND OISSEMINATE THIS MATERIAL IN PERMISSION TO REPRODUCE AND DISSEMINATE THIS MATERIAL HAS MICROFICHE. AND IN ELECTRONIC MEDIA DISSEMINATE ThlS MATERIAL IN BEEN GRANTED BY FOR ERIC COLLECTION SUBSCRIBERS ONLY, MICROFICHE ONLY HAS BEEN GRANTED BY HAS BEEN GRANTED BY \ei cP TO THE EDUCATIONAL RESOURCES TO THE EDUCATIONAL RESOURCES TO TTIE EDUCATIONAL RESOURCES INFORMATION CENTER (ERIC) INFORMATION CENTER (ERIC) INFORMATION CENTER (ERIC) 2A 2B Level 2A Leve) 28 El E

Cheek here for Level I release. pennielna reproducton Chock hero for Leve1 2A release, Immune reproduction Check here foe Level 28 release, panatella end dissemination in mit:roar:be or other ERIC archival and dluemlnadon In mictoihme AM M electonic media reproduction end dissemination In microfiche only media (cp., electronic) encl paper copy. for ERIC archival collection subsatbers cloy

Documents WU be processed as indicated provided reproduction eureirypereete. If permission to reproduce u granted, but no box Is checked. downtimes will tie processed at Levet I.

I hensby grant to the Educational Resources Information Center (ERIC) nonexclusive pemassion to reproduce and disseminatethis document es indicated above. Reproductio'n from the ERIC microfiche Or electronic media by parsons other thanERIC employees and.ifs system contractors requires permission ham the copyright holder Exception Is made for non-prolit reproduction by libraries end otherservice agencies to satisfy information needs of educators in response to discrete inquiries.

Primed NisfuVeositionnlrtle:srepgrAiA. I34RX4M1C Sign Sionatufe- ' 6E-AtioR PRoao2,4341 o 15-R .wc.I aft,OCAirinoVacer here,-) FAx: OreanixatioN Taleprions: Please iimmictouaNiE3 (C4il 1A1SrlrVi Set.21- 30/-.2../s-sv ref CC rs4Z:5 SEIX2.4D Na.PAID &UM AdVesr. Nft/o/3o/c7r AturVy Ca-M-Sg /112324V4.4--4.78.17 WoVW-hlanti,or issei4 er-ur diet IA'icàq:oi= oriLtar3 ArdZir sPecarri... pillteiGtEAMS (over)