The Cause and Effect of Recent Changes to the Louisiana Bar Examination

By Scott T. Whittaker and Dona K. Renegar

milestone in the annals of score of at least 650 out of the possible 2001: The Process Begins Louisiana legal practice was 900 total points that can be earned in the passed in July 2012 when a nine subjects tested. The nine subjects of the Louisiana Bar new method of grading the As far as anyone can remember, this exam (Code I, II and III; Louisiana Civil Louisiana Bar Examina- is the first change that has been made to Procedure; ; Federal Jurisdiction and tion (Bar exam) was instituted. No longer the grading standards of the Louisiana Procedure; Constitutional ; Criminal Ado applicants to the Louisiana Bar have Bar Examination since the Bar exam was Law and ; and Business Entities to worry about passing seven of the nine instituted. This article explains the reasons and Negotiable Instruments) are written subjects tested on the Bar exam, while not for the change, the long process that resulted and graded by examiners, who are ap- failing more than one of the five “Code” in the change, and the results of the two Bar pointed by the Louisiana Supreme Court subjects. No longer can an applicant “con- exams that have been administered since the and are members of COBA. Each examiner ditionally fail” the Louisiana Bar exam by change was implemented. It also describes has the help of assistant examiners in the passing at least five of the nine subjects and future changes (which are even more sig- grading of the exam answers. Prior to 2001, three of the five Code subjects, but failing nificant) to the Bar exam that have been the examiners acted independently of one to achieve a passing score on seven of the proposed by the Louisiana Supreme Court another, which resulted in significant dif- nine subjects. As of July 2012, Louisiana Committee on Bar Admissions (COBA). ferences among the subject matter exams Bar applicants simply have to achieve a in terms of the number of questions and

90 August / September 2013 format (long essays vs. short answer). limit the number of times an applicant may ► Should success on the Bar exam be In 2001, the Louisiana Supreme Court take the exam; and whether to continue to determined by means of one numerical restructured COBA by creating a subcom- allow applicants to “condition” the exam. score that would be arrived at by combin- mittee called the “Testing Committee” to ing the scores of the individual exams? study and recommend improvements to The Testing Committee began gather- ► Should all applicants be required all aspects of the Bar exam. The Testing ing information and input. By the end to take the entire Bar exam, thereby Committee was initially composed of of 2001, the committee had reviewed eliminating the “conditional failure” that Scott T. Whittaker (director), David E. information regarding the scope of the is currently allowed? Walle and Rebecca L. Hudsmith. All three bar exams administered by all of the ► Should the Bar exam be restruc- were members of COBA who had com- other states, as well as the critiques of tured so as to shift the orientation toward pleted 10-year terms as examiners. Over the Louisiana Bar exams for the past five legal practice and away from the current the ensuing years, the composition of the years, which had been submitted by the Bar system of testing traditional categories of Testing Committee changed. At various Admissions Advisory Committee (com- substantive law and procedure? times leading up to the COBA proposals prised of representatives of all Louisiana ► Should the Multi-State Performance of February 2010, the members of the law schools). The Testing Committee also Test (MPT) be added as an additional Testing Committee included Professor began drafting and planning a detailed requirement of the Bar exam? Michelle L. Ghetti of Southern University survey to gather input from members of ► Should the Louisiana Code of Law Center, Professor David W. Gruning the Louisiana Bar, the judiciary and the Professional Responsibility be included of Loyola University Law School, and law schools. in the essay portion of the Bar exam; former examiners Billy J. Domingue and and, if so, should this be in addition to or Hon. S. Maurice Hicks, Jr. 2002-2004: Studying and in lieu of the current practice of testing As stated in the Louisiana Supreme Gathering Input professional responsibility by means of Court’s 2001 Annual Report, the general the Multistate Professional Responsibility areas to be studied by the Testing Com- The members of the Testing Commit- Exam (MPRE)? mittee were: tee also felt that, to properly carry out ► Should the current policy of giving their charge, they needed to learn about greater weight to the “Code” sections of ► The substance of the Louisiana testing theory. Therefore, in 2002, they the Bar exam be continued? Bar Examination. This includes, with- contacted the National Conference of Bar out limitation, whether any new topics Examiners (NCBE), the organization that Then-Louisiana Supreme Court Chief should be added; whether any existing prepares and administers the Multi-State Justice Pascal F. Calogero, Jr. sent those topics should be deleted; whether to alter Bar Exam and whose staff includes some questions to the dean and/or chancellor of the current system of testing professional of the most highly respected profession- each Louisiana law school and convened responsibility (e.g., by ceasing to use als in the area of high-stakes testing. The a May 2004 meeting among the Louisiana the multi-state ethics exam, by testing NCBE staff helped educate Testing Com- Supreme Court justices, the Testing Com- professional responsibility on other por- mittee members about important testing mittee and the law schools. Later in 2004, tions of the exam, and/or by reinstituting concepts, such as validity, reliability and follow-up meetings took place between a professional responsibility essay exam); consistency across time (which are beyond the Testing Committee members and the and whether a performance exam such as the scope of this article but which played faculties and administration of Louisiana the multi-state performance test should a key role in the proposals for change that State University Paul M. Hebert Law Cen- be adopted. were eventually put forth). ter and Southern University Law Center. ► The structure of the Louisiana Bar In 2003, the Bar exam survey was con- Examination. This includes, without limi- ducted online to solicit input from mem- 2005: Hurricane Katrina tation, whether to keep the current nine- bers of the Louisiana Bar, the judiciary and Sidetracks Process exam format or combine the exams into the law schools. Significant efforts were two or more exams (e.g., simply “Code” made to publicize the survey, including Prior to the Louisiana Supreme Court and “Non-Code”); whether to continue sending letters to every member of the taking action on these possible areas of the practice of labeling examinations; and Bar, judiciary and law school faculties change, Hurricane Katrina struck dur- whether to reduce the number of days on encouraging them to take part. ing the July 2005 Louisiana Bar exam which the exam is held from three to two. In early 2004, after approximately two grading process. At the time Hurricane ► The procedural aspects of the and one-half years of study and gathering Katrina made landfall, approximately Louisiana Bar Examination. This in- input, the Testing Committee issued the 5,000 separate exam answers were in cludes, without limitation, whether to alter following written questions to the Loui- the hands of approximately 450 assistant the current system of volunteer Examiners siana Supreme Court in order to suggest examiners, most of whom were displaced and Assistant Examiners; whether to alter potential Bar exam changes: by the storm. At that time, no copies of the current grading procedure; whether to

Louisiana Bar Journal Vol. 61, No. 2 91 exam answers were kept. Therefore, the process. To assist in the preparation of in the development of multiple-choice original exam answers had to be recovered the proposed final recommendations for questions. The essay questions on the from offices and homes throughout the changes, COBA retained testing expert restructured exam would cover the subject hurricane-ravaged area. The examiners, Stephen P. Klein, Ph.D., and testing matter currently covered by the Bar exam assistant examiners, staff and other mem- consultant Karen Barbieri. tests in the following subject areas: Code bers of COBA exerted herculean efforts In September 2009, COBA officers I; Code II; Code III; Louisiana Code of to locate and properly grade all exams. met with then-Louisiana Supreme Court Civil Procedure; Torts; and Business Enti- To properly chronicle those efforts would Chief Justice Catherine D. (Kitty) Kimball ties and Negotiable Instruments. Profes- fill a book. In the end, the grades of only and then-LSBA President Kim M. Boyle sional responsibility, currently tested only 13 applicants were affected through their to discuss the proposed plan. COBA, through the MPRE, also would be added exam answers being lost in the storm. Klein and Barbieri created PowerPoint as an essay exam topic. The requirement COBA administered the first-ever make- presentations outlining the proposals of a current passing score on the MPRE up Bar exam to those 13 applicants, all for change and calling for comments. In would be retained under the restructured of whom, happily, passed the exam and January 2010, those PowerPoint presen- procedures. The essay exam testing, were sworn in only one month later than tations were circulated to the Louisiana however, would be “blind,” in that the the other applicants who passed the July Supreme Court, the LSBA, the Louisiana applicants would not be told which sub- 2005 Louisiana Bar exam. law schools and others. In February 2010, jects are being tested on a particular day. Needless to say, the proposals of the the presentations were posted on the Testing Committee to improve the Bar COBA website. LSBA Response exam were put on hold following Hurri- To summarize, the proposals consisted cane Katrina, while more pressing matters of a “short‑term plan” and a “long‑term In March 2010, then-LSBA President were addressed. Those matters included plan.” The short‑term plan was to change Boyle appointed a six-person committee procuring and installing a system of digi- the method of scoring the Bar exam from to review the proposed changes to the tizing exam answers, with secure off-site its historical, “conjunctive approach” of Louisiana Bar exam.1 The LSBA com- backup, so as to eliminate the possibility having to pass seven of nine subjects and mittee was tasked with reviewing the of losing exam answers in the future. four of five “code” subjects, to a single suggested short- and long-term changes grade of 630 for all nine exam subjects (a proposed by COBA and providing a 2006-2010: Process passing score of 70 multiplied by the nine report by May 1, 2010. The PowerPoint Continues and Proposed exam subjects). The long‑term plan was presentations prepared by COBA were a complete revamping of the Louisiana Changes Announced placed on the LSBA website and the Bar exam to be administered on Tuesday, LSBA solicited members’ comments on Wednesday and Thursday (as opposed to From February 2006 through July the proposed changes. Boyle requested the historical Monday, Wednesday and that the Louisiana Supreme Court delay 2009, COBA provided examination data Friday format), as follows: to the NCBE after each administration of action on the COBA proposals until the LSBA could obtain reaction from its the Bar exam. The NCBE analyzed the ► Tuesday morning: three-hour data and worked with COBA to develop members and respond. Multistate Performance Test prepared The first meeting of the LSBA com- testing improvements. By December by the NCBE; 2008, COBA was ready to move toward mittee was on April 5, 2010. The com- ► Tuesday afternoon: three hours of mittee members reviewed the proposals adopting a “compensatory” grading essay questions prepared by COBA; system, under which success on the Bar and questions raised by LSBA members ► Wednesday morning: three hours and deans and faculty of Louisiana’s exam would be determined by one score of multiple-choice questions prepared achieved by combining the scores of the law schools. Four meetings were held in by COBA; conjunction with local bar associations individual exams. This compensatory ► Wednesday afternoon: three hours grading system was to be the first step throughout the state in order to obtain of multiple-choice questions prepared responses from as many LSBA members in a two-part proposal to overhaul the by the NCBE; Bar exam. as possible to the proposed changes to ► Thursday morning: three hours of the Bar exam.2 COBA sought input from the Louisi- essay questions prepared by COBA; and ana Supreme Court, the law schools, the The LSBA committee met with COBA ► Thursday afternoon: three hours and NCBE representatives on May 15, Louisiana State Bar Association (LSBA) of essay questions prepared by COBA. and other interested groups. Based on 2010, to discuss responses from LSBA members and concerns expressed by feedback received, COBA worked with The long-term plan also included a the NCBE to develop a presentation the deans and faculty of Louisiana’s law proposed restructuring of COBA and a schools. Information was exchanged on addressing the questions and concerns proposal to use paid consultants in con- raised during the information-gathering May 15, and a subsequent meeting was nection with the Bar exam, especially held on Oct. 20, 2010, with COBA, its

9292 AugustAugust / September / September 2013 2013 consultants, NCBE representatives, and comprised of LSBA members and repre- and recommendations to be submitted the deans and some faculty of Louisiana’s sentatives from each of the state’s four law to the LSBA’s Board of Governors and law schools. schools. This committee would undertake House of Delegates and to the Louisi- The LSBA committee issued its final a systematic and comprehensive analysis ana Supreme Court. The reconstituted report on Nov. 29, 2010. The report of all proposed changes to the Louisiana committee allowed members who had concluded that the proposals by COBA Bar exam, with the input and advice of an different viewpoints from the majority to would improve the validity, reliability independent expert. The committee was submit separate concurring or dissenting and fairness of the Bar exam; however, asked to complete its analysis no later than reports. The final committee meeting was it suggested the following changes to the June 1, 2011, with reports being submitted held on May 31, 2011. COBA proposals: to the LSBA’s House of Delegates and Dr. Haladyna prepared three papers for Board of Governors and the Louisiana the reconstituted committee, including the ► raising the passing score from 630 Supreme Court. The resolution passed Validity for a Licensing Test, Analysis and to 650, which the COBA expert thought unanimously. Evaluation of the Current Louisiana Bar would result in a first-time passing rate In December 2010, then-LSBA Presi- Exam and Opinions about COBA’s Plan that was consistent with historical passing dent Michael A. Patterson appointed a for Short-Term Revision of the Current rates for first-time exam takers who sat Reconstituted Committee to Review Examination, and Long-Term Plan for Re- for all nine Bar exam sections; Proposed Changes to the Bar Exam. vision of the Louisiana Bar Examination, ► weighting the Code subjects twice The committee was comprised of 12 Observations and Recommendations. The that of the non-Code subjects, to ensure members, some of whom served on the reconstituted committee’s primary focus proficiency in Code subjects; original committee and some new mem- was to review Dr. Haladyna’s papers, ► making additional efforts to publi- bers.3 The reconstituted committee held discuss his conclusions, and arrive at cize and obtain additional input from the an initial meeting on Jan. 26, 2011, and a consensus regarding the conclusions practicing Bar prior to the adoption of the agreed that the November report prepared and recommendations to be made to short-term proposal; and by the original Committee to Review the LSBA. The reconstituted committee ► conducting a separate process the Proposed Changes to the Bar Exam met via conference call numerous times to publicize and obtain input from the would be supplemented by the recon- throughout the month of May 2011.4 practicing Bar, after the implementation stituted committee’s report. The recon- Like the original LSBA committee, of the short-term plan and prior to the stituted committee appointed an ad hoc the reconstituted committee concluded implementation of the long-term plan. subcommittee, led by Professor Melissa that the COBA proposals would improve Thornton Lonegrass, to conduct a search the validity, reliability and fairness of the The report was presented to the for an independent testing consultant to Louisiana Bar exam, but recommended LSBA’s Board of Governors by com- assist the reconstituted committee in its some changes. In regard to the short-term mittee members on Dec. 4, 2010. Jack determination. At the Feb. 22, 2011, com- plan, the reconstituted committee recom- M. Weiss, chancellor of Louisiana State mittee meeting, Dr. Thomas M. Haladyna, mended that: University Paul M. Hebert Law Center, Professor Emeritus at Arizona State Uni- also made a presentation to the LSBA’s versity, was recommended. The reconsti- ► a “cut-score study” be performed Board of Governors at that meeting, tuted committee retained Dr. Haladyna in to determine the passing score, and if generally recommending rejection of March, under a cost-sharing arrangement such a study was not performed, a 650 the short-term plan for compensatory among the LSBA and the law schools at passing score should be used instead of scoring. Based on the recommendations LSU, Loyola and Tulane. During March the 630 proposal; of the committee and Chancellor Weiss, 2011, the reconstituted committee held ► COBA re-grade exams in a “zone the Board of Governors passed a resolu- conference calls with Dr. Haladyna to of uncertainty” both above and below the tion strongly recommending the Supreme define the scope of the work he would passing score, rather than regarding only Court defer any decision until after the perform for the committee. He was pro- failing exams as proposed by COBA; and suggested changes were reviewed by an vided with COBA’s proposed changes to ► COBA immediately discontinue independent expert. the Bar exam, concerns regarding same reuse of essay exam questions. The resolution noted that there were from LSBA members and representatives significant issues which had not been from Louisiana’s law schools, and the data The reconstituted committee report fully discussed and analyzed by the from the eight administrations of the Bar also noted that its members were divided LSBA’s Board of Governors and House exam during 2007 through 2010. over the issue of weighting the Code of Delegates, so the Board resolved to Once the reconstituted committee con- courses twice as much as the non-Code recommend that the Louisiana Supreme cluded consultations with Dr. Haladyna, courses, although a majority of the re- Court publish the proposed rule changes several telephone conferences were con- constituted committee members favored by COBA to allow sufficient time for the ducted to reach a consensus among com- “double-weighting” of Code subjects. LSBA to appoint a special committee, mittee members regarding conclusions The reconstituted committee rec-

Louisiana Bar Journal Vol. 61, No. 2 93 ommended that the long-term plan be In July 2012, the pass rate for the Louisiana Bar exam was: adopted, with an altered method of School/Group Applicants Passed Failed Pass Rate “equating” essay answers, but only after additional study and verification LSU 179 135 44 75.42% of some of the underlying assumptions, Loyola 193 116 77 60.10% the conduct of additional publicity and information campaigns, and after the Southern 139 65 74 46.76% determination, made more than one year Tulane 107 81 26 75.70% after implementation of the short-term plan, of whether the goals of the short- Out-of-State 137 66 71 48.17% term plan were achieved, whether there TOTAL 755 463 292 61.32% were any unintended consequences, and whether the results of the short-term plan indicated any changes should be made to In July 2011, the pass rate for the Louisiana Bar exam was: the long-term plan. The final report of the reconstituted School/Group Applicants Passed Failed Pass Rate committee, including its conclusions and LSU 164 142 22 86.59% recommendations, was submitted to the LSBA on June 10, 2011. Loyola 171 120 51 70.17% Southern 127 79 48 62.2% Changes Adopted Tulane 98 80 18 81.63% On Oct. 19, 2011, the Louisiana Su- Out-of-State 190 122 68 64.21% preme Court issued orders implementing TOTAL 750 543 207 72.4% the short-term changes to the Louisiana Bar exam, the highlights of which includ- ed compensatory scoring taking effect in In February 2013, the pass rate for the Louisiana Bar exam was: July 2012, a passing score requirement of 650, and the code subjects being weighted School/Group Applicants Passed Failed Pass Rate two to one (2:1), extending applicants a LSU 34 18 16 52.94% final opportunity in February 2012 to take advantage of their current conditional Loyola 87 27 60 31.03% failure status, eliminating spotting, and Southern 67 22 45 32.84% placing a five-time limit on unsuccessful attempts to pass the Bar exam. The Loui- Tulane 24 9 15 37.5% siana Supreme Court intends to take time Out-of-State 98 50 48 51.02% to study the long-term proposed changes to the Bar exam. TOTAL 310 126 184 40.65%

Results of Changes This compares to the results from the Feb. 2012 administration: Bar applicants who conditionally School/Group Applicants Passed Failed Pass Rate failed the Bar exam were allowed to take LSU 30 22 8 73.3% single sections one last time in February 2012. Thereafter, any failing applicant Loyola 68 37 31 54.4% was required to retake all nine sections and Southern 64 27 37 42.2% comply with the changes implemented by the Louisiana Supreme Court. The first Tulane 26 16 10 61.5% examination implementing the changes Out-of-State 131 69 62 52.7% was administered in July 2012, and the second was administered in February TOTAL 319 171 148 53.6% 2013. COBA shared the data from those examinations for use in this article. These comparisons show a decline in the overall passage rate from 72.4% to 61.32% between July 2011 and July 2012, and from 53.6% to 40.65% between February 2012 and February 2013.

94 August / September 2013 The comparisons between the results achieved by first-time Bar exam takers, however, These comparisons show a decline in tell a different story. Those comparisons are as follows: the overall first-time taker passage rate of only about 6 percentage points (74.75% July 2012 first-time takers: to 66.98%) between July 2011 and July School/Group Applicants Passed Failed Pass Rate 2012, as opposed to the approximately LSU 170 132 38 77.65% 11-percentage-point drop indicated by the results of the total applicant pool (72.4% Loyola 170 108 62 63.53% to 61.32%). Also, the February exam pass Southern 108 60 48 55.56% rate for first-time takers actually went up under the new grading scheme — from Tulane 99 78 21 78.79% 44.62% in February 2012, to 48.74% in Out-of-State 98 54 44 55.1% February 2013. This is as opposed to the approximately 13-percentage-point drop TOTAL 645 432 213 66.98% indicated by the results of the total ap- plicant pool (53.6% to 40.65%). This compares to the following results who sat for all COBA also shared the results of reli- nine Bar exam subjects in July 2011: ability studies performed by COBA’s Failed expert psychometrician, Stephen P. Klein, School/Group Applicants Passed (including Pass Rate Ph.D., on the July 2012 and February 2013 Conditioned) examination data. In those studies, Dr. LSU 150 132 18 88% Klein concluded that the score reliability Loyola 132 94 38 71.21% for the July 2012 examination was 0.92, and the score reliability for the February Southern 89 59 30 66.29% 2013 examination was 0.90. The target Tulane 87 68 19 78.16% score reliability for high-stakes tests such as the Louisiana Bar exam is 0.90. Out-of-State 132 88 44 66.66% TOTAL 590 441 149 74.75% Conclusion

Although improving the reliability In February 2013, the pass rate for first-time takers was: of the Louisiana Bar exam was one of School/Group Applicants Passed Failed Pass Rate COBA’s goals in suggesting changes to the testing process, these results beg the ques- LSU 3 3 0 100% tion: Have the changes made the Louisi- Loyola 22 6 16 27.27% ana Bar Examination too difficult to pass? Southern 17 6 11 35.29% We believe further study and analysis is necessary before any conclusion can be Tulane 11 5 6 45.45% reached regarding this question. It may be Out-of-State 66 38 28 57.58% that passage rates will return to historical levels after a few more administrations of TOTAL 119 58 61 48.74% the Bar exam. Or, it may be that a drop in passage rates is appropriate because the historical passage rates included as “pass- This compares to the following results of those who sat for all ing” those applicants who had previously nine Bar exam subjects in February 2012: “conditionally failed” and who, therefore, School/Group Applicants Passed Failed Pass Rate were only taking a selected subset of the entire exam. One thing that is certain, LSU 7 5 2 71.43% however, is that the Louisiana Bar Ex- Loyola 20 4 16 20% amination will provide a good topic for discussion between the practicing Bar, the Southern 24 9 15 37.5% law schools and the Louisiana Supreme Tulane 9 6 3 66.67% Court for years to come. Out-of-State 70 34 36 48.57% TOTAL 130 58 72 44.62%

Louisiana Bar Journal Vol. 61, No. 2 95 FOOTNOTES Scott T. Whittaker, a Dona K. Renegar, a mem- member of Stone Pig- ber in the Lafayette office man Walther Wittmann, of Huval, Veazey, Felder 1. Members of the committee included John M. L.L.C., and chair of the & Renegar, L.L.C., cur- Frazier, Hon. Piper D. Griffin, Cordell H. Haymon, firm’s Business Section, rently represents the Third Alejandro R. Perkins, Dona K. Renegar and Chair received the Louisiana District on the Louisiana Scott T. Whittaker. State Bar Association’s State Bar Association’s 2. The Lafayette meeting was held on May 11, (LSBA) President’s Award (LSBA) Board of Gover- 2010. The Baton Rouge meeting was held on May in 2011 for his service as nors. She received two BA 26, 2010. The Shreveport meeting was held on June chair of the two LSBA degrees in 1988 in English 17, 2010. The New Orleans meeting was held on special committees seated and French, both from the June 29, 2010. to study the proposed University of Louisiana- 3. The Reconstituted Committee to Review Pro- changes to the Louisiana Lafayette, and her JD de- posed Changes to the Bar Examination consisted of Bar Exam. He received his undergraduate degree, gree in 1992 from Tulane Law School. She served Kim M. Boyle, Hon. Piper D. Griffin, Cordell H. cum laude, in 1982 from Tulane University and his on the two LSBA special committees seated to study Haymon, Robert A. Kutcher, Winfield E. Little, JD degree, magna cum laude, in 1984 from Tulane the proposed changes to the Louisiana Bar Exam. Jr., Melissa Thornton Lonegrass, John A. Lovett, Law School. He is a former chair of the Louisiana She has served in the LSBA’s House of Delegates Alainna R. Mire, Darrel J. Papillion, Alejandro R. Supreme Court’s Committee on Bar Admissions; and as chair of the LSBA’s Young Lawyers Division. Perkins, Dona K. Renegar and Chair Scott T. Whit- he served on this committee for more than 17 (2 Flagg Place, Lafayette, LA 70508) taker. years (including more than 10 years as examiner 4. The Reconstituted Committee held confer- for Business Entities and Negotiable Instruments). ence calls on May 4, May 6, May 13, May 18, May (546 Carondelet St., New Orleans, LA 70130) 25 and May 31, 2011. Ethics Advisory Service www.lsba.org/ethicsadvisory For assistance with dilemmas and decisions involving legal ethics, take full advantage of the LSBA’s Ethics Advisory Service, offering - at no charge - confidential, informal, non-binding advice and opinions regarding a member’s own prospective conduct. Eric K. Barefield, Ethics Counsel LSBA Ethics Advisory Service 601 St. Charles Ave., New Orleans, LA 70130-3404 (504)566-1600, ext. 122 • (504)619-0122 toll-free: (800)421-5722, ext. 122 • Fax: (504)598-6753 E-mail: [email protected]

96 August / September 2013