Examining the California Cut Score: An Empirical Analysis of Minimum Competency, Public Protection, Disparate Impact, and National Standards
minimum competence, cut scores, Bar Exam, Bar Passage, Minimum Passing Score, Disparate Impact, Public Protection, Attorney Licensing, Attorney Disciplinary Statistics, Legal Profession Diversity
The selection of a minimum bar exam passing score (“cut score”) shapes the representation of racial and ethnic minorities in the legal profession and the quality of access to justice in the state. California and national policy makers have not had the benefit of detailed exam performance data that analyzes the effect of the cut score on race and ethnicity. Because policy makers consider the cut score an important public protection mechanism, this study also explored whether the selection of higher cut scores better protected the public from attorneys who do not have the minimum competence to practice law.
To conduct the analysis, the study used two data sets. The first data set included 85,727 examinees who sat for 21 administrations of the CBX from 2009-18 and the race and ethnicity of each examinee. The second data set included the ABA discipline data from up to 48 U.S. jurisdictions from 2013-18 and the cut scores in each jurisdiction.
Using the first data set,the study determined how the selection of a minimum cut score (1) widens or narrows the racial and ethnic impacts of the bar exam and/or (2) alters the racial and ethnic composition of new attorneys joining the legal profession. Both historical actual and simulated cut scores were analyzed. Using the second data set, this study examined a third factor: the relationship, if any, between minimum cut scores and rates of attorney discipline.
This analysis determined that initial and eventual passing rates differed significantly between racial and ethnic groups, and this gap was wider at higher simulated cut scores. A simulation analysis using actual examinee scores confirmed that selecting a lower cut score would have significantly narrowed the achievement gap between Whites and racial and ethnic minorities and would have increased the number of newly admitted minority attorneys in California. For example, at 1440, the achievement gap between Whites and Blacks was 27.4 percentage points. But at a simulated cut score of 1300, the achievement gap between these two groups would have been only 14.5 percentage points. This 12.9 percentage point difference in the achievement gap at 1440 and 1300 demonstrates a disparate effect of the higher cut scores.
Using the second data set about disciplinary statistics, the study determined that no relationship exists between the selection of a cut score and the number of complaints, formal charges, or disciplinary actions taken against attorneys in the jurisdictions studied.
California’s recent decision to lower the cut score from 1440 to 1390 moved California from having the second-highest cut score to the fourth-highest cut score in the country. However, the report data established that at 1390 California will continue to produce significantly disparate pass rates on the basis of race and ethnicity when compared to the national norm of 1350, the New York standard of 1330, and the simulated model of 1300.
This study establishes that maintaining a high cut score does not result in greater public protection as measured by disciplinary statistics but does result in excluding minorities from admission to the bar and the practice of law at rates disproportionately higher than Whites.