Same law, different judge?

The same law should produce the same result whatever courtroom you are assigned to. Research in several court systems has found that the judge you draw can matter a lot. Here is what careful studies found, with their own limits, and what this site can and cannot show.

What the research found

Each card gives one finding with its number, where and when it was measured, and the study’s own main caveat. The link opens the study.

Asylum: 5% before one judge, 88% before another

Colombian applicants in the Miami immigration court had a 5% chance of prevailing before one judge and an 88% chance before another judge in the same building. The study covered US asylum decisions by 225 immigration judges over four and a half years.

Caveat: grant rates are not adjusted for the merits of each case, and the links to judges’ backgrounds are correlations, not causes.

Ramji-Nogales, Schoenholtz & Schrag, Refugee Roulette: Disparities in Asylum Adjudication, 60 Stanford Law Review 295 (2007–2008)

Federal sentences: a gap of about 17%, narrowed to about 11% by sentencing rules

For comparable caseloads, the expected difference between two typical federal judges’ average sentence lengths was about 17 percent (4.9 months) in 1986–87, before the Sentencing Guidelines, and about 11 percent (3.9 months) in 1988–93, after them.

Caveat: the authors did not measure parole before the Guidelines, or other sources of disparity such as prosecutors’ decisions.

Anderson, Kling & Stith, Measuring Interjudge Sentencing Disparity: Before and After the Federal Sentencing Guidelines, 42 Journal of Law and Economics 271 (1999)

Federal sentences: the spread between judges in the same city grew from 18.2 to 27.6 points

Averaged over 30 cities, the spread between the highest and lowest judge, measured as percentage points of distance from the guideline minimum, rose from 18.2 to 23.7 to 27.6 across three periods between fiscal years 2005 and 2017.

Caveat: the comparison relies on cases being assigned at random within a city; the trend slowed after 2011 and a few cities saw decreases.

U.S. Sentencing Commission, Intra-City Differences in Federal Sentencing Practices (January 2019)

Federal sentences: differences between judges “doubled” after the Guidelines became advisory

With data on almost 400,000 federal defendants linked to their sentencing judges, differences between judges doubled after the Booker decision made the Guidelines advisory.

Caveat: the author finds that prosecutors’ charging choices, which vary with the judge, are likely a prominent source of the disparity, so judges are not its only source.

Crystal S. Yang, Have Interjudge Sentencing Disparities Increased in an Advisory Guidelines Regime? Evidence from Booker, 89 New York University Law Review, no. 4 (2014)

Appeals: whether a ruling is reversed depends partly on which appeals panel is drawn

In Norway’s courts, where cases are assigned at random at both stages, randomly assigned appeal panels differ in how often they reverse, and trial judges who are reversed shift their later decisions in the reversal’s direction.

Caveat: a Norwegian study, read from its abstract; it is not a test of reversal as a measure of a judge’s quality. It is the important limit on reading any reversal record, including the one on this site.

Bhuller & Sigstad, Feedback and Learning: The Causal Effects of Reversals on Judicial Decision-Making, 92 Review of Economic Studies 2359 (2025)

What this means for Colorado, and what this site shows

Ratings from people who were in the courtroom: parties, jurors, observers and attorneys, shown separately, about conduct people can see.
For each trial judge, the Court of Appeals’ decisions on their rulings since June 2023, each linked to the court’s own announcement, and a statewide Appeals record page by type of case.
Colorado context: in a 2008 survey of Colorado judges by IAALS, 64% of appellate judges and 53% of trial judges called the small number of survey respondents a “major problem” of the state’s own judicial performance evaluation program. That is a survey of judges’ opinions from 2008, not a measure of differences between judges.

IAALS, The Bench Speaks on Judicial Performance Evaluation: A Survey of Colorado Judges (January 2008)

What this site cannot show

A judge’s whole caseload. Only appealed rulings appear in an appeals record, and there is no public count of every case a judge decided, so we show counts, never a rate.
Whether a reversal says more about the ruling or about the panel: reversal outcomes depend partly on which appeals panel is drawn (the last study above).
Whether one judge is “better” than another. No page here says that, and the research above does not support reading any single record that way.

Gaps we did not fill

We found no rigorous published study of how much family-court judges differ from one another. Studies of child-welfare decisions exist, but they measure caseworkers, not judges.
We found no peer-reviewed study that validates a reversal rate as a measure of a trial judge’s quality. Until one exists, an appeals record is a public record to read, not a score.
Several widely cited bail and sentencing figures exist only in working-paper versions. We left them out rather than print a number the published article may have changed.

Appeals record, statewide How to read an appeal Judges and prosecutors by courthouse