A large-scale field experiment spanning 1,559 judges across 118 courts in Pakistan has demonstrated that generative AI, when paired with targeted training, can dramatically boost judicial productivity while delivering an estimated return of $38.50 for every dollar invested. The study, conducted by researchers from ETH Zurich, the New Economic School, and Imperial College London, provides some of the strongest real-world evidence yet that AI assistants can enhance public sector efficiency without sacrificing quality or introducing bias.<\/p>\n
JudgeGPT is an AI assistant<\/a> built on OpenAI’s GPT-4, designed specifically for Pakistani trial courts. It uses retrieval augmented generation (RAG) to search a database of 129,235 documents, including 128,292 court rulings and 943 Pakistani laws. When a judge enters a query, the system identifies the ten most relevant passages and generates a cited answer, grounding its output in verifiable legal sources. The tool was deployed across roughly half of all Pakistani trial court judges, making it one of the largest randomized controlled trials of AI in a judicial setting ever conducted.<\/p>\n The RAG architecture is critical to the system’s reliability. Rather than relying on the model’s internal knowledge alone, JudgeGPT retrieves specific passages from an authoritative corpus and presents them alongside the generated response. This design reduces the risk of hallucination and allows judges to verify the AI’s output against source material, a non-negotiable requirement for any tool intended to support legal decision-making.<\/p>\n The researchers divided judges into three groups. The first group received JudgeGPT access plus targeted training: six 90-minute lectures delivered over three weeks by ETH Professor Elliott Ash after court hours. Judges learned which tasks the tool handled well, where it fell short, and how to verify its output. The second group received the same AI access but only a general seminar on technology and law. The control group attended that seminar with no JudgeGPT access.<\/p>\n The difference in adoption was stark. AI access alone did little. Judges who received targeted training used JudgeGPT four times as much as those who attended only the general seminar. Over 40 weeks, trained judges averaged nearly 60 logins and more than 200 prompts. The comparison group averaged about 20 logins and fewer than 50 prompts. This pattern suggests that providing a capable AI tool without structured guidance on how to use it effectively yields minimal engagement and, consequently, minimal impact.<\/p>\n Districts with more trained judges resolved significantly more cases. At moderate exposure levels, the increase amounted to roughly 1,848 extra cases per year per district, a 6.3 percent improvement. Even districts in the bottom quartile of usage cleared about 616 additional cases. For a judiciary grappling with chronic backlogs, these gains are substantial and operationally meaningful.<\/p>\n The productivity boost came without increasing judge workloads. The study found that judges worked the same hours and reported no change in work-life balance, suggesting that the AI tool helped them work more efficiently rather than longer. This is a crucial distinction: the gains came from better use of existing time, not from expecting more hours from already overburdened personnel.<\/p>\n The researchers calculated the economic impact by estimating what it would cost to hire enough additional judges to match the same output. The result was roughly $38.50 in value for every dollar invested in the AI system and training. Even under conservative assumptions, the return was at least $10 per dollar. These figures underscore the potential of well-deployed AI tools to deliver outsized value in resource-constrained public institutions where hiring additional staff is often slow, expensive, or politically difficult.<\/p>\n For context, the cost of recruiting, training, and compensating a single additional judge in many jurisdictions far exceeds the per-judge cost of an AI assistant with proper training. The experiment suggests that AI can function as a force multiplier, amplifying the output of existing personnel rather than requiring headcount expansion.<\/p>\n One of the most significant findings is that the productivity gains did not come at the expense of judgment quality. A review of roughly 4,000 court judgments found that readability, length, and the number of legal arguments held steady. The appeal rate per 1,000 resolved cases actually fell slightly, suggesting that rulings remained sound and may have improved.<\/p>\nThe Experiment: Three Groups, One Clear Result<\/h2>\n
How JudgeGPT Cleared Court Backlogs<\/h2>\n
Return on Investment: $38.50 per Dollar<\/h2>\n
Quality and Bias: No Trade-Offs Found<\/h2>\n