AI enables lawyers to produce better work in less time

“This is the first empirical evidence that AI tools can consistently and significantly improve the quality of legal work across a wide range of realistic legal assignments.”

That is the conclusion of a recent study by the University of Minnesota and the University of Michigan regarding the effect of two advanced AI tools on legal performance. Unlike previous studies, which primarily focused on speed, this research shows that AI can now also elevate the substantive quality of legal work to a higher level.

The study: realistic assignments, real insights

For the first time, a randomized field test was set up with 127 law students performing six realistic legal tasks, such as those typically expected at a law firm:

• Drafting an email for a client
• Drafting a legal memo
• Analyzing a complaint
• Drafting an NDA
• Drafting a motion for consolidation to merge two cases in a Minnesota court
• Drafting a persuasive letter regarding a non-compete clause

The students were then divided into three groups:

• without AI
• with Vincent AI (based on Retrieval Augmented Generation, also known as RAG)
• with o1-preview (OpenAI’s reasoning model)

The goal: to measure what AI actually adds in terms of speed, accuracy, structure, and legal depth.

The results: AI increases both productivity and quality

The figures speak for themselves:

• Vincent AI increased productivity by 38% to 115%
• o1-preview went even further, with an increase of 34% to 140%

But more importantly: in four of the six assignments, the quality of the legal work was statistically significantly improved. The o1-preview tool even provided up to 28% better scores on elements such as clarity, organization, and legal analysis.

Different AI tools, different strengths

Both tools improved legal output, but each in its own way.

Vincent AI (RAG technology)
Vincent AI combines generative AI with up-to-date legal sources. Instead of relying blindly on training data, Vincent retrieves relevant case law and legislation from a proprietary database of legal content and bases its output on that.

This resulted in:

• More professional and readable texts
• The lowest amount of hallucinated sources (only 3 cases)
• Fewer typos and a clearer structure

However, Vincent AI scored less strongly on legal depth and accuracy, especially in broader assignments where core legal issues had to be identified independently.

o1-preview (OpenAI reasoning model)
This tool tackles complex legal problems through a process of step-by-step reasoning.

The advantages:

• The greatest quality improvement in legal analysis
• Strong performance in in-depth assignments such as complaint analyses and in written pleadings supporting a motion or the written explanation for an interlocutory application
• Better focus on relevant parts of the assignment

Disadvantage: more hallucinations than Vincent AI and even more than the group without AI. In total, 11 hallucinated sources were counted with o1-preview, compared to 3 with Vincent and 4 without AI.

Why this research matters

Until now, it seemed that conducting legal research with AI led to faster output, but not always to more depth and quality.

This study proves that AI tools can substantially contribute to legal quality as well, provided:

• They are deployed correctly
• There is sufficient domain knowledge and supervision
• The tool matches the nature of the assignment

The latter was evident, for example, from the fact that no significant gain was achieved in drafting an NDA—neither in speed nor quality. For this assignment, participants were provided with a standard template. Since filling out such a template is common practice in contract work, there was little room for AI to add value.

The bigger picture: AI + lawyer = synergy

The most promising insight? The combination of reasoning (such as o1-preview) with RAG (such as Vincent AI) offers enormous potential. Researchers see this integration as the next step in legal AI.

This is because:

• RAG reduces the chance of hallucinations by incorporating legal sources from external datasets.
• Reasoning improves processes related to analysis, logic, and argumentation.

Conclusion

For the first time, it has been empirically demonstrated that generative AI can produce not only faster but also higher-quality legal output. Particularly in more complex tasks, the models studied show a demonstrable difference.

The next step? Combining both techniques—reasoning and RAG—to utilize the best of both worlds. The research results confirm that generative AI not only makes legal work faster but also improves it qualitatively—provided the technology is sufficiently anchored in legal sources and capable of in-depth reasoning. Where Vincent AI convinces through accuracy and source citation, o1-preview excels in argumentation and analysis. Both are essential in legal practice and are therefore the driving forces behind LegalMike.