To Use GenAI or Not to Use GenAI: A Practical Guide for Legal Technology Professionals
As Co-Chair of this project, James MacGregor founder of Ethical eDiscovery, authored a blog for ILTA announcing the launch of the ILTA Gen AI Best Practice Guide, and the steps that were taken in order to create it. The original blog can currently be found here on the ILTA website.
As someone who assisted with the creation of ILTA's GenAI Best Practice Guide for eDiscovery, I've spent considerable time wrestling with a question that's on every legal technology professional's mind: Should we be using Generative AI, and if so, how do we do it responsibly?
The answer, like most things in legal technology, is nuanced. But after months of research and collaboration with industry experts, I can tell you this: GenAI is already being widely used to solve legal problems, sometimes effectively, sometimes not and sometimes with disastrous consequences. It's a powerful tool, that, when used thoughtfully, can enhance the way legal professionals work, but to use it effectively we need to better understand the associated risks and rewards. It was with this in mind, that the members of ILTA’s UK Litigation Special Interest Group got together to create the Gen AI Best Practice Guide, an addendum to the Active Learning Best Practice Guide, to assist those litigating in English Courts with the tools they need to use Gen AI, if it is appropriate to do so.
The Genesis of Guidance
The ILTA Gen AI Best Practice guide has been created by a cross-firm collaboration between leading eDisclosure practitioners from UK-based commercial law firms to help encourage the responsible use of this technology, without endorsing any software or provider. The collaborative nature of this effort is significant because it represents a consensus view from practitioners who are actively grappling with these technologies in real-world scenarios. As the Guide states in its introduction:
"The Guide is drafted in the context of supporting document review processes which may be subject to external scrutiny or judicial interrogation, setting out flexible and defensible best practices, and is intended to be both practical and adaptable across a range of matters where the use of GenAI is considered appropriate."
This focus on defensibility and adaptability isn't just academic, it reflects the reality that legal professionals face when adopting new technologies. We need solutions that work not just in theory, but in practice, under scrutiny, and across diverse matter types.
Understanding What GenAI Really Means
Before diving into the decision framework, it's crucial to understand what we're actually talking about. The Guide provides a helpful definition drawn from the UK Courts and Tribunals Judiciary Guidance: GenAI is "A form of AI which generates new content, which can include text, images, sounds and computer code. Some generative AI tools are designed to take actions."
AI has been around for some time and has been used by litigators in the context of disclosure/discovery in the form of Active learning, however, as the Guide explains, "GenAI differs from Active Learning. Active Learning uses deterministic algorithms that are generally publicly available, have been subject to academic research and are tailored to electronic document review tasks. In contrast, GenAI relies on non-deterministic foundation models."
This distinction is critical. With traditional legal technology, we could trace exactly how a decision was made. With GenAI, we're dealing with "foundation models" that are "typically developed and owned by third parties" and "not designed for the purpose of electronic document review." This fundamental difference shapes everything about how we approach GenAI implementation in the context of eDisclosure/eDiscovery and these same principles are applicable when considering the use of Gen AI to assist with any legal task.
The GenAI Decision Framework
Before you even consider implementing GenAI, ask yourself these fundamental questions:
1. What problem are you trying to solve? If you can't articulate a specific challenge that GenAI addresses better than existing solutions, why is Gen AI the tool you’re considering? GenAI excels at pattern recognition, text analysis, and creative problem-solving, but it's not a magic wand for every legal tech challenge.
2. Do you have the infrastructure for responsible deployment? This isn't just about technical capability. It's about having the right safeguards, governance structures and critically, the ability to explain and defend your use of the technology. Are there “technical guardrails," such as "system-level constraints, controls, or safeguards that are implemented within a GenAI workflow to prevent inappropriate, inaccurate, or unverified outputs."
3. Can you maintain appropriate transparency? In legal work, the "black box" problem isn't just inconvenient, it can be case-losing. The Guide stresses that "Parties should seek to put the court in an informed position to be able to exercise necessary supervision of the disclosure process and the parties' compliance with applicable procedural rules and directions." See the Haringey Law Centre case as the most famous UK example of where inadequate supervision was applied resulting in disastrous consequences.
4. Have you considered the cost implications? Different GenAI tools have different pricing structures. Given the rapidly evolving nature of the Gen AI landscape, there is significant volatility in the marketplace meaning that pricing is subject to huge fluctuations, making it very difficult to compare the market and ensure you’re paying a fair price for a product.
The Rewards: Why GenAI Matters
During our work on the ILTA Guide, we identified numerous use cases where GenAI genuinely adds value in eDiscovery. The Guide provides a comprehensive list, including:
1. Processing Stage Applications
Data processing and clean-up: "Enhancing text recognition in documents to enable more effective text-based searching, analytics, and manual review."
Converting documents to text: "Transcribing video, audio or image files into text to enable text-based searches (such as keyword or concept searches) or to support analytics (such as clustering)."
2. Review Stage Applications
Issue identification and categorisation: The Guide explains this as "Using GenAI to identify conceptual issues either prior to or during document review, and to group thematically similar documents. This can assist in surfacing key themes more efficiently and may reveal patterns or connections that might not otherwise have been identified as early, as accurately, or at all."
First-pass relevance review: Here's where GenAI can really shine. The Guide notes: "GenAI may generate a relevance score to offer a rationale for why a document may (or may not) be relevant in response to a given prompt, which could include reference to an exemplar document(s). In these scenarios, the final relevance determination remains with the reviewer."
Privilege identification: "GenAI may assist by identifying key indicators such as the involvement of legal personnel, references to legal terminology, or the context of actual or anticipated litigation. However, privilege decisions should not be left solely to GenAI; practitioner involvement is essential."
Use Cases Beyond eDisclosure
While this Gen AI Guide was written in the context of the application of this technology during disclosure in English courts, its same principles apply when being applied to solve other legal tasks such as:
1. Contract Analysis
GenAI can identify unusual clauses, flag deviations from standard terms, and even suggest negotiation points. The same pattern recognition that helps identify privileged documents can spot non-standard indemnification clauses or unusual termination provisions.
2. Legal Research
Moving beyond Boolean searches to conceptual queries that understand context and nuance. Just as GenAI can perform "Chain of Inquiry Analysis" in eDiscovery "Identifying documents that may not be directly relevant themselves, but which lead to further lines of inquiry" it can identify related cases and statutes that traditional search might miss.
3. Knowledge Management
Automatically categorizing and connecting disparate pieces of institutional knowledge. The Guide's description of "Anomaly and Pattern Detection" for "Detecting outliers and anomalies in communication patterns" applies equally to identifying knowledge gaps or unusual precedent usage.
The Risks: What Keeps Me Up at Night
Let's be frank about the challenges. GenAI introduces risks that traditional legal technology doesn't face:
1. Non-Deterministic Outputs
Unlike traditional software where the same input produces the same output, GenAI can generate different responses to identical prompts. This variability demands rigorous testing and validation protocols: "The practical implications for the purposes of this Guide are that (i) GenAI demands a greater need to consider workflow design and testing; and (ii) it may only be possible to evaluate performance by assessing inputs and outputs, rather than examining the design of the foundation model itself."
2. The Hallucination Problem
GenAI can confidently produce plausible sounding but entirely fabricated information. In legal contexts, this isn't just embarrassing, it's potentially sanctionable. The Guide addresses this through its emphasis on technical guardrails, including "enforcing input/output validation rules" and "requiring a 'nil response' where the system lacks sufficient confidence to generate a reliable output."
3. Confidentiality and Security Concerns
Every interaction with a GenAI system potentially exposes sensitive client data: "The use of GenAI may give rise to confidentiality and information security risks. The Guide does not address those issues in detail. Parties should consult their GenAI / technology provider or other relevant experts to ensure appropriate safeguards are in place."
4. Automation Bias
There's a real risk that reviewers will over-rely on GenAI outputs, especially when under time pressure. The Guide specifically warns: "Parties should also be aware of the risks of automation bias." This is why building in human oversight isn't just best practice, it's essential.
5. Explainability Challenges
The Guide introduces important concepts around transparency: "Parties may be required to explain the intended or actual use of GenAI, such as how the GenAI workflow is designed, operates and has been tested." This requirement for "appropriate explainability" means you need to be prepared to defend your methodology in court.
How to Implement GenAI Responsibly
If you've decided the rewards outweigh the risks, here's your implementation roadmap, drawing heavily from the Guide's practitioner best practices:
1. Start with Governance
Before touching any technology, establish clear policies. "The party using a GenAI workflow should be accountable and responsible for its use throughout." Key governance elements include:
Acceptable use cases clearly defined
Data handling procedures documented
Approval processes for new applications
Audit and documentation requirements specified
2. Design Thoughtful Workflows
The Guide introduces a "GenAI workflow," defined as the “process by which GenAI tools are used to support or automate one or more steps in a disclosure exercise." Your workflow should:
Include multiple validation checkpoints
Maintain clear audit trails (the Guide recommends "maintaining a separate audit log of prompts")
Incorporate feedback loops defined as "a structured process by which insights or results from the review stage are communicated back"
Combine GenAI with proven technologies such as Active Learning to validate results
3. Invest time “Prompt Engineering”
"Iterating prompts to achieve the optimum output is known as prompt engineering." This isn't just about writing good prompts it's about treating prompt design as a core competency:
Document all prompts and their iterations
Test extensively before production use
Maintain prompt repositories for consistency
Train team members on effective prompt creation
The Guide recommends maintaining comprehensive records: "what prompts were used; for what purposes; when; over which part of the dataset; the rationale for each prompt; any amendments to those prompts; and explanations for changes and the timing of those changes."
4. Embrace Transparency
The Guide distinguishes between different types of transparency requirements: " what transparency looks like will depend on the circumstances of a given case." Be prepared to explain:
Why you chose to use GenAI for specific tasks
How your workflows operate
What validation measures you've implemented
Where human oversight occurs
5. Validate, Validate, Validate
Never trust GenAI outputs without verification. Employ these specific validation methods:
"Random Sampling – Reviewing a statistically significant sample of GenAI-classified documents"
"Elusion Testing – Elusion testing should be conducted where applicable"
"SME Testing – Applying independent second-tier review"
"Precision and Recall Metrics – Calculate precision and recall to assess GenAI efficacy"
6. Document Everything
The Guide emphasizes documentation throughout, particularly in the Disclosure Review Document (DRD), which is a component of Practice Direction 57AD (which defines the process for disclosure in the Business and Property Courts of England and Wales): "The intended use of GenAI should be clearly set out in Section 2 of the DRD, including how it will be deployed, its role in the disclosure workflow, and the specific use cases."
Beyond eDiscovery: Lessons for All Legal Tech
While this ILTA Guide is focused on disclosure and document review, the principles apply broadly across all legal technology applications. The emphasis on accountability, transparency, and validation remains constant whether you're analysing contracts, conducting research, or managing knowledge.
The Guide's approach to combining technologies is particularly instructive: "Deploying Active Learning alongside GenAI enhances legal defensibility whilst preserving the efficiency and cost benefits GenAI offers." This hybrid approach, using established technologies to validate and enhance newer ones, provides a template for responsible innovation across all legal tech domains.
The Path Forward
The question isn't really whether to use GenAI, it's how to use it responsibly. The technology is too powerful to ignore, but too risky to deploy carelessly. As the Guide concludes: "These differences should not preclude the use of GenAI, but they do give rise to additional considerations that parties should be aware of and seek to manage appropriately."
My advice?
Start small. Pick a low-risk, high-value use case. Build robust testing and validation protocols. Document everything. Learn from each deployment. And most importantly, never forget that in the practice of law, every action taken may need to be defended in the future. For that reason, you need to consider the use of Gen AI, because to ignore would be negligent, given its potential to drive efficiency and save costs. However, you need to be aware of the risks created by its use and have a plan in place to mitigate them.