Pyrrho Investments: English Predictive Coding Case Strikes New Balance in E-Discovery

On 16 February 2016, the English High Court sent a clear signal approving the use of predictive coding technology in electronic discovery (“e-discovery”). In Pyrrho Investments Ltd & Anor v MWB Property Limited & Ors [2016] EWHC 256 (Ch) (“Pyrrho”), Master Matthews provided directions on predictive coding and the suitable conditions in which it would be applied to expedite disclosure processes. Predictive coding software has been available commercially for several years to meet the exponential growth in documents handled in high volume, complex e-discovery. However, its slow adoption outside the US has been attributed to a lack of judicial guidance on the use of the technology in litigation. Given the English influence on Hong Kong’s discovery rules and e-discovery practice direction (“Pilot Scheme for Discovery and Provision of Electronically Stored Documents in Cases in the Commercial List”, Practice Direction SL1.2), this decision should bolster Hong Kong lawyers’ confidence in using predictive coding and similar technologies in dispute resolution.

Predictive Coding Explained

Predictive coding sorts electronic documents by using machine-learning algorithms, identifying and prioritising documents responsive to certain issues of a case for human review. It is a type of technology-assisted review (“TAR”) that significantly reduces review time and the number of non-responsive documents requiring review (the court in Pyrrho appears to refer to “predictive coding” and “TAR” interchangeably, which is not correct. The former concept is a subset of the latter). This results in substantial cost savings and reduction of human error.

Predictive coding software “learns” iteratively, adding to its artificial intelligence programming. Various predictive coding products on the market have different learning strategies and workflows, one version of which is referenced in Pyrrho (paras. 17–24). It is broken down as follows:

  • Firstly, it scans the documents in its database and classifies them according to common concepts and phrases.
  • Secondly, it selects a sample document set for manual categorisation by a lawyer familiar with the issues. The software categorises a new sample set by extrapolating logic from the human input.
  • Thirdly, the lawyer would check the software’s decisions to see if it has attained an acceptable level of accuracy (the tolerance level), and correct any erroneous categorisation.
  • Fourthly, this training process would be repeated until the tolerance level when the software would then categorise the remaining documents in its database.

Summary of Pyrrho

The underlying action in Pyrrho relates to breaches of fiduciary duties by directors, which led to interlocutory proceedings on the discovery of back-up tapes controlled by one of the claimants. The tapes contained over 17.6 million electronic files from four of the defendants’ email accounts. The volume is reduced to about 3.1 million files using an electronic de-duplication technique. The parties agreed to employ predictive coding to expedite the disclosure of 3.1 million files and reduce the costs of manual document review, and sought the court’s approval for the agreement.

In endorsing the use of predictive coding, Master Matthews enumerated ten guiding factors for his decision; the substance of which is distilled as follows:

  • the predictive coding methodology makes the document review more consistent, and is as accurate as, if not more accurate than, manual review alone or manual review combined with keyword searches;
  • the number of potential documents for review is “huge”, and the costs of reviewing them manually would be “enormous”;
  • the costs of using predictive coding are proportionate to the value of the claim;
  • there is sufficient time to consider contingent search methods should the use of predictive coding fail to generate satisfactory results; and
  • the parties agreed on the use of predictive coding.

Legal Analysis in the Case

The reasoning of Pyrrho focuses on: (a) the overriding objectives and the criteria for a reasonable search in disclosures under the English Civil Procedure Rules 1998 (“CPR”), respectively CPR 1.1 and 31.7; and (b) Practice Direction 31B (the English Practice Direction on Disclosure of Electronic Documents). Among other things, CPR 1.1(1) emphasises that the court must be able “to deal with cases justly and at proportionate cost”. CPR 31.7(2) also provides factors for determining the reasonableness of a search. These factors are: number of documents; nature and complexity of proceedings; ease and expense of retrieving any particular document; and significance of any document likely to be located during the search. Practice Direction 31B elaborates on these factors at paragraphs 21(3)(c) and 21(3)(e), and states that keyword searches or “other automated methods of searching” may be reasonable “if a full review of each and every document would be unreasonable.” In Goodale v Ministry of Justice [2009] EWHC B41 (QB), Master Whitaker only referred to TAR software in passing.

While the CPR and Practice Direction 31B do not prohibit the use of predictive coding (although they do not expressly provided for it), Master Matthews found that there was no English legal authority on when or how it should be deployed and how the principle of proportionality should apply to it. The Pyrrho decision also notes that finding search methods with suitable scope and quality could have a much greater impact on the time and costs of e-discovery than hardcopy discovery. This is because extensive e-discovery exercises tend to involve battalions of lawyers and paralegals reviewing tens to hundreds of thousands of documents.

Master Matthews relied on US and Irish judgments for persuasive guidance. The US District Court in Moore & Ors v Publicis Groupe SA & Anor 11 Civ 1279 (ALC)(AJP) found that the use of predictive coding was more appropriate than manual review with keyword searches in that case, while predictive coding would entail lower costs and inconsistencies. The plaintiffs raised several issues, including whether predictive coding would allow the defendants’ lawyers to certify discovery as complete when it was not; whether it was contrary to the Federal Rules of Evidence; and whether it was impossible to assess whether predictive coding would produce accurate results. Magistrate Judge Peck dismissed these issues, and the ultimate decision was affirmed on appeal. The Irish High Court in Irish Bank Resolution Corporation Ltd v Quinn [2015] IEHC 175 held that “predictive coding is at least as accurate as, and, probably more accurate than, the manual or linear method in identifying relevant documents” in large data sets, and that TAR was “more expeditious and economical” than manual review.

Impact of the Decision

The Pyrrho decision represents a cautious and balanced approach to incorporate predictive coding techniques into existing civil procedures, and offers guidance on the consideration of TAR in case management proceedings. It shows the English courts’ acceptance of an evolving concept of “reasonable search” that embraces advanced software technology. In particular, it acknowledges that predictive coding is at least as accurate as, if not more accurate than, the traditional manual review with keyword search. The decision notes that predictive coding could significantly reduce inconsistencies and costs.

However, the decision does not provide litigants carte blanche use of predictive coding technology in discovery. Predictive coding has some shortcomings and is not always the proper e-discovery tool. Master Matthews noted that manual review might be needed after predictive coding software “has done its best”, and predictive coding would not always guarantee low costs. One of the factors for endorsing predictive coding is that there would be sufficient time to rectify the search if predictive coding generated unsatisfactory results. The Pyrrho decision also indicates the importance of party consensus in the use of predictive coding, as litigants need to consider and agree to search parameters and other methodological considerations before deploying the technology. Indeed, whether predictive coding is appropriate in a particular case may depend on how it is used. An English court might therefore be reluctant to grant a unilateral request to use predictive coding.

It is likely that Hong Kong courts would adopt Pyrrho should a similar request for predictive coding arise in Hong Kong proceedings. Practice Direction SL1.2 is not only based on the English Practice Direction 31B, but also goes further to spell out “technology assisted reviews” in its provisions on “other automated searches”

(paras.9(3)(b) & 24). Nonetheless, Practice Direction SL1.2 currently provides a pilot scheme that is limited to cases on the commercial list. Moreover, the Court of First Instance in Chinacast Education Corporation & Ors v Chan & Ors [2014] 5 HKC 277 confirmed that requests for e-discovery under O.24 of the Rules of High Court (Cap.4A) (“RHC”) must be proportionate, economical and relevant (the case preceded Practice Direction SL1.2, which became operational on 1 September 2014). This follows the underlying objectives of the RHC (O.1A, r. 1), which provides for increasing “cost-effectiveness” and promoting “a sense of reasonable proportion and procedural economy”. English and Hong Kong civil procedures share many points of compatibility, and provide fertile legal environments to foster the use of predictive coding.


Increasingly, litigants cannot cope with the sheer document volume of massive e-discovery exercises and the associated time and costs using traditional search and review techniques. The Pyrrho court’s endorsement of predictive coding is timely and adds much-needed momentum to the global growth of TAR. The judicial guidance offers comfort to lawyers conducting complex cases, and encourages the use of automated search technologies in litigation. What remains to be seen is how the legal community will adapt its practices and workflows to leverage use of such technologies in more efficient and effective ways. Overall, Pyrrho is welcome news for Hong Kong where external and in-house legal teams are generally smaller than their counterparts in the US and the UK and have to do more with less.


Co-Founder & Chief Operating Officer, DHB Global (Hong Kong, CHN)

Sebastian is a lawtech and regtech expert. He was formerly senior legal counsel and Asia regional head of e-discovery review at a global leading legal technology solutions company. Previously, he practised financial regulatory law and commercial dispute resolution in international firms. He is a community organiser of lawtech and techlaw events, including the first Access to Justice Hackathon in Asia. He is a member of the InnoTech Committee of the Law Society of Hong Kong. He holds degrees in science and law, including the Bachelor of Civil Law (Oxon), and is legally qualified in Hong Kong, New York and at the U.S. Supreme Court.

Reuters News

Manager, Epiq Systems

Mr. Yuen is a manager of Epiq’s document review services in Asia, focusing on the Greater China market and supporting eDiscovery projects regionally, and is admitted to practice in California (USA). Epiq serves law firms, corporations, financial institutions and government agencies with 30 locations worldwide.