Predictive Coding and The Return on
Investment (ROI) of Advanced Review
Strategies in eDiscovery
Drew Lewis eDiscovery Counsel
AGENDA
A Predictive Coding Primer
Predictive Coding and Market Trends
Predictive Coding in Court
Selecting the Right Technology for the Job: Common Use Cases
The ROI of Predictive Coding
The “Hidden” ROI: Strategic Advantages of Technology
THE RECOMMIND STORY
• Founded 2000 • San Francisco (HQ),
Boston, New York, London, Sydney & Bonn
• #163 in Deloitte’s 2012 Technology Fast 500TM
• #10 in Fast Company’s 2013 Most Innovative Companies in Big Data
PREDICTIVE CODING 101
PREDICTIVE CODING DEFINED
People Case Experts
Reviewers
Technology Keyword agnostic analytics
Iterative machine learning
Process Principled, Measured, and Defensible
Statistically certain results
PREDICTIVE CODING BASICS
PREDICTIVE CODING OUTPUTS
Iteration Total Docs Computer Suggested
Percentage Suggested
Docs Reviewed Percentage Reviewed
Responsive Docs
Percentage Responsive of Docs Reviewed
1 948,271 3172 0.335% 3,172 0.33% 2,063 65.04%
2 948,271 1313 0.138% 1,313 0.14% 1,029 78.37%
3 948,271 636 0.067% 636 0.07% 421 66.19%
4 948,271 290 0.031% 290 0.03% 176 60.69%
5 948,271 5039 0.531% 5,039 0.53% 4,671 92.70%
6 948,271 1428 0.151% 1,428 0.15% 1,143 80.04%
7 948,271 687 0.072% 687 0.07% 540 78.60%
8 948,271 2270 0.239% 2,270 0.24% 1,983 87.36%
MARKET TRENDS
MASSIVE GROWTH OF UNSTRUCTURED
CONTENT
0
5,000
10,000
15,000
20,000
25,000
30,000
35,000
40,000
2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020
Exab
yte
s
Structured Data Unstructured Data
Worldwide Corporate Data Growth
Source: IDC. The Digital Universe 2010
80% of Data Growth is Unstructured
CURRENT TRENDS IN-HOUSE
Regulatory/Compliance Litigation: UP
Employment Litigation: UP
Legal Budgets: DOWN
Number one concern: Doing more with
less*
Three most important qualities for outside
counsel*:
˗ Responsiveness
˗ Budget
˗ Speed of work
* Source: ALM Corporate Counsel: Agenda 2013
CURRENT TRENDS
Majority of companies have used
predictive coding
Another 1/3 of companies considering
using predictive coding
Most use is “experimental” – less than
15% “systematic” use
* Source: eDJ Group’s Q1 2013 Predictive Coding Survey, February 2013
JUDICIAL TRENDS
PREDICTIVE CODING’S HOLY TRINITY
In re Actos (Pioglitazone)Products Liability Litigation
(W.D. La. July 27, 2012)
Da Silva Moore v. Publicas Groupe SA, (S.D.N.Y. Feb. 24, 2012)
Kleen Products LLC v. Packaging Corp. of Am. (N.D. III. Sept. 28, 2012)
PREDICTIVE CODING GAINING GROUND
EORHB, Inc., et al. v HOA Holdings, Inc., et. al. (Del. Ch. Ct. Oct 15,
2012)
Robocast, Inc. v. Apple, Inc. (D. Del. 2012)
Chevron Corp v. Donnziger (S.D.N.Y. Mar. 15, 2013)
Harris v. Subcontracting Concepts, LLC (S.D.N.Y. Mar. 11 2013)
COMMON USE CASES
THE RIGHT TECHNOLOGY FOR THE JOB
What is the problem you need to solve?
Expenses
Efficiency
Understand of data
Confidential data
Technology procurement should account for as many problems with as
few solutions when possible
•Check existing coding
•Confirming defensibility
Quality Control
•Hunt for smoking gun
•Who said what and when
Hot Doc Identification
• Identify custom document types for special handling
•Source code identification
•Potentially personal information (PPI)
Custom Use Cases
•Prioritize
•Minimize review population
•Confirm defensibility
Responsive Review
• Internal or regulatory investigation
• Identify potential topics and key facts
• Identify key docs in opposing counsel production
Investigative Workflow
THE ROI OF PREDICTIVE
CODING
CASE STUDY
Initial Review (linear):
33% cull rate (reduced to 68 GB)
679,349 documents for review
Approximately 47.5 decisions/hr.
14,302.1 hours needed for review
Contractor rate of $55/hr. (first
pass only)
$786,641.63 for first pass review
CASE STUDY (Cont’d)
Second Review (Predictive Coding):
92% cull rate (reduced to 17 GB)
22% reviewed
Approximately 32.5 decisions/hr.
1150.8 hours needed for
complete review (less validation
phase)
SME rate $500/hr.
$575,384.62 for Complete Review SAVINGS: $211,983.81
THE “HIDDEN” ROI
STRATEGIC THINKING
Increased and better visibility into data set
Increased speed in identification of pertinent documents
Increased level of information and understanding of unreviewed
documents
Increased level of information and understanding of document content
without granular document review
Converts reviewers into knowledge workers
QUESTIONS & DISCUSSION