A discussion of applying business intelligence concepts to eDiscovery project and program management to extract knowledge and value from discovery project data for the benefit of individual projects, overall programs, and your whole organization
In the first Part of this short blog series, we began by reviewing core business intelligence concepts and how they can apply during an individual eDiscovery project. In Part 2, we review how they apply across individual eDiscovery projects as part of effective eDiscovery program management.
Additionally, on Wednesday, January 25th, at 1:00 PM EST, Advanced Discovery will host a free, one-hour webinar on this topic to provide additional insights and an opportunity for questions. Reserve your free seat now by clicking here.
Moving Beyond the Individual Project
As we noted, each individual eDiscovery project provides opportunities for tracking and aggregating useful metrics about both substantive elements of the project (i.e., sources, materials, and their contents) and formal elements of the project (i.e., tools, individuals, and their processes). Tracking these metrics can provide insights that lead to improvements in efficiency and efficacy during a project, but there is third category of macro elements about which we can also track metrics to facilitate overall program management.
Macro elements are those that are unlikely to have utility during an individual project but that, if tracked, can still lead to improvements across projects. For example, the following are all macro elements about which metrics might be tracked to aid in program management:
- Pre-Discovery Negotiation Efforts and Their Effects
- Internal/External Resource Selection and Allocation
- Relative Efficacy of Different ECA and Review Strategies
- Costs, Broken Down by Resource, Process, and More
- Duration and Throughput for Resources and Processes
- Overlaps in Matters’ Scope, Implicated Sources, Etc.
Beyond these specific examples, tracking key metrics across projects, in an organized, aggregated way, makes it possible to establish efficiency, efficacy, and cost benchmarks for your processes and activities. And, from those benchmarks, concrete goals for iterative improvement can then be set.
Macro Metrics and Their Program Management Applications
Next, let’s review some of the major subcategories of macro, cross-project metrics that you might track and examples of the benefits that tracking them can yield:
Data Metrics: This subcategory of metrics covers information about your data and where it comes from, including: sources; source types; individual custodians; departmental custodians; mobile device sources; cloud sources; social media sources; file types; and, volumes for all of these and more. Tracking this data across projects can yield a variety of potential benefits:
- More accurately predict your collection resource needs for projects and the future
- Identify overlapping or repeated collections that can be combined or streamlined
- Compare relative cost/efficiency of various tools, techniques, and service providers
Processing Metrics: This subcategory of metrics covers information about your processing activities, including: platforms and tools used; throughput and error rates achieved; variations in completion time for various source types; filtering techniques employed and their rates of volume reduction; and, hours of employee work required per job and more. Tracking this data across projects can also yield a variety of potential benefits:
- Evaluate the relative performance of different tools on speed and accuracy
- Better predict the time and resources required for any given processing job
- Measure the efficacy of objective filtering techniques for various purposes
ECA Metrics: This subcategory of metrics covers information about your early case assessment activities, including: platforms and tools used; search and analytic features applied; volume reductions achieved; time spent; and, reviewers required and more. Tracking this data across projects can also yield a variety of potential benefits:
- Learn what approaches work best for what source or case types
- Compare ECA data volume reductions with review results to benchmark ECA
- Analyze assorted analytic tools for relative time required and benefit achieved
Review Metrics: Within a single eDiscovery project, review metrics are invaluable for team management and oversight, but across projects additional variables can be tracked, including: platforms and tools used; batch sizes and batch organization used; review team sizes and team organization used; application of e-mail threading and near-duplicate grouping; privilege logging and redaction workflows employed; and, details of any TAR process employed. Tracking this data across projects can also yield a variety of potential benefits:
- Measure the comparative efficiency of various team structures and workflows
- Discover optimal batch size and document organization strategies for your reviews
- Answer for your organization the question of whether and when TAR is worth it
Cross-Phase Metrics: This subcategory of metrics covers additional useful information that cuts across several or all phases of an eDiscovery project, including: matter type; jurisdiction; value at risk; outcome achieved; total project costs and total costs per document ultimately produced; internal resources and outside service providers employed; and, overall ratio of documents collected to documents produced and more. Tracking this additional data across projects is invaluable for overall program management and can yield a variety of potential benefits:
- Spot trends in your litigation portfolio that can aid in program planning and budgeting
- Establish overall program benchmarks for cost-per-document, collection efficiency, etc.
- Determine which tasks and phases are best handled internally and which externally
As you can see from this list of examples, the range of things you might track project-to-project is vast, and the range of insights you might gain is equally so. A key question for each organization will be which metrics are worth the cost and trouble of tracking, and that will be dictated, in part, by your long-term goals for the effort.
In the next Part of this short series, we will review some potential program goals for the tracking and aggregating of such metrics data, including continuous improvement programs and risk management.
MATTHEW VERGA, JD
VP, Marketing Content
Matthew Verga is an electronic discovery expert proficient at leveraging his legal experience as an attorney, his technical knowledge as a practitioner, and his skills as a communicator to make complex eDiscovery topics accessible. A nine-year industry veteran, Matthew has worked across every phase of the EDRM and at every level from the project trenches to enterprise program design. As VP, Marketing Content, for Advanced Discovery, he leverages this background to produce engaging educational content to empower practitioners at all levels with knowledge they can use to improve their projects, their careers, and their organizations.