Tracking Metrics for Fun and Profit

Thursday, January 5, 2017

Every eDiscovery practitioner has experienced the reactive urgency of a sudden and overwhelming discovery project.  Indeed, many spend long periods of time buffeted from one to the next. Projects can be completed this way, obligations met, but often only at great cost and with great effort. And, so, every eDiscovery practitioner has also felt the desire for more control, more insight, more efficiency.

Business intelligence is an umbrella term used to refer to the strategies, processes, and tools that businesses employ to extract knowledge and value from their vast oceans of structured and unstructured data. While the data volumes in eDiscovery are smaller, many of the approaches are equally applicable, and applying those approaches can yield more control, insight, and efficiency.

This month, in this short series, we will review some core business intelligence concepts and how they can apply during an eDiscovery project, across an eDiscovery project, and beyond. Additionally, on Wednesday, January 25th, at 1:00 PM EST, Advanced Discovery will host a free, one-hour webinar on the topic as well to provide additional insights and an opportunity for questions.  You can reserve your free seat now by clicking here.

In this first Part, we’ll begin by reviewing business intelligence and how it can apply during an eDiscovery project.

Business Intelligence

Although the phrase did not become widespread until the 1990’s, the phrase business intelligence dates back to at least 1865, and its modern incarnation has its roots in the 1950’s and 1960’s when the rise of computers and electronic databases created opportunities for new kinds of record keeping, reporting, and analysis. Today, Gartner defines business intelligence as including “the applications, infrastructure and tools, and best practices that enable access to and analysis of information to improve and optimize decisions and performance” (emphasis added).

Regardless of the specific business, technology, or industry context, business intelligence is about three core activities:

  1. Tracking, recording, aggregating, and storing relevant data
  2. Mining and analyzing those data stores for trends and insights
  3. Applying those insights to decision-making and process refinement

In the general business context, relevant data might include structured data such as that about customers, sales, the market, finances, operations, enterprise resource management, and more, as well as unstructured data such as written reports, emails, and other communications. Together, these abundant sources can represent enormous stores of data often aggregated in a central “data warehouse” for reporting and analysis.

Mining and analyzing those data stores is accomplished through reporting functions and through quantitative and qualitative analysis functions. Reporting functions are for providing accurate, timely information about the current state, while analysis functions are for surfacing trends and patterns in historical and current data to inform future decisions. Analysis functions may include both quantitative analysis functions for structured data as well as semantic analysis functions for unstructured data.

Applying those insights is both a tactical and a strategic exercise. Tactically, accurate reporting of timely, relevant details about organizational projects and processes facilitate effective day-to-day management and decision-making. Strategically, those tactical efforts can be integrated into a formal “continuous improvement program,” and the larger trends identified (both internally and externally) can guide higher-level organizational decision-making.

During an eDiscovery Project

In the context of an eDiscovery project, these activities can also be undertaken and a business intelligence approach employed. Within an individual project, there are many opportunities to track and aggregate relevant data, to generate reporting from and analysis of that data, and to leverage those reports and analyses to inform your decisions and improve your process. This is true for both the substantive and formal elements of the eDiscovery project:

  • The substantive elements of an eDiscovery project are the actual sources being collected, the materials obtained, and the legal and organizational information that can be gleaned from them. The substantive elements implicate both the unstructured (and occasionally structured) data collected for the project, as well as some of the structured data about that data and those sources.  These unstructured materials are already mined and analyzed to prepare the legal strategy and generate responsive productions.
  • The formal elements of an eDiscovery project are the tools chosen, the individuals involved, and the processes used. The formal elements implicate structured data generated by the discovery activities themselves: data about the data and its processing; data about the searches and filters used; data about the reviewers and the review effort; etc. These can be mined and analyzed to make your project more effective and efficient.

Taking a business intelligence approach to an individual eDiscovery project means thinking ahead about the formal elements of the project to identify key metrics that will be of use to you and the processes or tools that can track those metrics and support the reporting and analysis you need.

For example, early in an eDiscovery project, careful tracking of where relevant or hot materials are being found, how file types are distributed, or which custodians are in communication with which others can all be leveraged to inform searching and filtering strategies or to determine optimal downstream prioritization and organization for review. Later in a project, tracking detailed performance metrics for both individual reviewers and different review activities (e.g., first pass, quality control, redaction, privilege logging) can be leveraged to identify bottlenecks or weak links early and respond, to maximize the efficiency and efficacy of the review process.

To take this approach and reap these benefits, you must think ahead about what metrics you want to track and how you are going to track them:

  • What do you need and want to know?
  • At what level of granularity do you need to know it?
  • How and how often should reporting take place?
  • What thresholds for alerts should be set?

Modern review tools, like the current version of kCura’s Relativity, have incorporated robust tools to facilitate tracking project metrics and reporting on them, and eDiscovery service providers, like Advanced Discovery, have extended and customized those capabilities to maximize their flexibility and utility.

In the next Part of this series, we will turn to the tracking and leveraging of such data across projects for eDiscovery program management, which is where more significant benefits start to accrue to an organization.




VP, Marketing Content
Advanced Discovery

Matthew Verga is an electronic discovery expert proficient at leveraging his legal experience as an attorney, his technical knowledge as a practitioner, and his skills as a communicator to make complex eDiscovery topics accessible.  A nine-year industry veteran, Matthew has worked across every phase of the EDRM and at every level from the project trenches to enterprise program design.  As VP, Marketing Content, for Advanced Discovery, he leverages this background to produce engaging educational content to empower practitioners at all levels with knowledge they can use to improve their projects, their careers, and their organizations.

More Posts

View all Posts


    Subscribe to receive our
    Experts’ Insights Blog feed.

  • Get in Touch