Monday, May 13, 2013
This is a Bloomberg BNA Insight article written by Chris Cameron. To download the Bloomberg article in pdf form press here.
As new technologies and processes are developed to deal with increasing volumes of electronically stored information (ESI), it has become apparent to even novice legal and discovery professionals that the key to remaining in control is information. Until recently, however, once a project began, they lost the ability to monitor the process on a daily basis and decide whether a change in tactics was necessary. Instead of having their finger on the pulse of the project, in-house counsel and law firms would wait for secondhand reports to learn critical information.
Today’s best discovery platforms deliver metrics in real time to allow counsel to gain deep insight into how a discovery or review project is progressing. Counsel make changes that lead to continual improvement and also capture lessons learned about the organization, custodians, and the documents themselves to apply to future projects.
Leveraging the power of information can improve speed and accuracy in each phase of discovery while reducing costs. Firms and their clients can also evaluate data across multiple matters simultaneously, expanding their ability to optimize the discovery process from one project to an entire group of related cases. This article examines the types of information that counsel should have at their fingertips to optimize the discovery project, the matter as a whole, and the overall litigation portfolio.
An instant supply of data allows counsel to use statistical analysis to accelerate the process of assessing potential risks, decide on an appropriate case strategy, and gauge the likelihood of success. Counsel can make a more informed decision about whether to pursue or settle the matter and, in the case of settlement, avoid the need for additional collection, processing, and review. Without this information, counsel might continue to litigate, only to later determine that the case may not be defensible or may be too expensive to carry through to trial.
Throughout the course of the matter, a consistent flow of accurate information allows in-house counsel to explain to executives what is happening, as predictability is an absolute requirement in a corporate setting. As discovery progresses, counsel can monitor the number of custodians, the amount of data that has been reviewed and that remains for review, and the estimated date of completion for various segments of the project.
Occasional snapshots of data are not enough to lend insight, because many of these metrics are moving targets: custodians may be added, new data sources may be found, keywords may change, and the review may progress quicker or slower than expected. Basically, anything that can change does change.
With a continual stream of data at their fingertips, counsel can make adjustments on the fly—before it is too late to correct the course—if a case appears to be running ahead of budget or behind schedule. Information gives them the power to determine whether their initial case strategy is working or whether they might need to reconsider their tactics.
Good information can inform budgeting for both discovery and the matter as a whole. With instantaneous access to the estimated cost of the process, corporations have better vision of whether their current budgets are realistic and make adjustments as necessary. Through continuously updated information on the projected costs of discovery, counsel can make the optimal decision about settlement. They can also use this information offensively, to, for example, argue for narrowing the scope of discovery or the shifting of costs.
Historical case information can also support the negotiation of reasonable rates for law firms and service providers as well as forecast the budget for future projects. By analyzing the cost per document or gigabyte of past reviews, combined with measures of review rate and review accuracy, counsel can choose the most efficient law firms and review teams for their projects.
Knowing the types of data and the format of that data early in a matter helps counsel develop a process and technology to handle the matter, including whether they need to explore additional tools to handle their specialized needs. Without knowing the contents of the data set up front, corporations and their counsel might spend valuable resources on a platform or process that is not right.
Prior to the commencement of the review stage, data can also inform counsel on whether they have collected the right information to support a matter. Documents can be analyzed as they are identified to reach conclusions on which custodians will have the most important data, determine the keywords likely to yield the most relevant results, and project the amount of data that will continue to downstream discovery processes.
Counsel should obtain statistics about the progress of their data through the various phases of discovery, in real time. For example, they should review the number or volume of documents preserved against those collected or reviewed to determine a realistic project timeline. They should also learn how much of the data has been removed from processing through techniques such as culling and deduplicating. Finally, they should consider how much data has been loaded and is ready for review—and how quickly it will be reviewed—to ensure that they have devoted enough resources to the project and have allocated them appropriately. Real-time review metrics can also improve predictability as counsel can gauge whether they are on track to meet their deadlines on a regular basis. If there is a bottleneck in the process, counsel can find and address it quickly, before it has a detrimental effect on the overall timeline.
Based on the initial evaluation of the document pool, it may be advisable to allocate the custodians with the fewest relevant documents to lower-cost resources, such as less experienced reviewers, while sending the most critical and complex documents to more seasoned professionals. An analysis of the percentage of responsive documents found in each custodian’s files is the key data point.
In addition, counsel can determine whether the document collection includes documents written in a foreign language—or contains other oddities—that may require reviewers with specialized skills.
With real-time dashboards that illustrate team and project performance, counsel can decide whether the review is staffed with the right mix of people. For example, if the review is progressing too slowly based on the documents-per-hour metric, then more reviewers may need to be added; if too many documents are kicked out as ‘‘exceptions,’’ the makeup of the team can be changed accordingly.
Study metrics that show how quickly each reviewer is progressing through a batch of documents and adjust accordingly. This allows less efficient reviewers to be retrained to improve their performance, or if necessary, replaced. Another metric reveals the accuracy of each reviewer’s coding, allowing counsel the insight to compare reviewer performance, determine the source of errors, and, when necessary, remove or retrain reviewers who lack the appropriate knowledge or skills from the project.
Using data to create a higher-quality review team leads to more accurate review results. In turn, review teams will need to spend fewer hours rereading the same documents to correct mistakes during the quality control process.
Downstream, it reduces the likelihood of disputes with opposing counsel that might trigger expensive motions to compel and motions for sanctions, as well as discovery delays that will anger judges. Therefore, data that reveals the accuracy of each reviewer’s coding can help improve defensibility.
A technology-assisted review (TAR) engine learns based on an iterative process. That means the more data the review software ingests, the better results it produces. As the review progresses, TAR’s predicted results can be compared (daily, if necessary) to humancoded documents to improve the results on both ends: correcting discrepancies in coding can improve the TAR process as well as point out reviewers who may need retraining or removal.
Multiple cases often touch on the same issues and involve the same documents and custodians. Data stored in a centralized repository for use in similar future matters can save the costs of collecting, processing, and reviewing the data for the next project.
In addition, reusing this data can help ensure consistency in the treatment of data when used in multiple cases handled by different law firms. This is particularly critical when documents contain sensitive information or legal advice: recycling data that has already been vetted and redacted (if necessary) can improve the chances that confidentiality, the attorney-client privilege, and the work product doctrine are not inadvertently waived. This historical data can also help predict costs, risks, and results more reliably.
For many, access to information is like a shiny new toy. It’s fun to play with right out of the box, but eventually it’s important to learn what it really can do. There are so many elements to juggle: custodians, documents, data, time frames, reviewers, tools, data sources, and more. Good information can help refine review results, improve consistency, and save costs.
However, metrics are only useful when delivered in a unified platform capable of tracking all data points in real time. Tracking software must be convenient and simple to use. Organizations and their counsel should not have to wait for a project manager or administrator to deliver information—which, by the time it arrives, may be outdated.
— Chris Cameron is Client Services Manager at Advanced Discovery, where he oversees the Client Services and Hosting Operations Teams, which provide project management and technical support for ESI, forensics, and hosting matters. He has been providing project management in litigation support and eDiscovery for more than 10 years and has experience leveraging information to manage very large and complex projects in a variety of industries and practice areas.