The Experts below are selected from a list of 27318 Experts worldwide ranked by ideXlab platform
Maura R. Grossman - One of the best experts on this subject based on the ideXlab platform.
-
evaluation of machine learning protocols for technology assisted review in Electronic Discovery
International ACM SIGIR Conference on Research and Development in Information Retrieval, 2014Co-Authors: Gordon V. Cormack, Maura R. GrossmanAbstract:Abstract Using a novel evaluation toolkit that simulates a human reviewer in the loop, we compare the effectiveness of three machine-learning protocols for technology-assisted review as used in document review for Discovery in legal proceedings. Our comparison addresses a central question in the deployment of technology-assisted review: Should training documents be selected at random, or should they be selected using one or more non-random methods, such as keyword search or active learning? On eight review tasks -- four derived from the TREC 2009 Legal Track and four derived from actual legal matters -- recall was measured as a function of human review effort. The results show that entirely non-random training methods, in which the initial training documents are selected using a simple keyword search, and subsequent training documents are selected by active learning, require substantially and significantly less human review effort (P<0.01) to achieve any given level of recall, than passive learning, in which the machine-learning algorithm plays no role in the selection of training documents. Among passive-learning methods, significantly less human review effort (P<0.01) is required when keywords are used instead of random sampling to select the initial training documents. Among active-learning methods, continuous active learning with relevance feedback yields generally superior results to simple active learning with uncertainty sampling, while avoiding the vexing issue of "stabilization" -- determining when training is adequate, and therefore may stop.
-
Evaluation of machine-learning protocols for technology-assisted review in Electronic Discovery
Proceedings of the 37th international ACM SIGIR conference on Research & development in information retrieval - SIGIR '14, 2014Co-Authors: Gordon V. Cormack, Maura R. GrossmanAbstract:Using a novel evaluation toolkit that simulates a human reviewer in the loop, we compare the effectiveness of three machine-learning protocols for technology-assisted review as used in document review for Discovery in legal proceedings. Our comparison addresses a central question in the deployment of technology-assisted review: Should training documents be selected at random, or should they be selected using one or more non-random methods, such as keyword search or active learning? On eight review tasks -- four derived from the TREC 2009 Legal Track and four derived from actual legal matters -- recall was measured as a function of human review effort. The results show that entirely non-random training methods, in which the initial training documents are selected using a simple keyword search, and subsequent training documents are selected by active learning, require substantially and significantly less human review effort (P
Gordon V. Cormack - One of the best experts on this subject based on the ideXlab platform.
-
evaluation of machine learning protocols for technology assisted review in Electronic Discovery
International ACM SIGIR Conference on Research and Development in Information Retrieval, 2014Co-Authors: Gordon V. Cormack, Maura R. GrossmanAbstract:Abstract Using a novel evaluation toolkit that simulates a human reviewer in the loop, we compare the effectiveness of three machine-learning protocols for technology-assisted review as used in document review for Discovery in legal proceedings. Our comparison addresses a central question in the deployment of technology-assisted review: Should training documents be selected at random, or should they be selected using one or more non-random methods, such as keyword search or active learning? On eight review tasks -- four derived from the TREC 2009 Legal Track and four derived from actual legal matters -- recall was measured as a function of human review effort. The results show that entirely non-random training methods, in which the initial training documents are selected using a simple keyword search, and subsequent training documents are selected by active learning, require substantially and significantly less human review effort (P<0.01) to achieve any given level of recall, than passive learning, in which the machine-learning algorithm plays no role in the selection of training documents. Among passive-learning methods, significantly less human review effort (P<0.01) is required when keywords are used instead of random sampling to select the initial training documents. Among active-learning methods, continuous active learning with relevance feedback yields generally superior results to simple active learning with uncertainty sampling, while avoiding the vexing issue of "stabilization" -- determining when training is adequate, and therefore may stop.
-
Evaluation of machine-learning protocols for technology-assisted review in Electronic Discovery
Proceedings of the 37th international ACM SIGIR conference on Research & development in information retrieval - SIGIR '14, 2014Co-Authors: Gordon V. Cormack, Maura R. GrossmanAbstract:Using a novel evaluation toolkit that simulates a human reviewer in the loop, we compare the effectiveness of three machine-learning protocols for technology-assisted review as used in document review for Discovery in legal proceedings. Our comparison addresses a central question in the deployment of technology-assisted review: Should training documents be selected at random, or should they be selected using one or more non-random methods, such as keyword search or active learning? On eight review tasks -- four derived from the TREC 2009 Legal Track and four derived from actual legal matters -- recall was measured as a function of human review effort. The results show that entirely non-random training methods, in which the initial training documents are selected using a simple keyword search, and subsequent training documents are selected by active learning, require substantially and significantly less human review effort (P
Kristen A. Knapp - One of the best experts on this subject based on the ideXlab platform.
-
Enforcement of U.S. Electronic Discovery Law Against Foreign Companies: Should U.S. Courts Give Effect to the EU Data Protection Directive?
Richmond journal of global law and business, 2010Co-Authors: Kristen A. KnappAbstract:Enforcing Discovery against companies located in foreign nations is not a new phenomenon. The U.S. Supreme Court took up the conflict between U.S. Discovery rules and foreign non-disclosure law in a 1958 case. Despite more than fifty years to reach a settled jurisprudence regarding how to enforce U.S. law against foreign domiciled companies, there has yet to be a clear articulation of a standard applicable in all cases. Currently, there are two main sets of rules under which U.S. courts may enforce Discovery laws against foreign companies, and if necessary impose sanctions for non-compliance: the Hague Convention and the U.S. Federal Rules of Civil Procedure. The trend of authority favors the use of the U.S. Federal Rules of Civil Procedure, but there remain some circumstances under which the Hague Convention is favored. In 2006, amendments to the U.S. Federal Rules of Civil Procedure concerning Electronic Discovery (“e-Discovery”) procedures went into effect. These amendments have had and will continue to have a significant impact on the conduct of business both abroad and within the United States. Accordingly, “[m]ore and more companies with global operations are finding themselves enmeshed in e-Discovery that requires a greater understanding of the issues and laws from a global perspective” because “[i]t is challenging to navigate and manage e-Discovery when you have parent companies based overseas or U.S.-based companies with foreign subsidiaries.” However, U.S. courts have yet to systematically address what effect, if any, the 2006 amendments will have on enforcement of e-Discovery law against foreign domiciled companies, and in particular, against European companies. Surprisingly, to date, there is very little case law regarding the enforcement of e-Discovery production requests.
-
Enforcement of U.S. Electronic Discovery Law Against Foreign Companies: Should U.S. Courts Give Effect to the EU Data Protection Directive?
SSRN Electronic Journal, 2010Co-Authors: Kristen A. KnappAbstract:Although the U.S. Supreme Court first considered the conflict between U.S. Discovery rules and foreign non-disclosure law in 1958, a clear standard regarding how to enforce U.S. law against foreign domiciled companies has yet to emerge. As a result of the 2006 ammendments to the U.S. Federal Rules of Civil Procedure concerning Electronic Discovery (“e-Discovery”) procedures “[m]ore and more companies with global operations are finding themselves enmeshed in e-Discovery that requires a greater understanding of the issues and laws from a global perspective” because “[i]t is challenging to navigate and manage e-Discovery when you have parent companies based overseas or U.S.-based companies with foreign subsidiaries.”This paper looks at, in light of the 2006 amendments and the lack of case law regarding the affect of the 2006 amendments, whether the enforcement techniques, as applied to “paper” Discovery should be applied to e-Discovery and whether there is anything specific to the nature of e-Discovery that necessitates a change in the application of the law. Specifically, the paper addresses how the European data privacy regime may affect the application of paper Discovery enforcement techniques to e-Discovery. The paper suggests that it would be unwise for U.S. courts to afford the European Data Privacy regime significant deference. Instead, the European Data Privacy regime should be treated with skepticism, similarly to how the U.S. courts have viewed “blocking statutes” contained in foreign law. In particular, treating the EU Data Privacy regime with skepticism will help to prevent the creation of perverse incentives for companies to store their data abroad that hope to avoid legitimate Discovery production requests under the Federal Rules of Civil Procedure, by raising the transaction costs for such behavior.
Patrick Oot - One of the best experts on this subject based on the ideXlab platform.
-
document categorization in legal Electronic Discovery computer classification vs manual review
Journal of the Association for Information Science and Technology, 2010Co-Authors: Herbert L Roitblat, Anne Kershaw, Patrick OotAbstract:In litigation in the US, the parties are obligated to produce to one another, when requested, those documents that are potentially relevant to issues and facts of the litigation (called “Discovery”). As the volume of Electronic documents continues to grow, the expense of dealing with this obligation threatens to surpass the amounts at issue and the time to identify these relevant documents can delay a case for months or years. The same holds true for government investigations and third-parties served with subpoenas. As a result, litigants are looking for ways to reduce the time and expense of Discovery. One approach is to supplant or reduce the traditional means of having people, usually attorneys, read each document, with automated procedures that use information retrieval and machine categorization to identify the relevant documents. This study compared an original categorization, obtained as part of a response to a Department of Justice Request and produced by having one or more of 225 attorneys review each document with automated categorization systems provided by two legal service providers. The goal was to determine whether the automated systems could categorize documents at least as well as human reviewers could, thereby saving time and expense. The results support the idea that machine categorization is no less accurate at identifying relevant-responsive documents than employing a team of reviewers. Based on these results, it would appear that using machine categorization can be a reasonable substitute for human review. © 2010 Wiley Periodicals, Inc.
John H Beisner - One of the best experts on this subject based on the ideXlab platform.
-
discovering a better way the need for effective civil litigation reform
Duke Law Journal, 2010Co-Authors: John H BeisnerAbstract:This Article addresses the myriad problems posed by unfettered Discovery in the United States. Rather than promoting fairness and efficiency in the American legal system, plaintiffs today often use Discovery in an abusive and vexatious manner to coerce defendants into accepting quick settlements. Over the past several decades, Discovery has expanded in both scope and magnitude such that Discovery costs now account for at least half of the total litigation costs in any given case. The advent of Electronic Discovery has only exacerbated the problem, given the sheer number of Electronic documents generated in the course of business and the corresponding time, effort, and cost associated with Electronic Discovery. Although recent efforts to amend the Federal Rules of Civil Procedure have failed to combat the abuses of civil Discovery, meaningful and effective reform of the current system is possible. This Article Copyright © 2010 by John H. Beisner. † Co-head, Class Actions and Mass Torts Practice Group, Skadden, Arps, Slate, Meagher & Flom LLP. This Article was drafted on behalf of the U.S. Chamber Institute for Legal Reform. The author wishes to express appreciation for the considerable editorial and research assistance provided by Jessica Davidson Miller, Neil Lombardo, and Jordan Schwartz. 1. Judge Paul V. Niemeyer, Chair, Advisory Comm. on the Fed. Rules of Civil Procedure, Comments at the Roscoe Pound Institute 1999 Forum for State Court Judges (July 14, 1999), in ROSCOE POUND INST., CONTROVERSIES SURROUNDING Discovery AND ITS EFFECT ON THE COURTS: REPORT OF THE 1999 FORUM FOR STATE COURT JUDGES 33, 33 (1999), available at http://www.poundinstitute.org/images/1999ForumReport.pdf. BEISNER IN FINAL.DOC 11/29/2010 6:52:13 PM 548 DUKE LAW JOURNAL [Vol. 60:547 proposes a number of pragmatic reforms—including adopting the English rule for Discovery disputes and suspending Discovery during the pendency of a motion to dismiss—to mitigate the abusive and costly nature of Discovery in the United States.