The Experts below are selected from a list of 2247 Experts worldwide ranked by ideXlab platform
Joseph P. Romano - One of the best experts on this subject based on the ideXlab platform.
-
Control of Directional Errors in Fixed Sequence Multiple Testing
Statistica Sinica, 2019Co-Authors: Anjana Grandhi, Wenge Guo, Joseph P. RomanoAbstract:In this paper, we consider the problem of simultaneously testing many two-sided hypotheses when rejections of null hypotheses are accompanied by claims of the direction of the alternative. The fundamental goal is to construct methods that control the mixed directional Familywise Error Rate, which is the probability of making any type 1 or type 3 (directional) 1 Error. In particular, attention is focused on cases where the hypotheses are ordered as H1, . . . , Hn, so that Hi+1 is tested only if H1, . . . , Hi have all been previously rejected. In this situation, one can control the usual Familywise Error Rate under arbitrary dependence by the basic procedure which tests each hypothesis at level α, and no other multiplicity adjustment is needed. However, we show that this is far too liberal if one also accounts for directional Errors. But, by imposing certain dependence assumptions on the test statistics, one can retain the basic procedure.
-
Control of Directional Errors in Fixed Sequence Multiple Testing
arXiv: Statistics Theory, 2016Co-Authors: Anjana Grandhi, Wenge Guo, Joseph P. RomanoAbstract:In this paper, we consider the problem of simultaneously testing many two-sided hypotheses when rejections of null hypotheses are accompanied by claims of the direction of the alternative. The fundamental goal is to construct methods that control the mixed directional Familywise Error Rate, which is the probability of making any type 1 or type 3 (directional) Error. In particular, attention is focused on cases where the hypotheses are ordered as $H_1 , \ldots, H_n$, so that $H_{i+1}$ is tested only if $H_1 , \ldots, H_i$ have all been previously rejected. In this situation, one can control the usual Familywise Error Rate under arbitrary dependence by the basic procedure which tests each hypothesis at level $\alpha$, and no other multiplicity adjustment is needed. However, we show that this is far too liberal if one also accounts for directional Errors. But, by imposing certain dependence assumptions on the test statistics, one can retain the basic procedure. Through a simulation study and a clinical trial example, we numerically illustRate good performance of the proposed procedures compared to the existing mdFWER controlling procedures. The proposed procedures are also implemented in the R-package FixSeqMTP.
-
On stepwise control of directional Errors under independence and some dependence
Journal of Statistical Planning and Inference, 2015Co-Authors: Wenge Guo, Joseph P. RomanoAbstract:Abstract In this paper, the problem of Error control of stepwise multiple testing procedures is considered. For two-sided hypotheses, control of both type 1 and type 3 (or directional) Errors is required, and thus mixed directional Familywise Error Rate control and mixed directional false discovery Rate control are each considered by incorporating both types of Errors in the Error Rate. Mixed directional Familywise Error Rate control of stepwise methods in multiple testing has proven to be a challenging problem, as demonstRated in Shaffer (1980) . By an appropriate formulation of the problem, some new stepwise procedures are developed that control type 1 and directional Errors under independence and various dependencies.
-
Stepup procedures for control of generalizations of the Familywise Error Rate
The Annals of Statistics, 2006Co-Authors: Joseph P. Romano, Azeem M. ShaikhAbstract:Consider the multiple testing problem of testing null hypotheses $H_1,...,H_s$. A classical approach to dealing with the multiplicity problem is to restrict attention to procedures that control the Familywise Error Rate ($\mathit{FWER}$), the probability of even one false rejection. But if $s$ is large, control of the $\mathit{FWER}$ is so stringent that the ability of a procedure that controls the $\mathit{FWER}$ to detect false null hypotheses is limited. It is therefore desirable to consider other measures of Error control. This article considers two generalizations of the $\mathit{FWER}$. The first is the $k-\mathit{FWER}$, in which one is willing to toleRate $k$ or more false rejections for some fixed $k\geq 1$. The second is based on the false discovery proportion ($\mathit{FDP}$), defined to be the number of false rejections divided by the total number of rejections (and defined to be 0 if there are no rejections). Benjamini and Hochberg [J. Roy. Statist. Soc. Ser. B 57 (1995) 289--300] proposed control of the false discovery Rate ($\mathit{FDR}$), by which they meant that, for fixed $\alpha$, $E(\mathit{FDP})\leq\alpha$. Here, we consider control of the $\mathit{FDP}$ in the sense that, for fixed $\gamma$ and $\alpha$, $P\{\mathit{FDP}>\gamma\}\leq \alpha$. Beginning with any nondecreasing sequence of constants and $p$-values for the individual tests, we derive stepup procedures that control each of these two measures of Error control without imposing any assumptions on the dependence structure of the $p$-values. We use our results to point out a few interesting connections with some closely related stepdown procedures. We then compare and contrast two $\mathit{FDP}$-controlling procedures obtained using our results with the stepup procedure for control of the $\mathit{FDR}$ of Benjamini and Yekutieli [Ann. Statist. 29 (2001) 1165--1188].
-
Generalizations of the Familywise Error Rate
The Annals of Statistics, 2005Co-Authors: Erich L. Lehmann, Joseph P. RomanoAbstract:Consider the problem of simultaneously testing null hypotheses H_1,...,H_s. The usual approach to dealing with the multiplicity problem is to restrict attention to procedures that control the Familywise Error Rate (FWER), the probability of even one false rejection. In many applications, particularly if s is large, one might be willing to toleRate more than one false rejection provided the number of such cases is controlled, thereby increasing the ability of the procedure to detect false null hypotheses. This suggests replacing control of the FWER by controlling the probability of k or more false rejections, which we call the k-FWER. We derive both single-step and stepdown procedures that control the k-FWER, without making any assumptions concerning the dependence structure of the p-values of the individual tests. In particular, we derive a stepdown procedure that is quite simple to apply, and prove that it cannot be improved without violation of control of the k-FWER. We also consider the false discovery proportion (FDP) defined by the number of false rejections divided by the total number of rejections (defined to be 0 if there are no rejections). The false discovery Rate proposed by Benjamini and Hochberg [J. Roy. Statist. Soc. Ser. B 57 (1995) 289-300] controls E(FDP). Here, we construct methods such that, for any \gamma and \alpha, P{FDP>\gamma}\le\alpha. Two stepdown methods are proposed. The first holds under mild conditions on the dependence structure of p-values, while the second is more conservative but holds without any dependence assumptions.
Veronika Gontscharuk - One of the best experts on this subject based on the ideXlab platform.
-
controlling the Familywise Error Rate with plug in estimator for the proportion of true null hypotheses
Journal of The Royal Statistical Society Series B-statistical Methodology, 2009Co-Authors: Helmut Finner, Veronika GontscharukAbstract:Summary. Estimation of the number or proportion of true null hypotheses in multiple-testing problems has become an interesting area of research. The first important work in this field was performed by Schweder and Spjotvoll. Among others, they proposed to use plug-in estimates for the proportion of true null hypotheses in multiple-test procedures to improve the power. We investigate the problem of controlling the Familywise Error Rate FWER when such estimators are used as plug-in estimators in single-step or step-down multiple-test procedures. First we investigate the case of independent p-values under the null hypotheses and show that a suitable choice of plug-in estimates leads to control of FWER in single-step procedures. We also investigate the power and study the asymptotic behaviour of the number of false rejections. Although step-down procedures are more difficult to handle we briefly consider a possible solution to this problem. Anyhow, plug-in step-down procedures are not recommended here. For dependent p-values we derive a condition for asymptotic control of FWER and provide some simulations with respect to FWER and power for various models and hypotheses.
-
Controlling the Familywise Error Rate with plug‐in estimator for the proportion of true null hypotheses
Journal of the Royal Statistical Society: Series B (Statistical Methodology), 2009Co-Authors: Helmut Finner, Veronika GontscharukAbstract:Summary. Estimation of the number or proportion of true null hypotheses in multiple-testing problems has become an interesting area of research. The first important work in this field was performed by Schweder and Spjotvoll. Among others, they proposed to use plug-in estimates for the proportion of true null hypotheses in multiple-test procedures to improve the power. We investigate the problem of controlling the Familywise Error Rate FWER when such estimators are used as plug-in estimators in single-step or step-down multiple-test procedures. First we investigate the case of independent p-values under the null hypotheses and show that a suitable choice of plug-in estimates leads to control of FWER in single-step procedures. We also investigate the power and study the asymptotic behaviour of the number of false rejections. Although step-down procedures are more difficult to handle we briefly consider a possible solution to this problem. Anyhow, plug-in step-down procedures are not recommended here. For dependent p-values we derive a condition for asymptotic control of FWER and provide some simulations with respect to FWER and power for various models and hypotheses.
Gary W. Oehlert - One of the best experts on this subject based on the ideXlab platform.
-
Student-Newman-Kuels controls the false discovery Rate
Statistics & Probability Letters, 2000Co-Authors: Gary W. OehlertAbstract:The Student-Newman-Kuels procedure is a well-known step-down multiple comparisons procedure with critical values based on the Studentized range distribution. The False Discovery Rate is a Type I Error Rate for multiple comparisons, intermediate in stringency to the weak Familywise Error Rate (experimentwise Error Rate) and the strong Familywise Error Rate. We show that SNK controls FDR.
-
Student–Newman–Kuels controls the false discovery Rate
Statistics & Probability Letters, 2000Co-Authors: Gary W. OehlertAbstract:Abstract The Student–Newman–Kuels procedure is a well-known step-down multiple comparisons procedure with critical values based on the Studentized range distribution. The False Discovery Rate is a Type I Error Rate for multiple comparisons, intermediate in stringency to the weak Familywise Error Rate (experimentwise Error Rate) and the strong Familywise Error Rate. We show that SNK controls FDR.
Helmut Finner - One of the best experts on this subject based on the ideXlab platform.
-
On adaptive procedures controlling the Familywise Error Rate
Journal of Statistical Planning and Inference, 2012Co-Authors: Sanat K. Sarkar, Wenge Guo, Helmut FinnerAbstract:Abstract The idea of modifying, and potentially improving, classical multiple testing methods controlling the Familywise Error Rate (FWER) via an estimate of the unknown number of true null hypotheses has been around for a long time without a formal answer to the question whether or not such adaptive methods ultimately maintain the strong control of FWER, until Finner and Gontscharuk (2009) and Guo (2009) have offered some answers. A class of adaptive Bonferroni and S˘idak methods larger than considered in those papers is introduced, with the FWER control now proved under a weaker distributional setup. Numerical results show that there are versions of adaptive Bonferroni and S˘idak methods that can perform better under certain positive dependence situations than those previously considered. A different adaptive Holm method and its stepup analog, referred to as an adaptive Hochberg method, are also introduced, and their FWER control is proved asymptotically, as in those papers. These adaptive Holm and Hochberg methods are numerically seen to often outperform the previously considered adaptive Holm method.
-
controlling the Familywise Error Rate with plug in estimator for the proportion of true null hypotheses
Journal of The Royal Statistical Society Series B-statistical Methodology, 2009Co-Authors: Helmut Finner, Veronika GontscharukAbstract:Summary. Estimation of the number or proportion of true null hypotheses in multiple-testing problems has become an interesting area of research. The first important work in this field was performed by Schweder and Spjotvoll. Among others, they proposed to use plug-in estimates for the proportion of true null hypotheses in multiple-test procedures to improve the power. We investigate the problem of controlling the Familywise Error Rate FWER when such estimators are used as plug-in estimators in single-step or step-down multiple-test procedures. First we investigate the case of independent p-values under the null hypotheses and show that a suitable choice of plug-in estimates leads to control of FWER in single-step procedures. We also investigate the power and study the asymptotic behaviour of the number of false rejections. Although step-down procedures are more difficult to handle we briefly consider a possible solution to this problem. Anyhow, plug-in step-down procedures are not recommended here. For dependent p-values we derive a condition for asymptotic control of FWER and provide some simulations with respect to FWER and power for various models and hypotheses.
-
Controlling the Familywise Error Rate with plug‐in estimator for the proportion of true null hypotheses
Journal of the Royal Statistical Society: Series B (Statistical Methodology), 2009Co-Authors: Helmut Finner, Veronika GontscharukAbstract:Summary. Estimation of the number or proportion of true null hypotheses in multiple-testing problems has become an interesting area of research. The first important work in this field was performed by Schweder and Spjotvoll. Among others, they proposed to use plug-in estimates for the proportion of true null hypotheses in multiple-test procedures to improve the power. We investigate the problem of controlling the Familywise Error Rate FWER when such estimators are used as plug-in estimators in single-step or step-down multiple-test procedures. First we investigate the case of independent p-values under the null hypotheses and show that a suitable choice of plug-in estimates leads to control of FWER in single-step procedures. We also investigate the power and study the asymptotic behaviour of the number of false rejections. Although step-down procedures are more difficult to handle we briefly consider a possible solution to this problem. Anyhow, plug-in step-down procedures are not recommended here. For dependent p-values we derive a condition for asymptotic control of FWER and provide some simulations with respect to FWER and power for various models and hypotheses.
Adrian Mander - One of the best experts on this subject based on the ideXlab platform.
-
Group sequential crossover trial designs with strong control of the Familywise Error Rate
Sequential analysis, 2018Co-Authors: Michael J. Grayling, James Wason, Adrian ManderAbstract:Crossover designs are an extremely useful tool to investigators, and group sequential methods have proven highly proficient at improving the efficiency of parallel group trials. Yet, group sequenti...
-
An optimised multi-arm multi-stage clinical trial design for unknown variance
Contemporary clinical trials, 2018Co-Authors: Michael J. Grayling, James Wason, Adrian ManderAbstract:Abstract Multi-arm multi-stage trial designs can bring notable gains in efficiency to the drug development process. However, for normally distributed endpoints, the determination of a design typically depends on the assumption that the patient variance in response is known. In practice, this will not usually be the case. To allow for unknown variance, previous research explored the performance of t -test statistics, coupled with a quantile substitution procedure for modifying the stopping boundaries, at controlling the Familywise Error-Rate to the nominal level. Here, we discuss an alternative method based on Monte Carlo simulation that allows the group size and stopping boundaries of a multi-arm multi-stage t -test to be optimised, according to some nominated optimality criteria. We consider several examples, provide R code for general implementation, and show that our designs confer a Familywise Error-Rate and power close to the desired level. Consequently, this methodology will provide utility in future multi-arm multi-stage trials.
-
Efficient determination of optimised multi-arm multi-stage experimental designs with control of generalised Error-Rates
arXiv: Methodology, 2017Co-Authors: Michael J. Grayling, James Wason, Adrian ManderAbstract:Primarily motivated by the drug development process, several publications have now presented methodology for the design of multi-arm multi-stage experiments with normally distributed outcome variables of known variance. Here, we extend these past considerations to allow the design of what we refer to as an abcd multi-arm multi-stage experiment. We provide a proof of how strong control of the a-generalised type-I Familywise Error-Rate can be ensured. We then describe how to attain the power to reject at least b out of c false hypotheses, which is related to controlling the b-generalised type-II Familywise Error-Rate. Following this, we detail how a design can be optimised for a scenario in which rejection of any d null hypotheses brings about termination of the experiment. We achieve this by proposing a highly computationally efficient approach for evaluating the performance of a candidate design. Finally, using a real clinical trial as a motivating example, we explore the effect of the design's control parameters on the statistical operating characteristics.
-
Group Sequential Crossover Trial Designs with Strong Control of the Familywise Error Rate
arXiv: Methodology, 2017Co-Authors: Michael J. Grayling, James Wason, Adrian ManderAbstract:Crossover designs are an extremely useful tool to investigators, whilst group sequential methods have proven highly proficient at improving the efficiency of parallel group trials. Yet, group sequential methods and crossover designs have rarely been paired together. One possible explanation for this could be the absence of a formal proof of how to strongly control the Familywise Error Rate in the case when multiple comparisons will be made. Here, we provide this proof, valid for any number of initial experimental treatments and any number of stages, when results are analysed using a linear mixed model. We then establish formulae for the expected sample size and expected number of observations of such a trial, given any choice of stopping boundaries. Finally, utilising the four-treatment, four-period TOMADO trial as an example, we demonstRate group sequential methods in this setting could have reduced the trials expected number of observations under the global null hypothesis by over 33%.
