The Experts below are selected from a list of 318 Experts worldwide ranked by ideXlab platform
James E. Pustejovsky - One of the best experts on this subject based on the ideXlab platform.
-
procedural sensitivities of effect sizes for single case Designs with directly observed behavioral outcome measures
Psychological Methods, 2019Co-Authors: James E. PustejovskyAbstract:A wide variety of effect size indices have been proposed for quantifying the magnitude of treatment effects in Single-Case Designs. Commonly used measures include parametric indices such as the standardized mean difference as well as nonoverlap measures such as the percentage of nonoverlapping data, improvement rate difference, and nonoverlap of all pairs. Currently, little is known about the properties of these indices when applied to behavioral data collected by systematic direct observation, even though systematic direct observation is the most common method for outcome measurement in Single-Case research. This study uses Monte Carlo simulation to investigate the properties of several widely used Single-Case effect size measures when applied to systematic direct observation data. Results indicate that the magnitude of the nonoverlap measures and of the standardized mean difference can be strongly influenced by procedural details of the study's design, which is a significant limitation to using these indices as effect sizes for meta-analysis of Single-Case Designs. A less widely used parametric index, the log response ratio, has the advantage of being insensitive to sample size and observation session length, although its magnitude is influenced by the use of partial interval recording. (PsycINFO Database Record (c) 2019 APA, all rights reserved).
-
Between-case standardized effect size analysis of single case Designs: Examination of the two methods.
Research in developmental disabilities, 2018Co-Authors: Samuel L. Odom, Erin E. Barton, Hariharan Swaminathan, Brian Reichow, James E. PustejovskyAbstract:Abstract An increasing movement in single case research is to employ statistical analyses as one form of data analysis. Researchers have proposed different statistical approaches. The purpose of this paper is to examine the utility and discriminant validity of two novel types of between-case standardized effect size analyses with two existing systematic reviews. The between-case analyses found greater effect sizes for the studies in the object play review and smaller effect sizes for studies of sensory intervention, which were consistent with the overall conclusions reached in the original systematic reviews. These findings provide evidence of discriminant validity, although concerns remain around the methods’ utility across different single case research Designs. Future directions for research and development also are provided.
-
A Gradual Effects Model for Single-Case Designs
Multivariate behavioral research, 2018Co-Authors: Daniel M. Swan, James E. PustejovskyAbstract:Single-Case Designs are a class of repeated measures experiments used to evaluate the effects of interventions for small or specialized populations, such as individuals with low-incidence disabilities. There has been growing interest in systematic reviews and syntheses of evidence from Single-Case Designs, but there remains a need to further develop appropriate statistical models and effect sizes for data from the Designs. We propose a novel model for Single-Case data that exhibit nonlinear time trends created by an intervention that produces gradual effects, which build up and dissipate over time. The model expresses a structural relationship between a pattern of treatment assignment and an outcome variable, making it appropriate for both treatment reversal and multiple baseline Designs. It is formulated as a generalized linear model so that it can be applied to outcomes measured as frequency counts or proportions, both of which are commonly used in Single-Case research, while providing readily interpretable effect size estimates such as log response ratios or log odds ratios. We demonstrate the gradual effects model by applying it to data from a Single-Case study and examine the performance of proposed estimation methods in a Monte Carlo simulation of frequency count data.
-
Using response ratios for meta-analyzing Single-Case Designs with behavioral outcomes.
Journal of school psychology, 2018Co-Authors: James E. PustejovskyAbstract:Abstract Methods for meta-analyzing Single-Case Designs (SCDs) are needed to inform evidence-based practice in clinical and school settings and to draw broader and more defensible generalizations in areas where SCDs comprise a large part of the research base. The most widely used outcomes in Single-Case research are measures of behavior collected using systematic direct observation, which typically take the form of rates or proportions. For studies that use such measures, one simple and intuitive way to quantify effect sizes is in terms of proportionate change from baseline, using an effect size known as the log response ratio. This paper describes methods for estimating log response ratios and combining the estimates using meta-analysis. The methods are based on a simple model for comparing two phases, where the level of the outcome is stable within each phase and the repeated outcome measurements are independent. Although auto-correlation will lead to biased estimates of the sampling variance of the effect size, meta-analysis of response ratios can be conducted with robust variance estimation procedures that remain valid even when sampling variance estimates are biased. The methods are demonstrated using data from a recent meta-analysis on group contingency interventions for student problem behavior.
-
Procedural sensitivities of effect sizes for Single-Case Designs with directly observed behavioral outcome measures
2018Co-Authors: James E. PustejovskyAbstract:A wide variety of effect size indices have been proposed for quantifying the magnitude of treatment effects in Single-Case Designs. Commonly used measures include parametric indices such as the standardized mean difference, as well as non-overlap measures such as the percentage of non-overlapping data, improvement rate difference, and non-overlap of all pairs. Currently, little is known about the properties of these indices when applied to behavioral data collected by systematic direct observation, even though systematic direct observation is the most common method for outcome measurement in Single-Case research. This study uses Monte Carlo simulation to investigate the properties of several widely used Single-Case effect size measures when applied to systematic direct observation data. Results indicate that the magnitude of the non-overlap measures and of the standardized mean difference can be strongly influenced by procedural details of the study's design, which is a significant limitation to using these indices as effect sizes for meta-analysis of Single-Case Designs. A less widely used parametric index, the log-response ratio, has the advantage of being insensitive to sample size and observation session length, although its magnitude is influenced by the use of partial interval recording.
William R. Shadish - One of the best experts on this subject based on the ideXlab platform.
-
An introduction to modeling longitudinal data with generalized additive models: applications to Single-Case Designs.
Psychological methods, 2014Co-Authors: Kristynn J. Sullivan, William R. Shadish, Peter M. SteinerAbstract:Single-Case Designs (SCDs) are short time series that assess intervention effects by measuring units repeatedly over time in both the presence and absence of treatment. This article introduces a statistical technique for analyzing SCD data that has not been much used in psychological and educational research: generalized additive models (GAMs). In parametric regression, the researcher must choose a functional form to impose on the data, for example, that trend over time is linear. GAMs reverse this process by letting the data inform the choice of functional form. In this article we review the problem that trend poses in SCDs, discuss how current SCD analytic methods approach trend, describe GAMs as a possible solution, suggest a GAM model testing procedure for examining the presence of trend in SCDs, present a small simulation to show the statistical properties of GAMs, and illustrate the procedure on 3 examples of different lengths. Results suggest that GAMs may be very useful both as a form of sensitivity analysis for checking the plausibility of assumptions about trend and as a primary data analysis strategy for testing treatment effects. We conclude with a discussion of some problems with GAMs and some future directions for research on the application of GAMs to SCDs.
-
Analysis and meta-analysis of Single-Case Designs: an introduction.
Journal of school psychology, 2014Co-Authors: William R. ShadishAbstract:The last 10 years have seen great progress in the analysis and meta-analysis of Single-Case Designs (SCDs). This special issue includes five articles that provide an overview of current work on that topic, including standardized mean difference statistics, multilevel models, Bayesian statistics, and generalized additive models. Each article analyzes a common example across articles and presents syntax or macros for how to do them. These articles are followed by commentaries from Single-Case design researchers and journal editors. This introduction briefly describes each article and then discusses several issues that must be addressed before we can know what analyses will eventually be best to use in SCD research. These issues include modeling trend, modeling error covariances, computing standardized effect size estimates, assessing statistical power, incorporating more accurate models of outcome distributions, exploring whether Bayesian statistics can improve estimation given the small samples common in SCDs, and the need for annotated syntax and graphical user interfaces that make complex statistics accessible to SCD researchers. The article then discusses reasons why SCD researchers are likely to incorporate statistical analyses into their research more often in the future, including changing expectations and contingencies regarding SCD research from outside SCD communities, changes and diversity within SCD communities, corrections of erroneous beliefs about the relationship between SCD research and statistics, and demonstrations of how statistics can help SCD researchers better meet their goals.
-
Statistical Analyses of Single-Case Designs The Shape of Things to Come
Current Directions in Psychological Science, 2014Co-Authors: William R. ShadishAbstract:Single-Case-design researchers rarely used statistics in the past, but that is changing. In this article, I review the rapidly developing state of statistical analyses for Single-Case Designs, including effect sizes, multilevel models, and Bayesian analyses. No analysis meets all the desiderata for an optimal Single-Case-design analysis, but this may be remedied in the near future. Single-Case-design researchers will have incentives to use these analyses as they become more user-friendly and beneficial.
-
Analysis and meta-analysis of Single-Case Designs with a standardized mean difference statistic: A primer and applications
Journal of school psychology, 2013Co-Authors: William R. Shadish, Larry V. Hedges, James E. PustejovskyAbstract:This article presents a d-statistic for Single-Case Designs that is in the same metric as the d-statistic used in between-subjects Designs such as randomized experiments and offers some reasons why such a statistic would be useful in SCD research. The d has a formal statistical development, is accompanied by appropriate power analyses, and can be estimated using user-friendly SPSS macros. We discuss both advantages and disadvantages of d compared to other approaches such as previous d-statistics, overlap statistics, and multilevel modeling. It requires at least three cases for computation and assumes normally distributed outcomes and stationarity, assumptions that are discussed in some detail. We also show how to test these assumptions. The core of the article then demonstrates in depth how to compute d for one study, including estimation of the autocorrelation and the ratio of between case variance to total variance (between case plus within case variance), how to compute power using a macro, and how to use the d to conduct a meta-analysis of studies using Single-Case Designs in the free program R, including syntax in an appendix. This syntax includes how to read data, compute fixed and random effect average effect sizes, prepare a forest plot and a cumulative meta-analysis, estimate various influence statistics to identify studies contributing to heterogeneity and effect size, and do various kinds of publication bias analyses. This d may prove useful for both the analysis and meta-analysis of data from SCDs.
-
Using generalized additive (mixed) models to analyze single case Designs.
Journal of school psychology, 2013Co-Authors: William R. Shadish, Alain F. Zuur, Kristynn J. SullivanAbstract:This article shows how to apply generalized additive models and generalized additive mixed models to Single-Case design data. These models excel at detecting the functional form between two variables (often called trend), that is, whether trend exists, and if it does, what its shape is (e.g., linear and nonlinear). In many respects, however, these models are also an ideal vehicle for analyzing Single-Case Designs because they can consider level, trend, variability, overlap, immediacy of effect, and phase consistency that Single-Case design researchers examine when interpreting a functional relation. We show how these models can be implemented in a wide variety of ways to test whether treatment is effective, whether cases differ from each other, whether treatment effects vary over cases, and whether trend varies over cases. We illustrate diagnostic statistics and graphs, and we discuss overdispersion of data in detail, with examples of quasibinomial models for overdispersed data, including how to compute dispersion and quasi-AIC fit indices in generalized additive models. We show how generalized additive mixed models can be used to estimate autoregressive models and random effects and discuss the limitations of the mixed models compared to generalized additive models. We provide extensive annotated syntax for doing all these analyses in the free computer program R.
Rumen Manolov - One of the best experts on this subject based on the ideXlab platform.
-
Quantifying differences between conditions in Single-Case Designs: Possible analysis and meta-analysis
Developmental neurorehabilitation, 2016Co-Authors: Rumen Manolov, Antonio SolanasAbstract:The current paper is a call for and illustration of a way of closing the gap between basic research and professional practice in the field of neurorehabilitation. Methodologically, Single-Case experimental Designs and the guidelines created regarding their conduct are highlighted. Statistically, we review two data analytical options, namely (a) indices quantifying the difference between pairs of conditions in the same metric as the target behavior and (b) a formal statistical procedure offering a standardized overall quantification. The paper provides guidance in the analysis and suggests free software in order to illustrate, in the context of data from behavioral interventions with children with developmental disorders, that informative analyses are feasible. We also show how the results of individual studies can be made eligible for meta-analyses, which are useful for establishing the evidence basis of interventions. Nevertheless, we also point at decisions that need to be made during the process of data analysis.
-
Probability and visual aids for assessing intervention effectiveness in Single-Case Designs: A field test
Behavior modification, 2015Co-Authors: Rumen Manolov, Matthew Jamieson, Jonathan Evans, Vicenta SierraAbstract:Single-Case data analysis still relies heavily on visual inspection, and, at the same time, it is not clear to what extent the results of different quantitative procedures converge in identifying an intervention effect and its magnitude when applied to the same data; this is the type of evidence provided here for two procedures. One of the procedures, included due to the importance of providing objective criteria to visual analysts, is a visual aid fitting and projecting split-middle trend while taking into account data variability. The other procedure converts several different metrics into probabilities making their results comparable. In the present study, we expore to what extend these two procedures coincide in the magnitude of intervention effect taking place in a set of studies stemming from a recent meta-analysis. The procedures concur to a greater extent with the values of the indices computed and with each other and, to a lesser extent, with our own visual analysis. For distinguishing smaller from larger effects, the probability-based approach seems somewhat better suited. Moreover, the results of the field test suggest that the latter is a reasonably good mechanism for translating different metrics into similar labels. User friendly R code is provided for promoting the use of the visual aid, together with a quantification based on nonoverlap and the label provided by the probability approach.
-
Assessing Functional Relations in Single-Case Designs Quantitative Proposals in the Context of the Evidence-Based Movement
Behavior modification, 2014Co-Authors: Rumen Manolov, Antonio Solanas, Vicenta Sierra, Juan BotellaAbstract:In the context of the evidence-based practices movement, the emphasis on computing effect sizes and combining them via meta-analysis does not preclude the demonstration of functional relations. For the latter aim, we propose to augment the visual analysis to add consistency to the decisions made on the existence of a functional relation without losing sight of the need for a methodological evaluation of what stimuli and reinforcement or punishment are used to control the behavior. Four options for quantification are reviewed, illustrated, and tested with simulated data. These quantifications include comparing the projected baseline with the actual treatment measurements, on the basis of either parametric or nonparametric statistics. The simulated data used to test the quantifications include nine data patterns in terms of the presence and type of effect and comprise ABAB and multiple-baseline Designs. Although none of the techniques is completely flawless in terms of detecting a functional relation only when it is present but not when it is absent, an option based on projecting split-middle trend and considering data variability as in exploratory data analysis proves to be the best performer for most data patterns. We suggest that the information on whether a functional relation has been demonstrated should be included in meta-analyses. It is also possible to use as a weight the inverse of the data variability measure used in the quantification for assessing the functional relation. We offer an easy to use code for open-source software for implementing some of the quantifications.
-
Weighting strategies in the meta-analysis of Single-Case studies
Behavior research methods, 2014Co-Authors: Rumen Manolov, Georgina Guilera, Vicenta SierraAbstract:Establishing the evidence base of interventions taking place in areas such as psychology and special education is one of the research aims of Single-Case Designs, in conjunction with the aim of improving the well-being of participants in the studies. The scientific criteria for solid evidence focus on the internal and external validity of the studies, and for both types of validity, replicating studies and integrating the results of these replications (i.e., meta-analyzing) is crucial. In the present study, we deal with one of the aspects of meta-analysis—namely, the weighting strategy used when computing an average effect size across studies. Several weighting strategies suggested for Single-Case Designs are discussed and compared in the context of both simulated and real-life data. The results indicated that there are no major differences between the strategies, and thus, we consider that it is important to choose weights with a sound statistical and methodological basis, while scientific parsimony is another relevant criterion. More empirical research and conceptual discussion are warranted regarding the optimal weighting strategy in Single-Case Designs, alongside investigation of the optimal effect size measure in these types of Designs.
-
Regression-based techniques for statistical decision making in Single-Case Designs.
Psicothema, 2010Co-Authors: Rumen Manolov, Antonio Solanas, Jaume Arnau, Roser BonoAbstract:The present study evaluates the performance of four methods for estimating regression coefficients used to make statistical decisions about intervention effectiveness in Single-Case Designs. Ordinary least square estimation is compared to two correction techniques dealing with general trend and a procedure that eliminates autocorrelation whenever it is present. Type I error rates and statistical power are studied for experimental conditions defined by the presence or absence of treatment effect (change in level or in slope), general trend, and serial dependence. The results show that empirical Type I error rates do not approach the nominal ones in the presence of autocorrelation or general trend when ordinary and generalized least squares are applied. The techniques controlling trend show lower false alarm rates, but prove to be insufficiently sensitive to existing treatment effects. Consequently, the use of the statistical significance of the regression coefficients for detecting treatment effects is not recommended for short data series.
Patrick Onghena - One of the best experts on this subject based on the ideXlab platform.
-
the conditional power of randomization tests for single case effect sizes in Designs with randomized treatment order a monte carlo simulation study
Behavior Research Methods, 2018Co-Authors: Bart Michiels, Mieke Heyvaert, Patrick OnghenaAbstract:The conditional power (CP) of the randomization test (RT) was investigated in a simulation study in which three different Single-Case effect size (ES) measures were used as the test statistics: the mean difference (MD), the percentage of nonoverlapping data (PND), and the nonoverlap of all pairs (NAP). Furthermore, we studied the effect of the experimental design on the RT’s CP for three different Single-Case Designs with rapid treatment alternation: the completely randomized design (CRD), the randomized block design (RBD), and the restricted randomized alternation design (RRAD). As a third goal, we evaluated the CP of the RT for three types of simulated data: data generated from a standard normal distribution, data generated from a uniform distribution, and data generated from a first-order autoregressive Gaussian process. The results showed that the MD and NAP perform very similarly in terms of CP, whereas the PND performs substantially worse. Furthermore, the RRAD yielded marginally higher power in the RT, followed by the CRD and then the RBD. Finally, the power of the RT was almost unaffected by the type of the simulated data. On the basis of the results of the simulation study, we recommend at least 20 measurement occasions for Single-Case Designs with a randomized treatment order that are to be evaluated with an RT using a 5% significance level. Furthermore, we do not recommend use of the PND, because of its low power in the RT.
-
confidence intervals for single case effect size measures based on randomization test inversion
Behavior Research Methods, 2017Co-Authors: Bart Michiels, Mieke Heyvaert, Ann Meulders, Patrick OnghenaAbstract:In the current paper, we present a method to construct nonparametric confidence intervals (CIs) for Single-Case effect size measures in the context of various Single-Case Designs. We use the relationship between a two-sided statistical hypothesis test at significance level α and a 100 (1 – α) % two-sided CI to construct CIs for any effect size measure θ that contain all point null hypothesis θ values that cannot be rejected by the hypothesis test at significance level α. This method of hypothesis test inversion (HTI) can be employed using a randomization test as the statistical hypothesis test in order to construct a nonparametric CI for θ. We will refer to this procedure as randomization test inversion (RTI). We illustrate RTI in a situation in which θ is the unstandardized and the standardized difference in means between two treatments in a completely randomized Single-Case design. Additionally, we demonstrate how RTI can be extended to other types of Single-Case Designs. Finally, we discuss a few challenges for RTI as well as possibilities when using the method with other effect size measures, such as rank-based nonoverlap indices. Supplementary to this paper, we provide easy-to-use R code, which allows the user to construct nonparametric CIs according to the proposed method.
-
An R package for Single-Case randomization tests
Behavior research methods, 2008Co-Authors: Isis Bulté, Patrick OnghenaAbstract:Randomization tests are nonparametric statistical tests that obtain their validity by computationally mimicking the random assignment procedure that was used in the design phase of a study. Because randomization tests do not rely on a random sampling assumption, they can provide a better alternative than parametric statistical tests for analyzing data from Single-Case Designs. In this article, an R package is described for use in designing Single-Case phase (AB, ABA, and ABAB) and alternation (completely randomized, alternating treatments, and randomized block) experiments, as well as for conducting statistical analyses on data gathered by means of such Designs. The R code is presented in a step-by-step way, which at the same time clarifies the rationale behind Single-Case randomization tests.
-
The Aggregation of Single-Case Results Using Hierarchical Linear Models.
The Behavior Analyst Today, 2007Co-Authors: Wim Van Den Noortgate, Patrick OnghenaAbstract:To investigate the generalizability of the results of Single-Case experimental studies, evaluating the effect of one or more treatments, in applied research various simultaneous and sequential replication strategies are used. We discuss one approach for aggregating the results for Single-Cases: the use of hierarchical linear models. This approach has the potential to allow making improved inferences about the effects for the individual cases, but also to estimate and test the overall effect, and explore the generality of this effect across cases and under different conditions. Keywords: Single-Case; hierarchical linear model; replication; aggregation ********** Single-Case experimental Designs are used to evaluate the effect of one or more treatments on a single case. The case may be a subject or another single entity that forms the research unit, such as a school or a family. This entity is repeatedly observed, over the levels of one or several manipulated independent variables (Onghena, 2005). In the most basic design, the AB-phase design or interrupted time series design, the case is observed repeatedly during a first phase (A), typically a baseline phase before an intervention takes place, and in a second phase (B) after or during an intervention. To evaluate the effect of the intervention, scores in both phases are compared. Single-Case Designs have a long history in behavioral science (Ittenbach & Lawhead, 1997), but the last decades, Single-Case methodology has further been elaborated, aiming at improving the internal validity of the conclusions. For instance, reversal phase Designs (e.g., an ABAB-design) or alternation Designs with rapidly alternating conditions (e.g., an AABBBABAABB-design) rather than a simple AB-phase design may be used in order to assess or control statistically for the effect of history, maturation or other time-related confounding variables. The effect of such confounding variables may further be controlled by means of randomization while setting up the study, for instance by randomly assigning measurement occasions over treatments or randomizing the time of intervention (Edgington, 1996). Although group Designs receive much more attention in methodological courses and handbooks, in the last decades there has been renewed interest in Single-Case Designs, especially in behavior modification and clinical psychology (Barlow & Hersen, 1984. Kazdin, 1982), neuropsychology (Caramazza, 1990), psychopharmacology (Cook, 1996), and educational research (Kratochwill & Levin, 1992). The popularity of the Designs is also reflected in the relatively large number of articles published in the Behavior Analyst Today that discuss or apply a variety of Single-Case Designs (about twenty between 2001 and 2006). Single-Case Designs indeed are very attractive in several situations (Franklin, Allison, Gorman, 1997; Onghena, 2005). Single-Case studies may be relatively easy to set up and are much less expensive than large-scale group-comparison studies. This makes the Designs also attractive for practitioners, who want to get a first insight into the effect of a treatment. An additional strength of Single-Case Designs is that, in contrast to group Designs that give insight into the average effect of a treatment, they give an in-depth insight into the behavior of one single case. Especially in clinical settings, the research indeed often focuses on the effect of a treatment for a specific case. Finally, since only a single case is investigated, the design often allows making a large number of repeated observations, enabling a detailed study of the evolution of the behavior. Single-Case Designs thus are (initially) aimed at drawing valid conclusions regarding one entity. Sometimes, for instance in applied clinical settings, the primary interest may indeed be in this single entity, since it concerns a case that presented itself with a problem to solve. …
Antonio Solanas - One of the best experts on this subject based on the ideXlab platform.
-
Quantifying differences between conditions in Single-Case Designs: Possible analysis and meta-analysis
Developmental neurorehabilitation, 2016Co-Authors: Rumen Manolov, Antonio SolanasAbstract:The current paper is a call for and illustration of a way of closing the gap between basic research and professional practice in the field of neurorehabilitation. Methodologically, Single-Case experimental Designs and the guidelines created regarding their conduct are highlighted. Statistically, we review two data analytical options, namely (a) indices quantifying the difference between pairs of conditions in the same metric as the target behavior and (b) a formal statistical procedure offering a standardized overall quantification. The paper provides guidance in the analysis and suggests free software in order to illustrate, in the context of data from behavioral interventions with children with developmental disorders, that informative analyses are feasible. We also show how the results of individual studies can be made eligible for meta-analyses, which are useful for establishing the evidence basis of interventions. Nevertheless, we also point at decisions that need to be made during the process of data analysis.
-
Assessing Functional Relations in Single-Case Designs Quantitative Proposals in the Context of the Evidence-Based Movement
Behavior modification, 2014Co-Authors: Rumen Manolov, Antonio Solanas, Vicenta Sierra, Juan BotellaAbstract:In the context of the evidence-based practices movement, the emphasis on computing effect sizes and combining them via meta-analysis does not preclude the demonstration of functional relations. For the latter aim, we propose to augment the visual analysis to add consistency to the decisions made on the existence of a functional relation without losing sight of the need for a methodological evaluation of what stimuli and reinforcement or punishment are used to control the behavior. Four options for quantification are reviewed, illustrated, and tested with simulated data. These quantifications include comparing the projected baseline with the actual treatment measurements, on the basis of either parametric or nonparametric statistics. The simulated data used to test the quantifications include nine data patterns in terms of the presence and type of effect and comprise ABAB and multiple-baseline Designs. Although none of the techniques is completely flawless in terms of detecting a functional relation only when it is present but not when it is absent, an option based on projecting split-middle trend and considering data variability as in exploratory data analysis proves to be the best performer for most data patterns. We suggest that the information on whether a functional relation has been demonstrated should be included in meta-analyses. It is also possible to use as a weight the inverse of the data variability measure used in the quantification for assessing the functional relation. We offer an easy to use code for open-source software for implementing some of the quantifications.
-
Regression-based techniques for statistical decision making in Single-Case Designs.
Psicothema, 2010Co-Authors: Rumen Manolov, Antonio Solanas, Jaume Arnau, Roser BonoAbstract:The present study evaluates the performance of four methods for estimating regression coefficients used to make statistical decisions about intervention effectiveness in Single-Case Designs. Ordinary least square estimation is compared to two correction techniques dealing with general trend and a procedure that eliminates autocorrelation whenever it is present. Type I error rates and statistical power are studied for experimental conditions defined by the presence or absence of treatment effect (change in level or in slope), general trend, and serial dependence. The results show that empirical Type I error rates do not approach the nominal ones in the presence of autocorrelation or general trend when ordinary and generalized least squares are applied. The techniques controlling trend show lower false alarm rates, but prove to be insufficiently sensitive to existing treatment effects. Consequently, the use of the statistical significance of the regression coefficients for detecting treatment effects is not recommended for short data series.
-
Comparing “Visual” Effect Size Indices for Single-Case Designs
Methodology, 2010Co-Authors: Rumen Manolov, Antonio Solanas, David LeivaAbstract:Effect size indices are indispensable for carrying out meta-analyses and can also be seen as an alternative for making decisions about the effectiveness of a treatment in an individual applied study. The desirable features of the procedures for quantifying the magnitude of intervention effect include educational/clinical meaningfulness, calculus easiness, insensitivity to autocorrelation, low false alarm, and low miss rates. Three effect size indices related to visual analysis are compared according to the aforementioned criteria. The comparison is made by means of data sets with known parameters: degree of serial dependence, presence or absence of general trend, and changes in level and/or in slope. The percent of nonoverlapping data showed the highest discrimination between data sets with and without intervention effect. In cases when autocorrelation or trend is present, the percentage of data points exceeding the median may be a better option to quantify the effectiveness of a psychological treatment.
-
Factors Affecting Visual Inference in Single-Case Designs
The Spanish journal of psychology, 2009Co-Authors: Veronica M. Ximenes, Rumen Manolov, Antonio Solanas, Vicenç QueraAbstract:Visual inspection remains the most frequently applied method for detecting treatment effects in Single-Case Designs. The advantages and limitations of visual inference are here discussed in relation to other procedures for assessing intervention effectiveness. The first part of the paper reviews previous research on visual analysis, paying special attention to the validation of visual analysts' decisions, inter-judge agreement, and false alarm and omission rates. The most relevant factors affecting visual inspection (i.e., effect size, autocorrelation, data variability, and analysts' expertise) are highlighted and incorporated into an empirical simulation study with the aim of providing further evidence about the reliability of visual analysis. Our results concur with previous studies that have reported the relationship between serial dependence and increased Type I rates. Participants with greater experience appeared to be more conservative and used more consistent criteria when assessing graphed data. Nonetheless, the decisions made by both professionals and students did not match sufficiently the simulated data features, and we also found low intra-judge agreement, thus suggesting that visual inspection should be complemented by other methods when assessing treatment effectiveness.