Model Comparison

14,000,000 Leading Edge Experts on the ideXlab platform

Scan Science and Technology

Contact Leading Edge Experts & Companies

Scan Science and Technology

Contact Leading Edge Experts & Companies

The Experts below are selected from a list of 1062279 Experts worldwide ranked by ideXlab platform

Roberto Soler - One of the best experts on this subject based on the ideXlab platform.

  • Model Comparison for the density structure across solar coronal waveguides
    The Astrophysical Journal, 2015
    Co-Authors: Inigo Arregui, Roberto Soler, A. Asensio Ramos
    Abstract:

    The spatial variation of physical quantities, such as the mass density, across solar atmospheric waveguides governs the timescales and spatial scales for wave damping and energy dissipation. The direct measurement of the spatial distribution of density, however, is difficult and indirect seismology inversion methods have been suggested as an alternative. We applied Bayesian inference, Model Comparison, and Model-averaging techniques to the inference of the cross-field density structuring in solar magnetic waveguides using information on periods and damping times for resonantly damped magnetohydrodynamic (MHD) transverse kink oscillations. Three commonly employed alternative profiles were used to Model the variation of the mass density across the waveguide boundary. Parameter inference enabled us to obtain information on physical quantities such as the Alfv\'en travel time, the density contrast, and the transverse inhomogeneity length scale. The inference results from alternative density Models were compared and their differences quantified. Then, the relative plausibility of the considered Models was assessed by performing Model Comparison. Our results indicate that the evidence in favor of any of the three Models is minimal, unless the oscillations are strongly damped. In such a circumstance, the application of Model-averaging techniques enables the computation of an evidence-weighted inference that takes into account the plausibility of each Model in the calculation of a combined inversion for the unknown physical parameters.

  • Model Comparison for the density structure along solar prominence threads
    Astronomy & Astrophysics, 2015
    Co-Authors: Inigo Arregui, Roberto Soler
    Abstract:

    Context. Quiescent solar prominence fine structures are typically Modelled as density enhancements, called threads, which occupy a fraction of a longer magnetic flux tube. This is justified from the spatial distribution of the imaged plasma emission or absorption of prominences at small spatial scales. The profile of the mass density along the magnetic field is unknown, however, and several arbitrary alternatives are employed in prominence wave studies. The identification and measurement of period ratios from multiple harmonics in standing transverse thread oscillations offer a remote diagnostics method to probe the density variation of these structures. Aims. We present a Comparison of theoretical Models for the field-aligned density along prominence fine structures. They aim to imitate density distributions in which the plasma is more or less concentrated around the centre of the magnetic flux tube. We consider Lorentzian, Gaussian, and parabolic profiles. We compare theoretical predictions based on these profiles for the period ratio between the fundamental transverse kink mode and the first overtone to obtain estimates for the density ratios between the central part of the tube and its foot-points and to assess which one would better explain observed period ratio data. Methods. Bayesian parameter inference and Model Comparison techniques were developed and applied. To infer the parameters, we computed the posterior distribution for the density gradient parameter that depends on the observable period ratio. The Model Comparison involved computing the marginal likelihood as a function of the period ratio to obtain the plausibility of each density Model as a function of the observable. We also computed the Bayes factors to quantify the relative evidence for each Model, given a period ratio observation. Results. A Lorentzian density profile, with plasma density concentrated around the centre of the tube, seems to offer the most plausible inversion result. A Gaussian profile would require unrealistically high values of the density gradient parameter, and a parabolic density distribution does not enable us to obtain well-constrained posterior probability distributions. However, our Model Comparison results indicate that the evidence points to the Gaussian and parabolic profiles for period ratios in between 2 and 3, while the Lorentzian profile is preferred for higher period ratio values. The method we present can be used to obtain information on the plasma structure along threads, provided period ratio measurements become widely available.

  • Model Comparison for the density structure along solar prominence threads
    arXiv: Solar and Stellar Astrophysics, 2015
    Co-Authors: Inigo Arregui, Roberto Soler
    Abstract:

    Quiescent solar prominence fine structures are typically Modelled as density enhancements, called threads, which occupy a fraction of a longer magnetic flux tube. The profile of the mass density along the magnetic field is however unknown and several arbitrary alternatives are employed in prominence wave studies. We present a Comparison of theoretical Models for the field-aligned density along prominence fine structures. We consider Lorentzian, Gaussian, and parabolic profiles. We compare their theoretical predictions for the period ratio between the fundamental transverse kink mode and the first overtone to obtain estimates for the ratio of densities between the central part of the tube and its foot-points and to assess which one would better explain observed period ratio data. Bayesian parameter inference and Model Comparison techniques are developed and applied. Parameter inference requires the computation of the posterior distribution for the density gradient parameter conditional on the observable period ratio. Model Comparison involves the computation of the marginal likelihood as a function of the period ratio to obtain the plausibility of each density Model and the computation of Bayes Factors to quantify the relative evidence for each Model, given a period ratio observation. A Lorentzian density profile, with plasma density concentrated around the centre of the tube seems to offer the most plausible inversion result. A Gaussian profile would require unrealistically large values of the density gradient parameter and a parabolic density distribution does not enable us to obtain well constrained posterior probability distributions. However, our Model Comparison results indicate that the evidence points to the Gaussian and parabolic profiles for period ratios in between 2 and 3, while the Lorentzian profile is preferred for larger period ratio values.

Inigo Arregui - One of the best experts on this subject based on the ideXlab platform.

  • Model Comparison for the density structure across solar coronal waveguides
    The Astrophysical Journal, 2015
    Co-Authors: Inigo Arregui, Roberto Soler, A. Asensio Ramos
    Abstract:

    The spatial variation of physical quantities, such as the mass density, across solar atmospheric waveguides governs the timescales and spatial scales for wave damping and energy dissipation. The direct measurement of the spatial distribution of density, however, is difficult and indirect seismology inversion methods have been suggested as an alternative. We applied Bayesian inference, Model Comparison, and Model-averaging techniques to the inference of the cross-field density structuring in solar magnetic waveguides using information on periods and damping times for resonantly damped magnetohydrodynamic (MHD) transverse kink oscillations. Three commonly employed alternative profiles were used to Model the variation of the mass density across the waveguide boundary. Parameter inference enabled us to obtain information on physical quantities such as the Alfv\'en travel time, the density contrast, and the transverse inhomogeneity length scale. The inference results from alternative density Models were compared and their differences quantified. Then, the relative plausibility of the considered Models was assessed by performing Model Comparison. Our results indicate that the evidence in favor of any of the three Models is minimal, unless the oscillations are strongly damped. In such a circumstance, the application of Model-averaging techniques enables the computation of an evidence-weighted inference that takes into account the plausibility of each Model in the calculation of a combined inversion for the unknown physical parameters.

  • Model Comparison for the density structure along solar prominence threads
    Astronomy & Astrophysics, 2015
    Co-Authors: Inigo Arregui, Roberto Soler
    Abstract:

    Context. Quiescent solar prominence fine structures are typically Modelled as density enhancements, called threads, which occupy a fraction of a longer magnetic flux tube. This is justified from the spatial distribution of the imaged plasma emission or absorption of prominences at small spatial scales. The profile of the mass density along the magnetic field is unknown, however, and several arbitrary alternatives are employed in prominence wave studies. The identification and measurement of period ratios from multiple harmonics in standing transverse thread oscillations offer a remote diagnostics method to probe the density variation of these structures. Aims. We present a Comparison of theoretical Models for the field-aligned density along prominence fine structures. They aim to imitate density distributions in which the plasma is more or less concentrated around the centre of the magnetic flux tube. We consider Lorentzian, Gaussian, and parabolic profiles. We compare theoretical predictions based on these profiles for the period ratio between the fundamental transverse kink mode and the first overtone to obtain estimates for the density ratios between the central part of the tube and its foot-points and to assess which one would better explain observed period ratio data. Methods. Bayesian parameter inference and Model Comparison techniques were developed and applied. To infer the parameters, we computed the posterior distribution for the density gradient parameter that depends on the observable period ratio. The Model Comparison involved computing the marginal likelihood as a function of the period ratio to obtain the plausibility of each density Model as a function of the observable. We also computed the Bayes factors to quantify the relative evidence for each Model, given a period ratio observation. Results. A Lorentzian density profile, with plasma density concentrated around the centre of the tube, seems to offer the most plausible inversion result. A Gaussian profile would require unrealistically high values of the density gradient parameter, and a parabolic density distribution does not enable us to obtain well-constrained posterior probability distributions. However, our Model Comparison results indicate that the evidence points to the Gaussian and parabolic profiles for period ratios in between 2 and 3, while the Lorentzian profile is preferred for higher period ratio values. The method we present can be used to obtain information on the plasma structure along threads, provided period ratio measurements become widely available.

  • Model Comparison for the density structure along solar prominence threads
    arXiv: Solar and Stellar Astrophysics, 2015
    Co-Authors: Inigo Arregui, Roberto Soler
    Abstract:

    Quiescent solar prominence fine structures are typically Modelled as density enhancements, called threads, which occupy a fraction of a longer magnetic flux tube. The profile of the mass density along the magnetic field is however unknown and several arbitrary alternatives are employed in prominence wave studies. We present a Comparison of theoretical Models for the field-aligned density along prominence fine structures. We consider Lorentzian, Gaussian, and parabolic profiles. We compare their theoretical predictions for the period ratio between the fundamental transverse kink mode and the first overtone to obtain estimates for the ratio of densities between the central part of the tube and its foot-points and to assess which one would better explain observed period ratio data. Bayesian parameter inference and Model Comparison techniques are developed and applied. Parameter inference requires the computation of the posterior distribution for the density gradient parameter conditional on the observable period ratio. Model Comparison involves the computation of the marginal likelihood as a function of the period ratio to obtain the plausibility of each density Model and the computation of Bayes Factors to quantify the relative evidence for each Model, given a period ratio observation. A Lorentzian density profile, with plasma density concentrated around the centre of the tube seems to offer the most plausible inversion result. A Gaussian profile would require unrealistically large values of the density gradient parameter and a parabolic density distribution does not enable us to obtain well constrained posterior probability distributions. However, our Model Comparison results indicate that the evidence points to the Gaussian and parabolic profiles for period ratios in between 2 and 3, while the Lorentzian profile is preferred for larger period ratio values.

Shantanu Desai - One of the best experts on this subject based on the ideXlab platform.

  • Model Comparison of $\Lambda $CDM vs $R_h=ct$ using cosmic chronometers
    The European Physical Journal C, 2020
    Co-Authors: Haveesh Singirikonda, Shantanu Desai
    Abstract:

    In 2012, Bilicki and Seikel (Mon Not R Astron Soc 425:1664, 2012) showed that H(z) data reconstructed using Gaussian Process Regression from cosmic chronometers and baryon acoustic oscillations, conclusively rules out the $$R_h=ct$$ Model. These results were disputed by Melia and collaborators in two different works (Melia and Maier in Mon Not R Astron Soc 432:2669, 2013; Melia and Yennapureddy in JCAP 2018:034, 2018), who showed using both an unbinned analysis and Gaussian Process reconstructed H(z) data from chronometers, that $$R_h=ct$$ is favored over $$\Lambda $$CDM Model. To resolve this imbroglio, we carry out Model Comparison of $$\Lambda $$CDM versus $$R_h=ct$$ by independently reproducing the above claims using the latest chronometer data. We perform Model selection between these two Models using Bayesian Model Comparison. We find that no one Model between $$\Lambda $$CDM and $$R_h=ct$$ is decisively favored when uniform priors on $$\Lambda $$CDM parameters are used. However, if we use priors centered around the Planck best-fit values, then $$\Lambda $$CDM is very strongly preferred over $$R_h=ct$$.

  • Robust Model Comparison tests of DAMA/LIBRA annual modulation
    Journal of Cosmology and Astroparticle Physics, 2020
    Co-Authors: Aditi Krishak, Aisha Dantuluri, Shantanu Desai
    Abstract:

    We evaluate the statistical significance of the DAMA/LIBRA claims for annual modulation using three independent Model Comparison techniques, viz frequentist, information theory, and Bayesian analysis. We fit the data from the DAMA/LIBRA experiment to both cosine and a constant Model, and carry out Model Comparison by choosing the constant Model as the null hypothesis. For the frequentist test, we invoke Wilk's theorem and calculate the significance using $\Delta \chi^2$ between the two Models. For information theoretical tests, we calculate the difference in Akaike Information Criterion (AIC) and Bayesian Information criterion (BIC) between the two Models. We also compare the two Models in a Bayesian context by calculating the Bayes factor. We also search for higher harmonics in the DAMA/LIBRA data using generalized Lomb-Scargle periodogram. We finally test the sensitivity of these Model Comparison techniques in discriminating between pure noise and a cosine signal using synthetic data. This is the first proof of principles application of AIC, BIC as well as Bayes factor to the DAMA data. All our analysis codes along with the data used in this work have been made publicly available.

  • Classification of gamma-ray burst durations using robust Model-Comparison techniques
    Astrophysics and Space Science, 2017
    Co-Authors: Soham Kulkarni, Shantanu Desai
    Abstract:

    Gamma-Ray Bursts (GRBs) have been conventionally bifurcated into two distinct categories dubbed “short” and “long”, depending on whether their durations are less than or greater than two seconds respectively. However, many authors have pointed to the existence of a third class of GRBs with mean durations intermediate between the short and long GRBs. Here, we apply multiple Model Comparison techniques to verify these claims. For each category, we obtain the best-fit parameters by maximizing a likelihood function based on a weighted superposition of two (or three) lognormal distributions. We then do Model-Comparison between each of these hypotheses by comparing the chi-square probabilities, Akaike Information Criterion (AIC), and Bayesian Information Criterion (BIC). We uniformly apply these techniques to GRBs from Swift (both observer and intrinsic frame), BATSE, BeppoSAX, and Fermi-GBM. We find that the Swift GRB distributions (in the observer frame) for the entire dataset favor three categories at about \(2.4\sigma\) from difference in chi-squares, and show decisive evidence in favor of three components using both AIC and BIC. However, when the same analysis is done for the subset of Swift GRBs with measured redshifts, two components are favored with marginal significance. For all the other datasets, evidence for three components is either very marginal or disfavored.

  • Frequentist Model Comparison tests of sinusoidal variations in measurements of Newton's gravitational constant
    EPL (Europhysics Letters), 2016
    Co-Authors: Shantanu Desai
    Abstract:

    Anderson et al have claimed to find evidence for periodic sinusoidal variations (period=5.9 years) in measurements of Newton's Gravitational constant. These claims have been disputed by Pitkin. Using Bayesian Model Comparison, he argues that a Model with an unknown Gaussian noise component is favored over any periodic variations by more than $e^{30}$. We re-examine the claims of Anderson et al using frequentist Model Comparison tests, both with and without errors in the measurement times. Our findings lend support to Pitkin's claim that a constant term along with an unknown systematic offset provides a better fit to the measurements of Newton's constant, compared to any sinusoidal variations.

Ioannis Ntzoufras - One of the best experts on this subject based on the ideXlab platform.

  • Thermodynamic Bayesian Model Comparison
    Statistics and Computing, 2017
    Co-Authors: Silia Vitoratou, Ioannis Ntzoufras
    Abstract:

    Thermodynamics have been shown to have direct applications in Bayesian Model evaluation. Within a tempered transitions scheme, the Boltzmann–Gibbs distribution pertaining to different Hamiltonians is implemented to create a path which links the distributions of interest at the endpoints. As illustrated here, an optimal temperature exists along the path which directly provides the free energy, which in this context corresponds to the marginal likelihood and/or Bayes factor. Estimators which have been developed under this framework are organised here using a unifying approach, in parallel with their stepping-stone sampling counterparts. New estimators are presented and the use of compound paths is introduced. As a byproduct, it is shown how the thermodynamic integral allows for the estimation of probability distribution divergences and measures of statistical entropy. A geometric approach is employed here to illustrate the importance of the choice of the path in terms of the corresponding estimator’s error (path-related variance), which provides a more intuitive approach in tuning the error sources.

Devendra Singh Chaplot - One of the best experts on this subject based on the ideXlab platform.

  • CogSci - Comparing Model Comparison Methods
    Cognitive Science, 2013
    Co-Authors: Holger Schultheis, Ankit Singhaniya, Devendra Singh Chaplot
    Abstract:

    Comparing Model Comparison Methods Holger Schultheis (schulth@informatik.uni-bremen.de) Cognitive Systems, University of Bremen, Enrique-Schmidt-Str. 5, 28359 Bremen, Germany Ankit Singhaniya Computer Science and Engineering, NIT Nagpur, Nagpur 440010, India Devendra Singh Chaplot Computer Science and Engineering, IIT Bombay, Mumbai 400076, India Abstract Comparison of the ability of different computational cogni- tive Models to simulate empirical data should ideally take into account the complexity of the compared Models. Although several Comparison methods are available that are meant to achieve this, little information on the differential strengths and weaknesses of these methods is available. In this contribu- tion we present the results of a systematic Comparison of 5 Model Comparison methods. Employing Model recovery sim- ulations, the methods are examined with respect to their ability to identify the Model that actually generated the data across 3 pairs of Models and a number of Comparison situations. The simulations reveal several interesting aspects of the considered methods such as, for instance, the fact that in certain situa- tions methods perform worse than Model Comparison neglect- ing Model complexity. Based on the identified method charac- teristics, we derive a preliminary recommendation on when to use which of the 5 methods. Keywords: computational cognitive Models, Model compari- son, Model mimicry, Model generalization When computationally Modeling cognition, often several dif- ferent Models are available or conceivable as explanations for the cognitive ability in question. In such a situation, the aim is to select the best of these candidate Models according to a set of criteria. Among others (e.g., falsifiability or inter- pretability) the extent to which the different Models are able to simulate observed human behavior is usually considered a key criterion for selecting from the candidate Models. A na¨ive approach to gauge the Models’ ability to simulate the existing observations is to fit each Model to the available data and choose the Model that provides the tightest fit as indi- cated, for instance, by the Models’ Root Mean Squared Error (RMSE). Such an approach is problematic, because it does not take into account the complexity of the compared mod- els. As a result, there is a tendency for overfitting and for selecting more complex Models even if simpler Models pro- vide the better explanation of the considered cognitive ability (Pitt & Myung, 2002). Several methods taking into account Model complexity have been proposed to avoid the pitfalls of the na¨ive ap- proach (see Shiffrin, Lee, Kim, & Wagenmakers, 2008, for an overview). However, common use of such more sophisti- cated Model Comparison methods is partly hampered by the fact that many properties of the different methods are in- sufficiently investigated. Only very few studies (e.g., Co- hen, Sanborn, & Shiffrin, 2008) have systematically exam- ined different Comparison methods with respect to their dif- ferential advantages and disadvantages. Consequently, when faced with a situation that requires comparing Models regard- ing their ability for simulating human behavior, Modelers are often faced with the problem that it is unclear which Model Comparison methods could reasonably and should ideally be employed in a given situation. In this contribution we present the results of a systematic Comparison of 5 Model Comparison methods. The methods are examined with respect to their ability to select the Model that actually generated the data across 3 pairs of Models and a number of contextual variations (e.g., tightness of fits, amount of noise in the data). The obtained results highlight impor- tant properties of the different Comparison methods. Together with the fact that all 5 considered methods are general in the sense that they place no restrictions on the type of Models that can be compared, these results are, we believe, conducive to increasing the frequency with which more sophisticated com- parison methods instead of the na¨ive approach will be em- ployed for Model evaluation and Comparison. The remainder of this article is structured as follows. First, we list and briefly describe all considered methods. Second, the employed Models, contextual variations, and procedu- ral details of the method Comparison are described. Subse- quently, Comparison results are presented and discussed be- fore we conclude our considerations and highlight topics for future work. Methods The 5 methods we compared are the bootstrap, the bootstrap with standard error (SE) and confidence interval (CI), the data-uninformed parametric bootstrap cross-fitting method, henceforth called cross-fitting method (CM), the simple hold- out, and the prediction error difference method (PED). Each of these was applied to 3 pairs of Models and will be described in turn below. Bootstrap Given a set of n observations, the bootstrap method of Model Comparison proceeds as follows (see Efron & Tibshirani, 1993, for an overview of bootstrapping procedures). First, an arbitrary but fixed number B of bootstrap samples is gen- erated. A bootstrap sample is a set of n data points ran- domly drawn with replacement from the n original obser- vations. Due to sampling with replacement, most bootstrap samples will contain only a subset of all original observa-