The Experts below are selected from a list of 57711 Experts worldwide ranked by ideXlab platform

Bin Zou - One of the best experts on this subject based on the ideXlab platform.

  • Generalization Performance of lagrangian support vector machine based on markov sampling
    Journal of Statistical Planning and Inference, 2021
    Co-Authors: Jingjing Zeng, Bin Zou, Yuze Duan, Desheng Wang, Yue Yin
    Abstract:

    Abstract In this paper, we first establish the Generalization bounds of Lagrangian Support Vector Machines (LSVM) based on uniformly ergodic Markov chain (u.e.M.c.) samples. As an application, we also obtain the Generalization bounds of LSVM based on strongly mixing sequence, independent and identically distributed (i.i.d.) samples, respectively. The fast learning rates of LSVM for u.e.M.c., strongly mixing sequence and i.i.d. samples are established. We also propose a new LSVM algorithm based on Markov sampling (LSVM MS) and show the learning Performance of LSVM MS for UCI datasets. The experimental results show that the LSVM MS can improve obviously the learning Performance of the classical LSVM algorithm. If the sampling and training total time is a main concern, the LSVM MS algorithm is the preferred method compared the known SVM algorithm based on Markov sampling.

  • the Generalization Performance of regularized regression algorithms based on markov sampling
    IEEE Transactions on Systems Man and Cybernetics, 2014
    Co-Authors: Bin Zou, Yuan Yan Tang
    Abstract:

    This paper considers the Generalization ability of two regularized regression algorithms [least square regularized regression (LSRR) and support vector machine regression (SVMR)] based on non-independent and identically distributed (non-i.i.d.) samples. Different from the previously known works for non-i.i.d. samples, in this paper, we research the Generalization bounds of two regularized regression algorithms based on uniformly ergodic Markov chain (u.e.M.c.) samples. Inspired by the idea from Markov chain Monto Carlo (MCMC) methods, we also introduce a new Markov sampling algorithm for regression to generate u.e.M.c. samples from a given dataset, and then, we present the numerical studies on the learning Performance of LSRR and SVMR based on Markov sampling, respectively. The experimental results show that LSRR and SVMR based on Markov sampling can present obviously smaller mean square errors and smaller variances compared to random sampling.

  • Generalization Performance of gaussian kernels svmc based on markov sampling
    Neural Networks, 2014
    Co-Authors: Yuan Yan Tang, Bin Zou
    Abstract:

    In this paper we consider Gaussian RBF kernels support vector machine classification (SVMC) algorithm with uniformly ergodic Markov chain (u.e.M.c.) samples in reproducing kernel Hilbert spaces (RKHS). We analyze the learning rates of Gaussian RBF kernels SVMC based on u.e.M.c. samples and obtain the fast learning rate of Gaussian RBF kernels SVMC based on u.e.M.c. samples by using the strongly mixing property of u.e.M.c. samples. We also present the numerical studies on the learning Performance of Gaussian RBF kernels SVMC based on Markov sampling for real-world datasets. These experimental results show that Gaussian RBF kernels SVMC based on Markov sampling has better learning Performance compared to randomly independent sampling.

  • Generalization Performance of fisher linear discriminant based on markov sampling
    IEEE Transactions on Neural Networks, 2013
    Co-Authors: Bin Zou, Tao Luo, Yuan Yan Tang
    Abstract:

    Fisher linear discriminant (FLD) is a well-known method for dimensionality reduction and classification that projects high-dimensional data onto a low-dimensional space where the data achieves maximum class separability. The previous works describing the Generalization ability of FLD have usually been based on the assumption of independent and identically distributed (i.i.d.) samples. In this paper, we go far beyond this classical framework by studying the Generalization ability of FLD based on Markov sampling. We first establish the bounds on the Generalization Performance of FLD based on uniformly ergodic Markov chain (u.e.M.c.) samples, and prove that FLD based on u.e.M.c. samples is consistent. By following the enlightening idea from Markov chain Monto Carlo methods, we also introduce a Markov sampling algorithm for FLD to generate u.e.M.c. samples from a given data of finite size. Through simulation studies and numerical studies on benchmark repository using FLD, we find that FLD based on u.e.M.c. samples generated by Markov sampling can provide smaller misclassification rates compared to i.i.d. samples.

  • Generalization Performance of least square regularized regression algorithm with markov chain samples
    Journal of Mathematical Analysis and Applications, 2012
    Co-Authors: Bin Zou
    Abstract:

    Abstract The previously known works describing the Generalization of least-square regularized regression algorithm are usually based on the assumption of independent and identically distributed (i.i.d.) samples. In this paper we go far beyond this classical framework by studying the Generalization of least-square regularized regression algorithm with Markov chain samples. We first establish a novel concentration inequality for uniformly ergodic Markov chains, then we establish the bounds on the Generalization of least-square regularized regression algorithm with uniformly ergodic Markov chain samples, and show that least-square regularized regression algorithm with uniformly ergodic Markov chains is consistent.

Yuan Yan Tang - One of the best experts on this subject based on the ideXlab platform.

  • the Generalization Performance of regularized regression algorithms based on markov sampling
    IEEE Transactions on Systems Man and Cybernetics, 2014
    Co-Authors: Bin Zou, Yuan Yan Tang
    Abstract:

    This paper considers the Generalization ability of two regularized regression algorithms [least square regularized regression (LSRR) and support vector machine regression (SVMR)] based on non-independent and identically distributed (non-i.i.d.) samples. Different from the previously known works for non-i.i.d. samples, in this paper, we research the Generalization bounds of two regularized regression algorithms based on uniformly ergodic Markov chain (u.e.M.c.) samples. Inspired by the idea from Markov chain Monto Carlo (MCMC) methods, we also introduce a new Markov sampling algorithm for regression to generate u.e.M.c. samples from a given dataset, and then, we present the numerical studies on the learning Performance of LSRR and SVMR based on Markov sampling, respectively. The experimental results show that LSRR and SVMR based on Markov sampling can present obviously smaller mean square errors and smaller variances compared to random sampling.

  • Generalization Performance of gaussian kernels svmc based on markov sampling
    Neural Networks, 2014
    Co-Authors: Yuan Yan Tang, Bin Zou
    Abstract:

    In this paper we consider Gaussian RBF kernels support vector machine classification (SVMC) algorithm with uniformly ergodic Markov chain (u.e.M.c.) samples in reproducing kernel Hilbert spaces (RKHS). We analyze the learning rates of Gaussian RBF kernels SVMC based on u.e.M.c. samples and obtain the fast learning rate of Gaussian RBF kernels SVMC based on u.e.M.c. samples by using the strongly mixing property of u.e.M.c. samples. We also present the numerical studies on the learning Performance of Gaussian RBF kernels SVMC based on Markov sampling for real-world datasets. These experimental results show that Gaussian RBF kernels SVMC based on Markov sampling has better learning Performance compared to randomly independent sampling.

  • Generalization Performance of fisher linear discriminant based on markov sampling
    IEEE Transactions on Neural Networks, 2013
    Co-Authors: Bin Zou, Tao Luo, Yuan Yan Tang
    Abstract:

    Fisher linear discriminant (FLD) is a well-known method for dimensionality reduction and classification that projects high-dimensional data onto a low-dimensional space where the data achieves maximum class separability. The previous works describing the Generalization ability of FLD have usually been based on the assumption of independent and identically distributed (i.i.d.) samples. In this paper, we go far beyond this classical framework by studying the Generalization ability of FLD based on Markov sampling. We first establish the bounds on the Generalization Performance of FLD based on uniformly ergodic Markov chain (u.e.M.c.) samples, and prove that FLD based on u.e.M.c. samples is consistent. By following the enlightening idea from Markov chain Monto Carlo methods, we also introduce a Markov sampling algorithm for FLD to generate u.e.M.c. samples from a given data of finite size. Through simulation studies and numerical studies on benchmark repository using FLD, we find that FLD based on u.e.M.c. samples generated by Markov sampling can provide smaller misclassification rates compared to i.i.d. samples.

Wensheng Zhang - One of the best experts on this subject based on the ideXlab platform.

  • Generalization Performance of radial basis function networks
    IEEE Transactions on Neural Networks, 2015
    Co-Authors: Yunwen Lei, Lixin Ding, Wensheng Zhang
    Abstract:

    This paper studies the Generalization Performance of radial basis function (RBF) networks using local Rademacher complexities. We propose a general result on controlling local Rademacher complexities with the L-1-metric capacity. We then apply this result to estimate the RBF networks' complexities, based on which a novel estimation error bound is obtained. An effective approximation error bound is also derived by carefully investigating the Holder continuity of the l(p) loss function's derivative. Furthermore, it is demonstrated that the RBF network minimizing an appropriately constructed structural risk admits a significantly better learning rate when compared with the existing results. An empirical study is also performed to justify the application of our structural risk in model selection.

Akiko Takeda - One of the best experts on this subject based on the ideXlab platform.

  • using financial risk measures for analyzing Generalization Performance of machine learning models
    Neural Networks, 2014
    Co-Authors: Akiko Takeda, Takafumi Kanamori
    Abstract:

    We propose a unified machine learning model (UMLM) for two-class classification, regression and outlier (or novelty) detection via a robust optimization approach. The model embraces various machine learning models such as support vector machine-based and minimax probability machine-based classification and regression models. The unified framework makes it possible to compare and contrast existing learning models and to explain their differences and similarities. In this paper, after relating existing learning models to UMLM, we show some theoretical properties for UMLM. Concretely, we show an interpretation of UMLM as minimizing a well-known financial risk measure (worst-case value-at risk (VaR) or conditional VaR), derive Generalization bounds for UMLM using such a risk measure, and prove that solving problems of UMLM leads to estimators with the minimized Generalization bounds. Those theoretical properties are applicable to related existing learning models.

  • on Generalization Performance and non convex optimization of extended ν support vector machine
    New Generation Computing, 2009
    Co-Authors: Akiko Takeda, Masashi Sugiyama
    Abstract:

    The ν-support vector classification (ν-SVC) algorithm was shown to work well and provide intuitive interpretations, e.g., the parameter ν roughly specifies the fraction of support vectors. Although ν corresponds to a fraction, it cannot take the entire range between 0 and 1 in its original form. This problem was settled by a non-convex extension of ν-SVC and the extended method was experimentally shown to generalize better than original ν-SVC. However, its good Generalization Performance and convergence properties of the optimization algorithm have not been studied yet. In this paper, we provide new theoretical insights into these issues and propose a novel ν-SVC algorithm that has guaranteed Generalization Performance and convergence properties.

  • Generalization Performance of ν support vector classifier based on conditional value at risk minimization
    Neurocomputing, 2009
    Co-Authors: Akiko Takeda
    Abstract:

    We extend the conditional geometric score (CGS) classifier of Gotoh and Takeda for binary linear classification to a nonlinear one, which we call the @b-support vector classifier (SVC), and investigate the equivalence between the @b-SVC and the (extended) @n-SVC. The CGS classifier has recently been found to be equivalent to the extended @n-SVC of Perez-Cruz et al. and, especially in the convex case, equivalent to the @n-SVC of Scholkopf et al. The CGS problem is to minimize a risk measure known as the conditional value-at-risk (@b-CVaR). In this paper, we discuss theoretical aspects, mainly Generalization Performance, of the @b-SVC. The formula of a Generalization error bound includes the @b-CVaR or a related quantity. It implies that the minimum @b-CVaR leads to a small Generalization error bound of the @b-SVC. The viewpoint of CVaR minimization is useful to ensure the validity of not only the @b-SVC but also the (extended) @n-SVC.

Nadim Obeid - One of the best experts on this subject based on the ideXlab platform.

  • improving extreme learning machine by competitive swarm optimization and its application for medical diagnosis problems
    Expert Systems With Applications, 2018
    Co-Authors: Mohammed Eshtay, Hossam Faris, Nadim Obeid
    Abstract:

    Abstract Extreme Learning Machine (ELM) is swiftly gaining popularity as a way to train Single hidden Layer Feedforward Networks (SLFN) for its attractive properties. ELM is a fast learning network with remarkable Generalization Performance. Although ELM generally can outperform traditional gradient descent-based algorithms such as Backpropagation, its Performance can be highly affected by the random selection of the input weights and hidden biases of SLFN. Moreover, ELM networks tend to have more hidden neurons due to this random selection. In this paper, we propose a new model that uses Competitive Swarm Optimizer (CSO) to optimize the values of the input weights and hidden neurons of ELM. Two versions of ELM are considered: the classical ELM and the regularized version. The goal of the model is to increase the Generalization Performance, stabilize the classifier, and to produce more compact networks by reducing the number of neurons in the hidden layer. The proposed model is experimented based on 15 medical classification problems. Experimental results demonstrate that the proposed model can achieve better Generalization Performance with smaller number of hidden neurons and with higher stability. In addition, it requires much less training time compared to other metaheuristic based ELMs.