The Experts below are selected from a list of 22038 Experts worldwide ranked by ideXlab platform
P N Suganthan - One of the best experts on this subject based on the ideXlab platform.
-
enhancing multi class classification of Random forest using Random Vector functional neural network and oblique decision surfaces
International Joint Conference on Neural Network, 2018Co-Authors: Rakesh Katuwal, P N SuganthanAbstract:Both neural networks and decision trees are popular machine learning methods and are widely used to solve problems from diverse domains. These two classifiers are commonly used base classifiers in an ensemble framework. In this paper, we first present a new variant of oblique decision tree based on a linear classifier, then construct an ensemble classifier based on the fusion of a fast neural network, Random Vector functional link network and oblique decision trees. Random Vector Functional Link Network has an elegant closed form solution with extremely short training time. The neural network partitions each training bag (obtained using bagging) at the root level into C subsets where C is the number of classes in the dataset and subsequently, C oblique decision trees are trained on such partitions. The proposed method provides a rich insight into the data by grouping the confusing or hard to classify samples for each class and thus, provides an opportunity to employ fine-grained classification rule over the data. The performance of the ensemble classifier is evaluated on several multi-class datasets where it demonstrates a superior performance compared to other state-of-the-art classifiers.
-
ensemble incremental learning Random Vector functional link network for short term electric load forecasting
Knowledge Based Systems, 2018Co-Authors: P N Suganthan, G A J AmaratungaAbstract:Abstract Short-term electric load forecasting plays an important role in the management of modern power systems. Improving the accuracy and efficiency of electric load forecasting can help power utilities design reasonable operational planning which will lead to the improvement of economic and social benefits of the systems. A hybrid incremental learning approach composed of Discrete Wavelet Transform (DWT), Empirical Mode Decomposition (EMD) and Random Vector Functional Link network (RVFL) is presented in this work. RVFL network is a universal approximator with good efficiency because of the Randomly generated weights between input and hidden layers and the close form solution for parameter computation. By introducing incremental learning, along with ensemble approach via DWT and EMD into RVFL network, the forecasting performance can be significantly improved with respect to both efficiency and accuracy. The electric load datasets from Australian Energy Market Operator (AEMO) were used to evaluate the effectiveness of the proposed incremental DWT-EMD based RVFL network. Moreover, the attractiveness of the proposed method can be demonstrated by the comparison with eight benchmark forecasting methods.
-
benchmarking ensemble classifiers with novel co trained kernal ridge regression and Random Vector functional link ensembles research frontier
IEEE Computational Intelligence Magazine, 2017Co-Authors: Le Zhang, P N SuganthanAbstract:Studies in machine learning have shown promising classification performance of ensemble methods employing "perturb and combine" strategies. In particular, the classical Random forest algorithm performs the best among 179 classifiers on 121 UCI datasets from different domains. Motivated by this observation, we extend our previous work on oblique decision tree ensemble. We also propose an efficient co-trained kernel ridge regression method. In addition, a Random Vector functional link network ensemble is also introduced. Our experiments show that our two oblique decision tree ensemble variants and the co-trained kernel ridge regression ensemble are the top three ranked methods among the 183 classifiers. The proposed Random Vector functional link network ensemble also outperforms all neural network based methods used in the experiments.
-
visual tracking with convolutional Random Vector functional link network
IEEE Transactions on Systems Man and Cybernetics, 2017Co-Authors: Le Zhang, P N SuganthanAbstract:Deep neural network-based methods have recently achieved excellent performance in visual tracking task. As very few training samples are available in visual tracking task, those approaches rely heavily on extremely large auxiliary dataset such as ImageNet to pretrain the model. In order to address the discrepancy between the source domain (the auxiliary data) and the target domain (the object being tracked), they need to be finetuned during the tracking process. However, those methods suffer from sensitivity to the hyper-parameters such as learning rate, maximum number of epochs, size of mini-batch, and so on. Thus, it is worthy to investigate whether pretraining and fine tuning through conventional back-prop is essential for visual tracking. In this paper, we shed light on this line of research by proposing convolutional Random Vector functional link (CRVFL) neural network, which can be regarded as a marriage of the convolutional neural network and Random Vector functional link network, to simplify the visual tracking system. The parameters in the convolutional layer are Randomly initialized and kept fixed. Only the parameters in the fully connected layer need to be learned. We further propose an elegant approach to update the tracker. In the widely used visual tracking benchmark, without any auxiliary data, a single CRVFL model achieves 79.0% with a threshold of 20 pixels for the precision plot. Moreover, an ensemble of CRVFL yields comparatively the best result of 86.3%.
-
an ensemble of decision trees with Random Vector functional link networks for multi class classification
Applied Soft Computing, 2017Co-Authors: Rakesh Katuwal, P N Suganthan, Le ZhangAbstract:Abstract Ensembles of decision trees and neural networks are popular choices for solving classification and regression problems. In this paper, a new ensemble of classifiers that consists of decision trees and Random Vector functional link network is proposed for multi-class classification. The Random Vector functional link network (RVFL) partitions the original training samples into K distinct subsets, where K is the number of classes in a data set, and a decision tree is induced for each subset. Both univariate and multivariate (oblique) decision trees are used with RVFL. The performance of the proposed method is evaluated on 65 multi-class UCI datasets. The results demonstrate that the classification accuracy of the proposed ensemble method is significantly better than other state-of-the-art classifiers for medium and large sized data sets.
Le Zhang - One of the best experts on this subject based on the ideXlab platform.
-
received signal strength based indoor positioning using a Random Vector functional link network
IEEE Transactions on Industrial Informatics, 2018Co-Authors: Le Zhang, Bing Li, Wei Meng, Haixia WangAbstract:Fingerprinting based indoor positioning system is gaining more research interest under the umbrella of location-based services. However, existing works have certain limitations in addressing issues such as noisy measurements, high computational complexity, and poor generalization ability. In this work, a Random Vector functional link network based approach is introduced to address these issues. In the proposed system, a subset of informative features from many Randomized noisy features is selected to both reduce the computational complexity and boost the generalization ability. Moreover, the feature selector and predictor are jointly learned iteratively in a single framework based on an augmented Lagrangian method. The proposed system is appealing as it can be naturally fit into parallel or distributed computing environment. Extensive real-world indoor localization experiments are conducted on users with smartphone devices and results demonstrate the superiority of the proposed method over the existing approaches.
-
benchmarking ensemble classifiers with novel co trained kernal ridge regression and Random Vector functional link ensembles research frontier
IEEE Computational Intelligence Magazine, 2017Co-Authors: Le Zhang, P N SuganthanAbstract:Studies in machine learning have shown promising classification performance of ensemble methods employing "perturb and combine" strategies. In particular, the classical Random forest algorithm performs the best among 179 classifiers on 121 UCI datasets from different domains. Motivated by this observation, we extend our previous work on oblique decision tree ensemble. We also propose an efficient co-trained kernel ridge regression method. In addition, a Random Vector functional link network ensemble is also introduced. Our experiments show that our two oblique decision tree ensemble variants and the co-trained kernel ridge regression ensemble are the top three ranked methods among the 183 classifiers. The proposed Random Vector functional link network ensemble also outperforms all neural network based methods used in the experiments.
-
visual tracking with convolutional Random Vector functional link network
IEEE Transactions on Systems Man and Cybernetics, 2017Co-Authors: Le Zhang, P N SuganthanAbstract:Deep neural network-based methods have recently achieved excellent performance in visual tracking task. As very few training samples are available in visual tracking task, those approaches rely heavily on extremely large auxiliary dataset such as ImageNet to pretrain the model. In order to address the discrepancy between the source domain (the auxiliary data) and the target domain (the object being tracked), they need to be finetuned during the tracking process. However, those methods suffer from sensitivity to the hyper-parameters such as learning rate, maximum number of epochs, size of mini-batch, and so on. Thus, it is worthy to investigate whether pretraining and fine tuning through conventional back-prop is essential for visual tracking. In this paper, we shed light on this line of research by proposing convolutional Random Vector functional link (CRVFL) neural network, which can be regarded as a marriage of the convolutional neural network and Random Vector functional link network, to simplify the visual tracking system. The parameters in the convolutional layer are Randomly initialized and kept fixed. Only the parameters in the fully connected layer need to be learned. We further propose an elegant approach to update the tracker. In the widely used visual tracking benchmark, without any auxiliary data, a single CRVFL model achieves 79.0% with a threshold of 20 pixels for the precision plot. Moreover, an ensemble of CRVFL yields comparatively the best result of 86.3%.
-
an ensemble of decision trees with Random Vector functional link networks for multi class classification
Applied Soft Computing, 2017Co-Authors: Rakesh Katuwal, P N Suganthan, Le ZhangAbstract:Abstract Ensembles of decision trees and neural networks are popular choices for solving classification and regression problems. In this paper, a new ensemble of classifiers that consists of decision trees and Random Vector functional link network is proposed for multi-class classification. The Random Vector functional link network (RVFL) partitions the original training samples into K distinct subsets, where K is the number of classes in a data set, and a decision tree is induced for each subset. Both univariate and multivariate (oblique) decision trees are used with RVFL. The performance of the proposed method is evaluated on 65 multi-class UCI datasets. The results demonstrate that the classification accuracy of the proposed ensemble method is significantly better than other state-of-the-art classifiers for medium and large sized data sets.
-
a comprehensive evaluation of Random Vector functional link networks
Information Sciences, 2016Co-Authors: Le Zhang, P N SuganthanAbstract:With Randomly generated weights between input and hidden layers, a Random Vector functional link network is a universal approximator for continuous functions on compact sets with fast learning property. Though it was proposed two decades ago, the classification ability of this family of networks has not been fully investigated yet. Through a very comprehensive evaluation by using 121 UCI datasets, the effect of bias in the output layer, direct links from the input layer to the output layer and type of activation functions in the hidden layer, scaling of parameter Randomization as well as the solution procedure for the output weights are investigated in this work. Surprisingly, we found that the direct link plays an important performance enhancing role in RVFL, while the bias term in the output neuron had no significant effect. The ridge regression based closed-form solution was better than those with Moore-Penrose pseudoinverse. Instead of using a uniform Randomization in - 1,+1 for all datasets, tuning the scaling of the uniform Randomization range for each dataset enhances the overall performance. Six commonly used activation functions were investigated in this work and we found that hardlim and sign activation functions degenerate the overall performance. These basic conclusions can serve as general guidelines for designing RVFL networks based classifiers.
Aurelio Uncini - One of the best experts on this subject based on the ideXlab platform.
-
a semi supervised Random Vector functional link network based on the transductive framework
Information Sciences, 2016Co-Authors: Simone Scardapane, Danilo Comminiello, Michele Scarpiniti, Aurelio UnciniAbstract:Semi-supervised learning (SSL) is the problem of learning a function with only a partially labeled training set. It has considerable practical interest in applications where labeled data is costly to obtain, while unlabeled data is abundant. One approach to SSL in the case of binary classification is inspired by work on transductive learning (TL) by Vapnik. It has been applied prevalently using support Vector machines (SVM) as the base learning algorithm, giving rise to the so-called transductive SVM (TR-SVM). The resulting optimization problem, however, is highly non-convex and complex to solve. In this paper, we propose an alternative semi-supervised training algorithm based on the TL theory, namely semi-supervised Random Vector functional-link (RVFL) network, which is able to obtain state-of-the-art performance, while resulting in a standard convex optimization problem. In particular we show that, thanks to the characteristics of RVFLs networks, the resulting optimization problem can be safely approximated with a standard quadratic programming problem solvable in polynomial time. A wide range of experiments validate our proposal. As a comparison, we also propose a semi-supervised algorithm for RVFLs based on the theory of manifold regularization.
-
A comparison of consensus strategies for distributed learning of Random Vector functional-link networks
Smart Innovation Systems and Technologies, 2016Co-Authors: Roberto Fierimonte, Simone Scardapane, Massimo Panella, Aurelio UnciniAbstract:© Springer International Publishing Switzerland 2016.Distributed machine learning is the problem of inferring a desired relation when the training data is distributed throughout a network of agents (e.g. robots in a robot swarm). Multiple families of distributed learning algorithms are based on the decentralized average consensus (DAC) protocol, an efficient algorithm for computing an average starting from local measurement Vectors. The performance of DAC, however, is strongly dependent on the choice of a weighting matrix associated to the network. In this paper, we perform a comparative analysis of the relative performance of 4 different strategies for choosing the weighting matrix. As an applicative example, we consider the distributed sequential algorithm for Random Vector Functional-Link networks. As expected, our experimental simulations show that the training time required by the algorithm is drastically reduced when considering a proper initialization of the weights.
-
distributed music classification using Random Vector functional link nets
International Joint Conference on Neural Network, 2015Co-Authors: Simone Scardapane, Roberto Fierimonte, Dianhui Wang, Massimo Panella, Aurelio UnciniAbstract:In this paper, we investigate the problem of music classification when training data is distributed throughout a network of interconnected agents (e.g. computers, or mobile devices), and it is available in a sequential stream. Under the considered setting, the task is for all the nodes, after receiving any new chunk of training data, to agree on a single classifier in a decentralized fashion, without reliance on a master node. In particular, in this paper we propose a fully decentralized, sequential learning algorithm for a class of neural networks known as Random Vector Functional-Link nets. The proposed algorithm does not require the presence of a single coordinating agent, and it is formulated exclusively in term of local exchanges between neighboring nodes, thus making it useful in a wide range of realistic situations. Experimental simulations on four music classification benchmarks show that the algorithm has comparable performance with respect to a centralized solution, where a single agent collects all the local data from every node and subsequently updates the model.
-
Learning from distributed data sources using Random Vector functional-link networks
Procedia Computer Science, 2015Co-Authors: Simone Scardapane, Danilo Comminiello, Massimo Panella, Aurelio UnciniAbstract:One of the main characteristics in many real-world big data scenarios is their distributed nature. In a machine learning context, distributed data, together with the requirements of preserving privacy and scaling up to large networks, brings the challenge of designing fully decentralized training protocols. In this paper, we explore the problem of distributed learning when the features of every pattern are available throughout multiple agents (as is happening, for example, in a distributed database scenario). We propose an algorithm for a particular class of neural networks, known as Random Vector Functional-Link (RVFL), which is based on the Alternating Direction Method of Multipliers optimization algorithm. The proposed algorithm allows to learn an RVFL network from multiple distributed data sources, while restricting communication to the unique operation of computing a distributed average. Our experimental simulations show that the algorithm is able to achieve a generalization accuracy comparable to a fully centralized solution, while at the same time being extremely efficient.
-
Distributed learning for Random Vector Functional-Link networks
Information Sciences, 2015Co-Authors: Simone Scardapane, Dianhui Wang, Massimo Panella, Aurelio UnciniAbstract:This paper aims to develop distributed learning algorithms for Random Vector Functional-Link (RVFL) networks, where training data is distributed under a decentralized information structure. Two algorithms are proposed by using Decentralized Average Consensus (DAC) and Alternating Direction Method of Multipliers (ADMM) strategies, respectively. These algorithms work in a fully distributed fashion and have no requirement on coordination from a central agent during the learning process. For distributed learning, the goal is to build a common learner model which optimizes the system performance over the whole set of local data. In this work, it is assumed that all stations know the initial weights of the input layer, the output weights of local RVFL networks can be shared through communication channels among neighboring nodes only, and local datasets are blocked strictly. The proposed learning algorithms are evaluated over five benchmark datasets. Experimental results with comparisons show that the DAC-based learning algorithm performs favorably in terms of effectiveness, efficiency and computational complexity, followed by the ADMM-based learning algorithm with promising accuracy but higher computational burden.
G A J Amaratunga - One of the best experts on this subject based on the ideXlab platform.
-
ensemble incremental learning Random Vector functional link network for short term electric load forecasting
Knowledge Based Systems, 2018Co-Authors: P N Suganthan, G A J AmaratungaAbstract:Abstract Short-term electric load forecasting plays an important role in the management of modern power systems. Improving the accuracy and efficiency of electric load forecasting can help power utilities design reasonable operational planning which will lead to the improvement of economic and social benefits of the systems. A hybrid incremental learning approach composed of Discrete Wavelet Transform (DWT), Empirical Mode Decomposition (EMD) and Random Vector Functional Link network (RVFL) is presented in this work. RVFL network is a universal approximator with good efficiency because of the Randomly generated weights between input and hidden layers and the close form solution for parameter computation. By introducing incremental learning, along with ensemble approach via DWT and EMD into RVFL network, the forecasting performance can be significantly improved with respect to both efficiency and accuracy. The electric load datasets from Australian Energy Market Operator (AEMO) were used to evaluate the effectiveness of the proposed incremental DWT-EMD based RVFL network. Moreover, the attractiveness of the proposed method can be demonstrated by the comparison with eight benchmark forecasting methods.
-
Random Vector functional link network for short term electricity load demand forecasting
Information Sciences, 2016Co-Authors: P N Suganthan, Narasimalu Srikanth, G A J AmaratungaAbstract:Short-term electricity load forecasting plays an important role in the energy market as accurate forecasting is beneficial for power dispatching, unit commitment, fuel allocation and so on. This paper reviews a few single hidden layer network configurations with Random weights (RWSLFN). The RWSLFN was extended to eight variants based on the presence or absence of input layer bias, hidden layer bias and direct input-output connections. In order to avoid mapping the weighted inputs into the saturation region of the enhancement nodes' activation function and to suppress the outliers in the input data, a quantile scaling algorithm to re-distribute the Randomly weighted inputs is proposed. The eight variations of RWSLFN are assessed using six generic time series datasets and 12 load demand time series datasets. The result shows that the RWSLFNs with direct input-output connections (known as the Random Vector functional link network or RVFL network) have statistically significantly better performance than the RWSLFN configurations without direct input-output connections, possibly due to the fact that the direct input-output connections in the RVFL network emulate the time delayed finite impulse response (FIR) filter. However the RVFL network has simpler training and higher accuracy than the FIR based two stage neural network. The RVFL network is also compared with some reported forecasting methods. The RVFL network overall outperforms the non-ensemble methods, namely the persistence method, seasonal autoregressive integrated moving average (sARIMA), artificial neural network (ANN). In addition, the testing time of the RVFL network is the shortest while the training time is comparable to the other reported methods. Finally, possible future research directions are pointed out.
-
electricity load demand time series forecasting with empirical mode decomposition based Random Vector functional link network
Systems Man and Cybernetics, 2016Co-Authors: P N Suganthan, G A J AmaratungaAbstract:Short-term electricity load demand forecasting is a critical process in the management of modern power system. An ensemble method composed of Empirical Mode Decomposition (EMD) and Random Vector Functional Link network (RVFL) is presented in this paper. Due to the Randomly generated weights between input and hidden layers and the close form solution for parameter tuning, RVFL network is a universal approximator with the advantages of fast training. By introducing ensemble approach via EMD into RVFL network, the performance can be significantly improved. Five electricity load demand datasets from Australian Energy Market Operator (AEMO) were used to evaluate the performance of the proposed method. The attractiveness of the proposed EMD based RVFL network can be demonstrated by the comparison with six benchmark methods.
-
detecting wind power ramp with Random Vector functional link rvfl network
IEEE Symposium Series on Computational Intelligence, 2015Co-Authors: P N Suganthan, G A J AmaratungaAbstract:Due to the intermittent nature of the wind, the wind speed is fluctuating. Fluctuating wind speed cause even more fluctuation in wind power generation. The sudden changes of the wind power injected into the power grid within a short time frame is known as power ramp, which can be harmful to the grid. This paper presents algorithms to detect the wind power ramps in a certain forecasting horizon. The importance and challenges of wind power ramp detection are addressed. Several different Wind power ramps are defined in this paper. A Random Vector functional link (RVFL) network is employed to predict the future occurrence of wind power ramp. The forecasting methods are evaluated with a real world wind power data set. The RVFL network has comparable performance as the benchmark methods: Random forests (RF) and support Vector machine (SVM) but it has better performance than the artificial neural network (ANN). The computation time of training and testing is also in favor of the RVFL network.
Rakesh Katuwal - One of the best experts on this subject based on the ideXlab platform.
-
enhancing multi class classification of Random forest using Random Vector functional neural network and oblique decision surfaces
International Joint Conference on Neural Network, 2018Co-Authors: Rakesh Katuwal, P N SuganthanAbstract:Both neural networks and decision trees are popular machine learning methods and are widely used to solve problems from diverse domains. These two classifiers are commonly used base classifiers in an ensemble framework. In this paper, we first present a new variant of oblique decision tree based on a linear classifier, then construct an ensemble classifier based on the fusion of a fast neural network, Random Vector functional link network and oblique decision trees. Random Vector Functional Link Network has an elegant closed form solution with extremely short training time. The neural network partitions each training bag (obtained using bagging) at the root level into C subsets where C is the number of classes in the dataset and subsequently, C oblique decision trees are trained on such partitions. The proposed method provides a rich insight into the data by grouping the confusing or hard to classify samples for each class and thus, provides an opportunity to employ fine-grained classification rule over the data. The performance of the ensemble classifier is evaluated on several multi-class datasets where it demonstrates a superior performance compared to other state-of-the-art classifiers.
-
an ensemble of decision trees with Random Vector functional link networks for multi class classification
Applied Soft Computing, 2017Co-Authors: Rakesh Katuwal, P N Suganthan, Le ZhangAbstract:Abstract Ensembles of decision trees and neural networks are popular choices for solving classification and regression problems. In this paper, a new ensemble of classifiers that consists of decision trees and Random Vector functional link network is proposed for multi-class classification. The Random Vector functional link network (RVFL) partitions the original training samples into K distinct subsets, where K is the number of classes in a data set, and a decision tree is induced for each subset. Both univariate and multivariate (oblique) decision trees are used with RVFL. The performance of the proposed method is evaluated on 65 multi-class UCI datasets. The results demonstrate that the classification accuracy of the proposed ensemble method is significantly better than other state-of-the-art classifiers for medium and large sized data sets.