The Experts below are selected from a list of 23007 Experts worldwide ranked by ideXlab platform
Yuya Kaneda - One of the best experts on this subject based on the ideXlab platform.
-
study on the effect of learning parameters on Decision Boundary making algorithm
Systems Man and Cybernetics, 2014Co-Authors: Yuya Kaneda, Qiangfu Zhao, Yan Pei, Yong LiuAbstract:The purpose of our study is to induce compact and high performance machine learning models. In our earlier study, we proposed a Decision Boundary making (DBM) algorithm. The main philosophy of the DBM algorithm is to reconstruct a high performance model with much smaller cost. In our study, we use support vector machine as a high performance model, and a multilayer neural network, i.e., multilayer perceptron (MLP), as the small model. Experimental results obtained so far show that high performance and compact MLPs can be obtained using DBM. However, there are several parameters of DBM that need to be adjusted appropriately in order to achieve better performance. In this paper, we investigate the effect of parameter N ,w hich is the number of newly generated data, on the performance of obtained MLPs. We discuss the issue that how many new data we should generate to obtain a better performance of DBM. We also investigate the effect of outliers on the performance of the obtained MLPs. Outliers are generally known to be harmful for pattern recognition. Our experimental results show, however, that for some databases, outliers can be useful for obtaining high performance MLPs. Index Terms—Support Vector Machine, Neural Network, Deci- sion Boundary Learning, Decision Boundary Making, Awareness Agents
-
SMC - Study on the Effect of Learning Parameters on Decision Boundary Making Algorithm
2014 IEEE International Conference on Systems Man and Cybernetics (SMC), 2014Co-Authors: Yuya Kaneda, Qiangfu Zhao, Yan Pei, Yong LiuAbstract:The purpose of our study is to induce compact and high performance machine learning models. In our earlier study, we proposed a Decision Boundary making (DBM) algorithm. The main philosophy of the DBM algorithm is to reconstruct a high performance model with much smaller cost. In our study, we use support vector machine as a high performance model, and a multilayer neural network, i.e., multilayer perceptron (MLP), as the small model. Experimental results obtained so far show that high performance and compact MLPs can be obtained using DBM. However, there are several parameters of DBM that need to be adjusted appropriately in order to achieve better performance. In this paper, we investigate the effect of parameter N ,w hich is the number of newly generated data, on the performance of obtained MLPs. We discuss the issue that how many new data we should generate to obtain a better performance of DBM. We also investigate the effect of outliers on the performance of the obtained MLPs. Outliers are generally known to be harmful for pattern recognition. Our experimental results show, however, that for some databases, outliers can be useful for obtaining high performance MLPs. Index Terms—Support Vector Machine, Neural Network, Deci- sion Boundary Learning, Decision Boundary Making, Awareness Agents
-
inducing high performance neural networks based on an improved Decision Boundary making algorithm
International Joint Conference on Awareness Science and Technology & Ubi-Media Computing, 2013Co-Authors: Yuya Kaneda, Qiangfu Zhao, Yong Liu, Neil Y YenAbstract:In recent years, portable computing devices (PCDs) such as smart phones are becoming more and more popular, and many users are using applications on their PCDs. To customize applications for each user, we suggest to use awareness agents (A-agents) that can help users. However, A-agents usually become large. To reduce the size of A-agents, we have proposed Decision Boundary learning (DBL) based on particle swarm optimization (PSO) algorithm. Through experiments, we can get a compact and high performance A-agent. However, the training time becomes very long. Because, the calculation cost of PSO algorithm is very high. To reduce the calculation cost, we propose a simple method called Decision Boundary making (DBM) algorithm in this paper. The basic idea of this algorithm is to generate new training data around support vectors (S Vs) of an S VM. Then, an NN is obtained from these new training data. And, for generating data effectively, we set a condition for adding data. Experimental results show that the proposed DBM outperforms DBL, and its learning time is shorter.
-
induction of high performance neural networks based on Decision Boundary making
Systems Man and Cybernetics, 2013Co-Authors: Yuya Kaneda, Qiangfu Zhao, Yong Liu, Neil Y YenAbstract:Smartphone, in recent year, becomes popular and has been widely applied by users. In order to meet different needs from users, embedding "awareness" providing supports by understanding onto smartphone devices is necessry. Due to the limitations (e.g. computing resources, etc.) on smartphone, methods that is light but with high performance are strongly expected. In this study, the concept of awareness agent (A-agent) is proposed for the purpose. For this purpose, we have proposed Decision Boundary learning (DBL) based on particle swarm optimization (PSO). Results show that this method can yield compact neural network (NNs) agents that are comparable in performance to support vector machines (SVMs). However, the computational cost of PSO is high, and the method cannot be used in smartphone environments. To reduce the computational cost, we propose a simple method called Decision Boundary making (DBM). The basic idea of DBM is to generate new training data around the support vectors of an SVM, add them to the training set, and then induce an NN agent. We conducted experiments using several public databases, and experimental results show that the proposed DBM is comparable to DBL in performance, and the computational cost can be greatly reduced.
-
Decision Boundary learning based on an improved pso algorithm
Systems Man and Cybernetics, 2012Co-Authors: Kyohei Watarai, Qiangfu Zhao, Yuya KanedaAbstract:The goal of this research is to design a multimedia analyzer (MA) that can be embedded in portable devices. This MA can recognize different multimedia (e.g. text and image) patterns and help the user to analyze the multimedia contents more efficiently. To realize the MA in an environment with limited computing resource, we propose a new concept called Decision Boundary learning (DBL). The basic idea is to generate training patterns close to the Decision Boundary (DB), so that a neural network (NN) with high generalization ability can be obtained. In this paper, the DB is first obtained approximately using a support vector machine (SVM), and the desired training patterns are found using an improved particle swarm optimization (PSO) algorithm. Experimental results show that the NNs so obtained are comparable in performance to the SVMs although the former are much more compact.
Qiangfu Zhao - One of the best experts on this subject based on the ideXlab platform.
-
ICIA - Bounded learning for neural network ensembles
2015 IEEE International Conference on Information and Automation, 2015Co-Authors: Qiangfu ZhaoAbstract:Two error bounds were introduced in the learning process of balanced ensemble learning. They are the lower bound of error rate (LBER) and the upper bound of error output (UBEO) on the training set, respectively. These two error bounds would decide whether a training data point should be further learned or not after balanced ensemble learning has reached certain stage. Before the error rates are higher than LBER, the whole training set is fed to balanced ensemble learning. After the error rates are lower than LBER, not the whole training set but only those data points near to the learned Decision Boundary should be learned. Other data points further away from the Decision Boundary could either be learned well or not be learned at all. In order to cope with these not-yet-learned data far away from the learned Decision Boundary, balanced ensemble learning has too make so big changes to the learned Decision Boundary that the ensemble could grow too complex for the applications. Therefore, these not-yet-learned data should be excluded from the training set. There is no much impact to the learned Decision Boundary by removing those well-learned data points that are far away from the Decision Boundary. Experimental results would display how LBER and UBEO would let balanced ensemble learning avoid overfitting.
-
study on the effect of learning parameters on Decision Boundary making algorithm
Systems Man and Cybernetics, 2014Co-Authors: Yuya Kaneda, Qiangfu Zhao, Yan Pei, Yong LiuAbstract:The purpose of our study is to induce compact and high performance machine learning models. In our earlier study, we proposed a Decision Boundary making (DBM) algorithm. The main philosophy of the DBM algorithm is to reconstruct a high performance model with much smaller cost. In our study, we use support vector machine as a high performance model, and a multilayer neural network, i.e., multilayer perceptron (MLP), as the small model. Experimental results obtained so far show that high performance and compact MLPs can be obtained using DBM. However, there are several parameters of DBM that need to be adjusted appropriately in order to achieve better performance. In this paper, we investigate the effect of parameter N ,w hich is the number of newly generated data, on the performance of obtained MLPs. We discuss the issue that how many new data we should generate to obtain a better performance of DBM. We also investigate the effect of outliers on the performance of the obtained MLPs. Outliers are generally known to be harmful for pattern recognition. Our experimental results show, however, that for some databases, outliers can be useful for obtaining high performance MLPs. Index Terms—Support Vector Machine, Neural Network, Deci- sion Boundary Learning, Decision Boundary Making, Awareness Agents
-
SMC - Study on the Effect of Learning Parameters on Decision Boundary Making Algorithm
2014 IEEE International Conference on Systems Man and Cybernetics (SMC), 2014Co-Authors: Yuya Kaneda, Qiangfu Zhao, Yan Pei, Yong LiuAbstract:The purpose of our study is to induce compact and high performance machine learning models. In our earlier study, we proposed a Decision Boundary making (DBM) algorithm. The main philosophy of the DBM algorithm is to reconstruct a high performance model with much smaller cost. In our study, we use support vector machine as a high performance model, and a multilayer neural network, i.e., multilayer perceptron (MLP), as the small model. Experimental results obtained so far show that high performance and compact MLPs can be obtained using DBM. However, there are several parameters of DBM that need to be adjusted appropriately in order to achieve better performance. In this paper, we investigate the effect of parameter N ,w hich is the number of newly generated data, on the performance of obtained MLPs. We discuss the issue that how many new data we should generate to obtain a better performance of DBM. We also investigate the effect of outliers on the performance of the obtained MLPs. Outliers are generally known to be harmful for pattern recognition. Our experimental results show, however, that for some databases, outliers can be useful for obtaining high performance MLPs. Index Terms—Support Vector Machine, Neural Network, Deci- sion Boundary Learning, Decision Boundary Making, Awareness Agents
-
inducing high performance neural networks based on an improved Decision Boundary making algorithm
International Joint Conference on Awareness Science and Technology & Ubi-Media Computing, 2013Co-Authors: Yuya Kaneda, Qiangfu Zhao, Yong Liu, Neil Y YenAbstract:In recent years, portable computing devices (PCDs) such as smart phones are becoming more and more popular, and many users are using applications on their PCDs. To customize applications for each user, we suggest to use awareness agents (A-agents) that can help users. However, A-agents usually become large. To reduce the size of A-agents, we have proposed Decision Boundary learning (DBL) based on particle swarm optimization (PSO) algorithm. Through experiments, we can get a compact and high performance A-agent. However, the training time becomes very long. Because, the calculation cost of PSO algorithm is very high. To reduce the calculation cost, we propose a simple method called Decision Boundary making (DBM) algorithm in this paper. The basic idea of this algorithm is to generate new training data around support vectors (S Vs) of an S VM. Then, an NN is obtained from these new training data. And, for generating data effectively, we set a condition for adding data. Experimental results show that the proposed DBM outperforms DBL, and its learning time is shorter.
-
induction of high performance neural networks based on Decision Boundary making
Systems Man and Cybernetics, 2013Co-Authors: Yuya Kaneda, Qiangfu Zhao, Yong Liu, Neil Y YenAbstract:Smartphone, in recent year, becomes popular and has been widely applied by users. In order to meet different needs from users, embedding "awareness" providing supports by understanding onto smartphone devices is necessry. Due to the limitations (e.g. computing resources, etc.) on smartphone, methods that is light but with high performance are strongly expected. In this study, the concept of awareness agent (A-agent) is proposed for the purpose. For this purpose, we have proposed Decision Boundary learning (DBL) based on particle swarm optimization (PSO). Results show that this method can yield compact neural network (NNs) agents that are comparable in performance to support vector machines (SVMs). However, the computational cost of PSO is high, and the method cannot be used in smartphone environments. To reduce the computational cost, we propose a simple method called Decision Boundary making (DBM). The basic idea of DBM is to generate new training data around the support vectors of an SVM, add them to the training set, and then induce an NN agent. We conducted experiments using several public databases, and experimental results show that the proposed DBM is comparable to DBL in performance, and the computational cost can be greatly reduced.
Kyohei Watarai - One of the best experts on this subject based on the ideXlab platform.
-
Decision Boundary learning based on an improved pso algorithm
Systems Man and Cybernetics, 2012Co-Authors: Kyohei Watarai, Qiangfu Zhao, Yuya KanedaAbstract:The goal of this research is to design a multimedia analyzer (MA) that can be embedded in portable devices. This MA can recognize different multimedia (e.g. text and image) patterns and help the user to analyze the multimedia contents more efficiently. To realize the MA in an environment with limited computing resource, we propose a new concept called Decision Boundary learning (DBL). The basic idea is to generate training patterns close to the Decision Boundary (DB), so that a neural network (NN) with high generalization ability can be obtained. In this paper, the DB is first obtained approximately using a support vector machine (SVM), and the desired training patterns are found using an improved particle swarm optimization (PSO) algorithm. Experimental results show that the NNs so obtained are comparable in performance to the SVMs although the former are much more compact.
-
Decision Boundary learning based on particle swarm optimization
IEEE International Conference on Adaptive Science & Technology, 2012Co-Authors: Kyohei Watarai, Qiangfu Zhao, Yuya KanedaAbstract:In neural network (NN) learning, we usually find an NN to minimize the approximation error for a given training set. Depends on the data given, the performance of the NN can vary significantly. In fact, if the training data are close to the true Decision Boundary (DB), the NN can generalize well. On the other hand, if the given data are far away from the true DB, the DB formed by the NN can be very different from the original one, and the genelization ability of the NN cannot be high. Based on this observation, we propose a new concept called Decision Boundary learning (DBL) in this study. A direct way for DBL is to approximate the true DB using a support vector machine (SVM), and then find a set of training patterns using particle swarm optimization (PSO). Experimental results on four public databases show that the training patterns so obtained may generate much better NNs, and in all cases, the NNs are comparable to or better than SVMs, although they are much simpler.
-
SMC - Decision Boundary learning based on an improved PSO algorithm
2012 IEEE International Conference on Systems Man and Cybernetics (SMC), 2012Co-Authors: Kyohei Watarai, Qiangfu Zhao, Yuya KanedaAbstract:The goal of this research is to design a multimedia analyzer (MA) that can be embedded in portable devices. This MA can recognize different multimedia (e.g. text and image) patterns and help the user to analyze the multimedia contents more efficiently. To realize the MA in an environment with limited computing resource, we propose a new concept called Decision Boundary learning (DBL). The basic idea is to generate training patterns close to the Decision Boundary (DB), so that a neural network (NN) with high generalization ability can be obtained. In this paper, the DB is first obtained approximately using a support vector machine (SVM), and the desired training patterns are found using an improved particle swarm optimization (PSO) algorithm. Experimental results show that the NNs so obtained are comparable in performance to the SVMs although the former are much more compact.
-
iCAST - Decision Boundary learning based on particle swarm optimization
4th International Conference on Awareness Science and Technology, 2012Co-Authors: Kyohei Watarai, Qiangfu Zhao, Yuya KanedaAbstract:In neural network (NN) learning, we usually find an NN to minimize the approximation error for a given training set. Depends on the data given, the performance of the NN can vary significantly. In fact, if the training data are close to the true Decision Boundary (DB), the NN can generalize well. On the other hand, if the given data are far away from the true DB, the DB formed by the NN can be very different from the original one, and the genelization ability of the NN cannot be high. Based on this observation, we propose a new concept called Decision Boundary learning (DBL) in this study. A direct way for DBL is to approximate the true DB using a support vector machine (SVM), and then find a set of training patterns using particle swarm optimization (PSO). Experimental results on four public databases show that the training patterns so obtained may generate much better NNs, and in all cases, the NNs are comparable to or better than SVMs, although they are much simpler.
Yong Liu - One of the best experts on this subject based on the ideXlab platform.
-
study on the effect of learning parameters on Decision Boundary making algorithm
Systems Man and Cybernetics, 2014Co-Authors: Yuya Kaneda, Qiangfu Zhao, Yan Pei, Yong LiuAbstract:The purpose of our study is to induce compact and high performance machine learning models. In our earlier study, we proposed a Decision Boundary making (DBM) algorithm. The main philosophy of the DBM algorithm is to reconstruct a high performance model with much smaller cost. In our study, we use support vector machine as a high performance model, and a multilayer neural network, i.e., multilayer perceptron (MLP), as the small model. Experimental results obtained so far show that high performance and compact MLPs can be obtained using DBM. However, there are several parameters of DBM that need to be adjusted appropriately in order to achieve better performance. In this paper, we investigate the effect of parameter N ,w hich is the number of newly generated data, on the performance of obtained MLPs. We discuss the issue that how many new data we should generate to obtain a better performance of DBM. We also investigate the effect of outliers on the performance of the obtained MLPs. Outliers are generally known to be harmful for pattern recognition. Our experimental results show, however, that for some databases, outliers can be useful for obtaining high performance MLPs. Index Terms—Support Vector Machine, Neural Network, Deci- sion Boundary Learning, Decision Boundary Making, Awareness Agents
-
SMC - Study on the Effect of Learning Parameters on Decision Boundary Making Algorithm
2014 IEEE International Conference on Systems Man and Cybernetics (SMC), 2014Co-Authors: Yuya Kaneda, Qiangfu Zhao, Yan Pei, Yong LiuAbstract:The purpose of our study is to induce compact and high performance machine learning models. In our earlier study, we proposed a Decision Boundary making (DBM) algorithm. The main philosophy of the DBM algorithm is to reconstruct a high performance model with much smaller cost. In our study, we use support vector machine as a high performance model, and a multilayer neural network, i.e., multilayer perceptron (MLP), as the small model. Experimental results obtained so far show that high performance and compact MLPs can be obtained using DBM. However, there are several parameters of DBM that need to be adjusted appropriately in order to achieve better performance. In this paper, we investigate the effect of parameter N ,w hich is the number of newly generated data, on the performance of obtained MLPs. We discuss the issue that how many new data we should generate to obtain a better performance of DBM. We also investigate the effect of outliers on the performance of the obtained MLPs. Outliers are generally known to be harmful for pattern recognition. Our experimental results show, however, that for some databases, outliers can be useful for obtaining high performance MLPs. Index Terms—Support Vector Machine, Neural Network, Deci- sion Boundary Learning, Decision Boundary Making, Awareness Agents
-
inducing high performance neural networks based on an improved Decision Boundary making algorithm
International Joint Conference on Awareness Science and Technology & Ubi-Media Computing, 2013Co-Authors: Yuya Kaneda, Qiangfu Zhao, Yong Liu, Neil Y YenAbstract:In recent years, portable computing devices (PCDs) such as smart phones are becoming more and more popular, and many users are using applications on their PCDs. To customize applications for each user, we suggest to use awareness agents (A-agents) that can help users. However, A-agents usually become large. To reduce the size of A-agents, we have proposed Decision Boundary learning (DBL) based on particle swarm optimization (PSO) algorithm. Through experiments, we can get a compact and high performance A-agent. However, the training time becomes very long. Because, the calculation cost of PSO algorithm is very high. To reduce the calculation cost, we propose a simple method called Decision Boundary making (DBM) algorithm in this paper. The basic idea of this algorithm is to generate new training data around support vectors (S Vs) of an S VM. Then, an NN is obtained from these new training data. And, for generating data effectively, we set a condition for adding data. Experimental results show that the proposed DBM outperforms DBL, and its learning time is shorter.
-
induction of high performance neural networks based on Decision Boundary making
Systems Man and Cybernetics, 2013Co-Authors: Yuya Kaneda, Qiangfu Zhao, Yong Liu, Neil Y YenAbstract:Smartphone, in recent year, becomes popular and has been widely applied by users. In order to meet different needs from users, embedding "awareness" providing supports by understanding onto smartphone devices is necessry. Due to the limitations (e.g. computing resources, etc.) on smartphone, methods that is light but with high performance are strongly expected. In this study, the concept of awareness agent (A-agent) is proposed for the purpose. For this purpose, we have proposed Decision Boundary learning (DBL) based on particle swarm optimization (PSO). Results show that this method can yield compact neural network (NNs) agents that are comparable in performance to support vector machines (SVMs). However, the computational cost of PSO is high, and the method cannot be used in smartphone environments. To reduce the computational cost, we propose a simple method called Decision Boundary making (DBM). The basic idea of DBM is to generate new training data around the support vectors of an SVM, add them to the training set, and then induce an NN agent. We conducted experiments using several public databases, and experimental results show that the proposed DBM is comparable to DBL in performance, and the computational cost can be greatly reduced.
Jin Young Choi - One of the best experts on this subject based on the ideXlab platform.
-
AAAI - Knowledge Distillation with Adversarial Samples Supporting Decision Boundary
Proceedings of the AAAI Conference on Artificial Intelligence, 2019Co-Authors: Byeongho Heo, Minsik Lee, Sangdoo Yun, Jin Young ChoiAbstract:Many recent works on knowledge distillation have provided ways to transfer the knowledge of a trained network for improving the learning process of a new one, but finding a good technique for knowledge distillation is still an open problem. In this paper, we provide a new perspective based on a Decision Boundary, which is one of the most important component of a classifier. The generalization performance of a classifier is closely related to the adequacy of its Decision Boundary, so a good classifier bears a good Decision Boundary. Therefore, transferring information closely related to the Decision Boundary can be a good attempt for knowledge distillation. To realize this goal, we utilize an adversarial attack to discover samples supporting a Decision Boundary. Based on this idea, to transfer more accurate information about the Decision Boundary, the proposed algorithm trains a student classifier based on the adversarial samples supporting the Decision Boundary. Experiments show that the proposed method indeed improves knowledge distillation and achieves the state-of-the-arts performance.
-
Knowledge Distillation with Adversarial Samples Supporting Decision Boundary
arXiv: Learning, 2018Co-Authors: Byeongho Heo, Minsik Lee, Sangdoo Yun, Jin Young ChoiAbstract:Many recent works on knowledge distillation have provided ways to transfer the knowledge of a trained network for improving the learning process of a new one, but finding a good technique for knowledge distillation is still an open problem. In this paper, we provide a new perspective based on a Decision Boundary, which is one of the most important component of a classifier. The generalization performance of a classifier is closely related to the adequacy of its Decision Boundary, so a good classifier bears a good Decision Boundary. Therefore, transferring information closely related to the Decision Boundary can be a good attempt for knowledge distillation. To realize this goal, we utilize an adversarial attack to discover samples supporting a Decision Boundary. Based on this idea, to transfer more accurate information about the Decision Boundary, the proposed algorithm trains a student classifier based on the adversarial samples supporting the Decision Boundary. Experiments show that the proposed method indeed improves knowledge distillation and achieves the state-of-the-arts performance.
