The Experts below are selected from a list of 36687 Experts worldwide ranked by ideXlab platform
Richard J Samworth - One of the best experts on this subject based on the ideXlab platform.
-
local Nearest Neighbour classification with applications to semi supervised learning
Annals of Statistics, 2020Co-Authors: Timothy I Cannings, Thomas B Berrett, Richard J SamworthAbstract:We derive a new asymptotic expansion for the global excess risk of a local-$k$-Nearest Neighbour classifier, where the choice of $k$ may depend upon the test point. This expansion elucidates conditions under which the dominant contribution to the excess risk comes from the decision boundary of the optimal Bayes classifier, but we also show that if these conditions are not satisfied, then the dominant contribution may arise from the tails of the marginal distribution of the features. Moreover, we prove that, provided the $d$-dimensional marginal distribution of the features has a finite $\rho $th moment for some $\rho >4$ (as well as other regularity conditions), a local choice of $k$ can yield a rate of convergence of the excess risk of $O(n^{-4/(d+4)})$, where $n$ is the sample size, whereas for the standard $k$-Nearest Neighbour classifier, our theory would require $d\geq 5$ and $\rho >4d/(d-4)$ finite moments to achieve this rate. These results motivate a new $k$-Nearest Neighbour classifier for semi-supervised learning problems, where the unlabelled data are used to obtain an estimate of the marginal feature density, and fewer Neighbours are used for classification when this density estimate is small. Our worst-case rates are complemented by a minimax lower bound, which reveals that the local, semi-supervised $k$-Nearest Neighbour classifier attains the minimax optimal rate over our classes for the excess risk, up to a subpolynomial factor in $n$. These theoretical improvements over the standard $k$-Nearest Neighbour classifier are also illustrated through a simulation study.
-
efficient multivariate entropy estimation via k Nearest Neighbour distances
Annals of Statistics, 2019Co-Authors: Thomas B Berrett, Richard J Samworth, Ming YuanAbstract:Many statistical procedures, including goodness-of-fit tests and methods for independent component analysis, rely critically on the estimation of the entropy of a distribution. In this paper, we seek entropy estimators that are efficient and achieve the local asymptotic minimax lower bound with respect to squared error loss. To this end, we study weighted averages of the estimators originally proposed by Kozachenko and Leonenko [Probl. Inform. Transm. 23 (1987), 95–101], based on the $k$-Nearest Neighbour distances of a sample of $n$ independent and identically distributed random vectors in $\mathbb{R}^{d}$. A careful choice of weights enables us to obtain an efficient estimator in arbitrary dimensions, given sufficient smoothness, while the original unweighted estimator is typically only efficient when $d\leq 3$. In addition to the new estimator proposed and theoretical understanding provided, our results facilitate the construction of asymptotically valid confidence intervals for the entropy of asymptotically minimal width.
-
optimal weighted Nearest Neighbour classifiers
Annals of Statistics, 2012Co-Authors: Richard J SamworthAbstract:We derive an asymptotic expansion for the excess risk (regret) of a weighted Nearest-Neighbour classifier. This allows us to find the asymptotically optimal vector of nonnegative weights, which has a rather simple form. We show that the ratio of the regret of this classifier to that of an unweighted $k$-Nearest Neighbour classifier depends asymptotically only on the dimension $d$ of the feature vectors, and not on the underlying populations. The improvement is greatest when $d=4$, but thereafter decreases as $d\rightarrow\infty$. The popular bagged Nearest Neighbour classifier can also be regarded as a weighted Nearest Neighbour classifier, and we show that its corresponding weights are somewhat suboptimal when $d$ is small (in particular, worse than those of the unweighted $k$-Nearest Neighbour classifier when $d=1$), but are close to optimal when $d$ is large. Finally, we argue that improvements in the rate of convergence are possible under stronger smoothness assumptions, provided we allow negative weights. Our findings are supported by an empirical performance comparison on both simulated and real data sets.
-
properties of bagged Nearest Neighbour classifiers
Journal of The Royal Statistical Society Series B-statistical Methodology, 2005Co-Authors: Peter Hall, Richard J SamworthAbstract:Summary. It is shown that bagging, a computationally intensive method, asymptotically improves the performance of Nearest Neighbour classifiers provided that the resample size is less than 69% of the actual sample size, in the case of with-replacement bagging, or less than 50% of the sample size, for without-replacement bagging. However, for larger sampling fractions there is no asymptotic difference between the risk of the regular Nearest Neighbour classifier and its bagged version. In particular, neither achieves the large sample performance of the Bayes classifier. In contrast, when the sampling fractions converge to 0, but the resample sizes diverge to 1, the bagged classifier converges to the optimal Bayes rule and its risk converges to the risk of the latter. These results are most readily seen when the two populations have well-defined densities, but they may also be derived in other cases, where densities exist in only a relative sense. Cross-validation can be used effectively to choose the sampling fraction. Numerical calculation is used to illustrate these theoretical properties.
Chris Cornelis - One of the best experts on this subject based on the ideXlab platform.
-
fuzzy rough Nearest Neighbour classification and prediction
Theoretical Computer Science, 2011Co-Authors: Richard Jensen, Chris CornelisAbstract:Nearest Neighbour (NN) approaches are inspired by the way humans make decisions, comparing a test object to previously encountered samples. In this paper, we propose an NN algorithm that uses the lower and upper approximations from fuzzy-rough set theory in order to classify test objects, or predict their decision value. It is shown experimentally that our method outperforms other NN approaches (classical, fuzzy and fuzzy-rough ones) and that it is competitive with leading classification and prediction methods. Moreover, we show that the robustness of our methods against noise can be enhanced effectively by invoking the approximations of the Vaguely Quantified Rough Set (VQRS) model, which emulates the linguistic quantifiers ''some'' and ''most'' from natural language.
-
fuzzy rough Nearest Neighbour classification
Lecture Notes in Computer Science, 2011Co-Authors: Richard Jensen, Chris CornelisAbstract:A new fuzzy-rough Nearest Neighbour (FRNN) classification algorithm is presented in this paper, as an alternative to Sarkar's fuzzy-rough ownership function (FRNN-O) approach. By contrast to the latter, our method uses the Nearest Neighbours to construct lower and upper approximations of decision classes, and classifies test instances based on their membership to these approximations. In the experimental analysis, we evaluate our approach with both classical fuzzy-rough approximations (based on an implicator and a t-norm), as well as with the recently introduced vaguely quantified rough sets. Preliminary results are very good, and in general FRNN outperforms FRNN-O, as well as the traditional fuzzy Nearest Neighbour (FNN) algorithm.
-
a new approach to fuzzy rough Nearest Neighbour classification
LECTURE NOTES IN ARTIFICIAL INTELLIGENCE, 2008Co-Authors: Richard Jensen, Chris CornelisAbstract:In this paper; we present a new fuzzy-rough Nearest Neighbour (FRNN) classification algorithm, as an alternative to Sarkar's fuzzy-rough ownership function (FRNN-O) approach. By contrast to the latter, our method uses the Nearest Neighbours to construct lower and ripper approximations of decision classes; and classifies test instances based on their membership to these approximations. In the experimental analysis; we evaluate our approach with both classical fuzzy-rough approximations (based on an implicator and a t-norm), as well as with the recently introduced vaguely quantified rough sets. Preliminary results are very good, and in general FRNN outperforms both FRNN-O; as well as the traditional frizzy Nearest Neighbour (FNN) algorithm.
Hongdong Li - One of the best experts on this subject based on the ideXlab platform.
-
Semi-dense visual odometry for RGB-D cameras using approximate Nearest Neighbour fields
2017 IEEE International Conference on Robotics and Automation (ICRA), 2017Co-Authors: Yi Zhou, Laurent Kneip, Hongdong LiAbstract:This paper presents a robust and efficient semidense visual odometry solution for RGB-D cameras. The core of our method is a 2D-3D ICP pipeline which estimates the pose of the sensor by registering the projection of a 3D semidense map of a reference frame with the 2D semi-dense region extracted in the current frame. The processing is speeded up by efficiently implemented approximate Nearest Neighbour fields under the Euclidean distance criterion, which permits the use of compact Gauss-Newton updates in the optimization. The registration is formulated as a maximum a posterior problem to deal with outliers and sensor noise, and the equivalent weighted least squares problem is consequently solved by iteratively reweighted least squares method. A variety of robust weight functions are tested and the optimum is determined based on the probabilistic characteristics of the sensor model. Extensive evaluation on publicly available RGB-D datasets shows that the proposed method predominantly outperforms existing state-of-the-art methods.
Richard Jensen - One of the best experts on this subject based on the ideXlab platform.
-
fuzzy rough Nearest Neighbour classification and prediction
Theoretical Computer Science, 2011Co-Authors: Richard Jensen, Chris CornelisAbstract:Nearest Neighbour (NN) approaches are inspired by the way humans make decisions, comparing a test object to previously encountered samples. In this paper, we propose an NN algorithm that uses the lower and upper approximations from fuzzy-rough set theory in order to classify test objects, or predict their decision value. It is shown experimentally that our method outperforms other NN approaches (classical, fuzzy and fuzzy-rough ones) and that it is competitive with leading classification and prediction methods. Moreover, we show that the robustness of our methods against noise can be enhanced effectively by invoking the approximations of the Vaguely Quantified Rough Set (VQRS) model, which emulates the linguistic quantifiers ''some'' and ''most'' from natural language.
-
fuzzy rough Nearest Neighbour classification
Lecture Notes in Computer Science, 2011Co-Authors: Richard Jensen, Chris CornelisAbstract:A new fuzzy-rough Nearest Neighbour (FRNN) classification algorithm is presented in this paper, as an alternative to Sarkar's fuzzy-rough ownership function (FRNN-O) approach. By contrast to the latter, our method uses the Nearest Neighbours to construct lower and upper approximations of decision classes, and classifies test instances based on their membership to these approximations. In the experimental analysis, we evaluate our approach with both classical fuzzy-rough approximations (based on an implicator and a t-norm), as well as with the recently introduced vaguely quantified rough sets. Preliminary results are very good, and in general FRNN outperforms FRNN-O, as well as the traditional fuzzy Nearest Neighbour (FNN) algorithm.
-
a new approach to fuzzy rough Nearest Neighbour classification
LECTURE NOTES IN ARTIFICIAL INTELLIGENCE, 2008Co-Authors: Richard Jensen, Chris CornelisAbstract:In this paper; we present a new fuzzy-rough Nearest Neighbour (FRNN) classification algorithm, as an alternative to Sarkar's fuzzy-rough ownership function (FRNN-O) approach. By contrast to the latter, our method uses the Nearest Neighbours to construct lower and ripper approximations of decision classes; and classifies test instances based on their membership to these approximations. In the experimental analysis; we evaluate our approach with both classical fuzzy-rough approximations (based on an implicator and a t-norm), as well as with the recently introduced vaguely quantified rough sets. Preliminary results are very good, and in general FRNN outperforms both FRNN-O; as well as the traditional frizzy Nearest Neighbour (FNN) algorithm.
Yi Zhou - One of the best experts on this subject based on the ideXlab platform.
-
semi dense visual odometry for rgb d cameras using approximate Nearest Neighbour fields
arXiv: Computer Vision and Pattern Recognition, 2017Co-Authors: Yi Zhou, Laurent KneipAbstract:This paper presents a robust and efficient semi-dense visual odometry solution for RGB-D cameras. The core of our method is a 2D-3D ICP pipeline which estimates the pose of the sensor by registering the projection of a 3D semi-dense map of the reference frame with the 2D semi-dense region extracted in the current frame. The processing is speeded up by efficiently implemented approximate Nearest Neighbour fields under the Euclidean distance criterion, which permits the use of compact Gauss-Newton updates in the optimization. The registration is formulated as a maximum a posterior problem to deal with outliers and sensor noises, and consequently the equivalent weighted least squares problem is solved by iteratively reweighted least squares method. A variety of robust weight functions are tested and the optimum is determined based on the characteristics of the sensor model. Extensive evaluation on publicly available RGB-D datasets shows that the proposed method predominantly outperforms existing state-of-the-art methods.
-
Semi-dense visual odometry for RGB-D cameras using approximate Nearest Neighbour fields
2017 IEEE International Conference on Robotics and Automation (ICRA), 2017Co-Authors: Yi Zhou, Laurent Kneip, Hongdong LiAbstract:This paper presents a robust and efficient semidense visual odometry solution for RGB-D cameras. The core of our method is a 2D-3D ICP pipeline which estimates the pose of the sensor by registering the projection of a 3D semidense map of a reference frame with the 2D semi-dense region extracted in the current frame. The processing is speeded up by efficiently implemented approximate Nearest Neighbour fields under the Euclidean distance criterion, which permits the use of compact Gauss-Newton updates in the optimization. The registration is formulated as a maximum a posterior problem to deal with outliers and sensor noise, and the equivalent weighted least squares problem is consequently solved by iteratively reweighted least squares method. A variety of robust weight functions are tested and the optimum is determined based on the probabilistic characteristics of the sensor model. Extensive evaluation on publicly available RGB-D datasets shows that the proposed method predominantly outperforms existing state-of-the-art methods.
