The Experts below are selected from a list of 30126 Experts worldwide ranked by ideXlab platform
Michael W. Mahoney - One of the best experts on this subject based on the ideXlab platform.
-
Sub-sampled Newton Methods with Non-Uniform Sampling
arXiv: Optimization and Control, 2016Co-Authors: Jiyan Yang, Farbod Roosta-khorasani, Michael W. MahoneyAbstract:We consider the problem of finding the minimizer of a convex function $F: \mathbb R^d \rightarrow \mathbb R$ of the form $F(w) := \sum_{i=1}^n f_i(w) + R(w)$ where a low-rank factorization of $\nabla^2 f_i(w)$ is readily available. We consider the regime where $n \gg d$. As second-order methods prove to be effective in finding the minimizer to a high-precision, in this work, we propose randomized Newton-type algorithms that exploit \textit{non-Uniform} sub-Sampling of $\{\nabla^2 f_i(w)\}_{i=1}^{n}$, as well as inexact updates, as means to reduce the computational complexity. Two non-Uniform Sampling distributions based on {\it block norm squares} and {\it block partial leverage scores} are considered in order to capture important terms among $\{\nabla^2 f_i(w)\}_{i=1}^{n}$. We show that at each iteration non-Uniformly Sampling at most $\mathcal O(d \log d)$ terms from $\{\nabla^2 f_i(w)\}_{i=1}^{n}$ is sufficient to achieve a linear-quadratic convergence rate in $w$ when a suitable initial point is provided. In addition, we show that our algorithms achieve a lower computational complexity and exhibit more robustness and better dependence on problem specific quantities, such as the condition number, compared to similar existing methods, especially the ones based on Uniform Sampling. Finally, we empirically demonstrate that our methods are at least twice as fast as Newton's methods with ridge logistic regression on several real datasets.
-
sub sampled newton methods with non Uniform Sampling
Neural Information Processing Systems, 2016Co-Authors: Jiyan Yang, Farbod Roostakhorasani, Michael W. MahoneyAbstract:We consider the problem of finding the minimizer of a convex function $F: \mathbb R^d \rightarrow \mathbb R$ of the form $F(w) \defeq \sum_{i=1}^n f_i(w) + R(w)$ where a low-rank factorization of $\nabla^2 f_i(w)$ is readily available.We consider the regime where $n \gg d$. We propose randomized Newton-type algorithms that exploit \textit{non-Uniform} sub-Sampling of $\{\nabla^2 f_i(w)\}_{i=1}^{n}$, as well as inexact updates, as means to reduce the computational complexity, and are applicable to a wide range of problems in machine learning. Two non-Uniform Sampling distributions based on {\it block norm squares} and {\it block partial leverage scores} are considered. Under certain assumptions, we show that our algorithms inherit a linear-quadratic convergence rate in $w$ and achieve a lower computational complexity compared to similar existing methods. In addition, we show that our algorithms exhibit more robustness and better dependence on problem specific quantities, such as the condition number. We numerically demonstrate the advantages of our algorithms on several real datasets.
Gerhard Wagner - One of the best experts on this subject based on the ideXlab platform.
-
applications of non Uniform Sampling and processing
Topics in Current Chemistry, 2012Co-Authors: Sven G Hyberts, Haribabu Arthanari, Gerhard WagnerAbstract:Modern high-field NMR instruments provide unprecedented resolution. To make use of the resolving power in multidimensional NMR experiment standard linear Sampling through the indirect dimensions to the maximum optimal evolution times (~1.2T 2) is not practical because it would require extremely long measurement times. Thus, alternative Sampling methods have been proposed during the past 20 years. Originally, random nonlinear Sampling with an exponentially decreasing Sampling density was suggested, and data were transformed with a maximum entropy algorithm (Barna et al., J Magn Reson 73:69–77, 1987). Numerous other procedures have been proposed in the meantime. It has become obvious that the quality of spectra depends crucially on the Sampling schedules and the algorithms of data reconstruction. Here we use the forward maximum entropy (FM) reconstruction method to evaluate several alternate Sampling schedules. At the current stage, multidimensional NMR spectra that do not have a serious dynamic range problem, such as triple resonance experiments used for sequential assignments, are readily recorded and faithfully reconstructed using non-Uniform Sampling. Thus, these experiments can all be recorded non-Uniformly to utilize the power of modern instruments. On the other hand, for spectra with a large dynamic range, such as 3D and 4D NOESYs, choosing optimal Sampling schedules and the best reconstruction method is crucial if one wants to recover very weak peaks. Thus, this chapter is focused on selecting the best Sampling schedules and processing methods for high-dynamic range spectra.
-
accelerated acquisition of high resolution triple resonance spectra using non Uniform Sampling and maximum entropy reconstruction
Journal of Magnetic Resonance, 2004Co-Authors: David Rovnyak, Jeffrey C. Hoch, Dominique P Frueh, Mallika Sastry, Zhenyu J Sun, Alan S Stern, Gerhard WagnerAbstract:Abstract Non-Uniform Sampling is shown to provide significant time savings in the acquisition of a suite of three-dimensional NMR experiments utilized for obtaining backbone assignments of H, N, C ′ , CA, and CB nuclei in proteins : HNCO, HN(CA)CO, HNCA, HN(CO)CA, HNCACB, and HN(CO)CACB. Non-Uniform Sampling means that data were collected for only a subset of all incremented evolution periods, according to a user-specified Sampling schedule. When the suite of six 3D experiments was acquired in a Uniform fashion for an 11 kDa cytoplasmic domain of a membrane protein at 1.5 mM concentration, a total of 146 h was consumed. With non-Uniform Sampling, the same experiments were acquired in 32 h and, through subsequent maximum entropy reconstruction, yielded spectra of similar quality to those obtained by conventional Fourier transform of the Uniformly acquired data. The experimental time saved with this methodology can significantly accelerate protein structure determination by NMR, particularly when combined with the use of automated assignment software, and enable the study of samples with poor stability at room temperature. Since it is also possible to use the time savings to acquire a greater numbers of scans to increase sensitivity while maintaining high resolution, this methodology will help extend the size limit of proteins accessible to NMR studies, and open the way to studies of samples that suffer from solubility problems.
Mary Wootters - One of the best experts on this subject based on the ideXlab platform.
-
weighted matrix completion from non random non Uniform Sampling patterns
IEEE Transactions on Information Theory, 2021Co-Authors: Simon Foucart, Deanna Needell, Reese Pathak, Yaniv Plan, Mary WoottersAbstract:We study the matrix completion problem when the observation pattern is deterministic and possibly non-Uniform. We propose a simple and efficient debiased projection scheme for recovery from noisy observations and analyze the error under a suitable weighted metric. We introduce a simple function of the weight matrix and the Sampling pattern that governs the accuracy of the recovered matrix. We derive theoretical guarantees that upper bound the recovery error and nearly matching lower bounds that showcase optimality in several regimes. Our numerical experiments demonstrate the computational efficiency and accuracy of our approach, and show that debiasing is essential when using non-Uniform Sampling patterns.
Ananth Grama - One of the best experts on this subject based on the ideXlab platform.
-
distributed Uniform Sampling in unstructured peer to peer networks
Hawaii International Conference on System Sciences, 2006Co-Authors: Asad Awan, Ronaldo A. Ferreira, Suresh Jagannathan, Ananth GramaAbstract:Uniform Sampling in networks is at the core of a wide variety of randomized algorithms. Random Sampling can be performed by modeling the system as an undirected graph with associated transition probabilities and defining a corresponding Markov chain (MC). A random walk of prescribed minimum length, performed on this graph, yields a stationary distribution, and the corresponding random sample. This sample, however, is not Uniform when network nodes have a non-Uniform degree distribution. This poses a significant practical challenge since typical large scale real-world unstructured networks tend to have non-Uniform degree distributions, e.g., power-law degree distribution in unstructured peer-to-peer networks. In this paper, we present a distributed algorithm that enables efficient Uniform Sampling in large unstructured non-Uniform networks. Specifically, we prescribe necessary conditions for Uniform Sampling in such networks and present distributed algorithms that satisfy these requirements. We empirically evaluate the performance of our algorithm in comparison to known algorithms. The performance parameters include computational complexity, length of random walk, and Uniformity of the Sampling. Simulation results support our claims of performance improvements due to our algorithm.
-
HICSS - Distributed Uniform Sampling in Unstructured Peer-to-Peer Networks
Proceedings of the 39th Annual Hawaii International Conference on System Sciences (HICSS'06), 2006Co-Authors: Asad Awan, Ronaldo A. Ferreira, Suresh Jagannathan, Ananth GramaAbstract:Uniform Sampling in networks is at the core of a wide variety of randomized algorithms. Random Sampling can be performed by modeling the system as an undirected graph with associated transition probabilities and defining a corresponding Markov chain (MC). A random walk of prescribed minimum length, performed on this graph, yields a stationary distribution, and the corresponding random sample. This sample, however, is not Uniform when network nodes have a non-Uniform degree distribution. This poses a significant practical challenge since typical large scale real-world unstructured networks tend to have non-Uniform degree distributions, e.g., power-law degree distribution in unstructured peer-to-peer networks. In this paper, we present a distributed algorithm that enables efficient Uniform Sampling in large unstructured non-Uniform networks. Specifically, we prescribe necessary conditions for Uniform Sampling in such networks and present distributed algorithms that satisfy these requirements. We empirically evaluate the performance of our algorithm in comparison to known algorithms. The performance parameters include computational complexity, length of random walk, and Uniformity of the Sampling. Simulation results support our claims of performance improvements due to our algorithm.
-
Distributed Uniform Sampling in Real- World Networks
2004Co-Authors: Asad Awan, Ronaldo A. Ferreira, Suresh Jagannathan, Ananth GramaAbstract:Uniform Sampling in networks is at the core of a wide variety of randomized algorithms. Random Sampling can be peJjonned by modeling the system as a graph with associated transition probabilities and defining a corresponding Markov chain (Me). A random walk ofprescribed minimum length, pelfonned on this graph, yields a stationary dim'ibution, and the corresponding random sample. This sample, however, is not Uniform when network nodes have a nonUniform degree distribution. This poses a significant practical challenge since typical large scale, real-world, unstructured networks tend to have non-Uniform degree distributions, e.g.. power-law degree distribution iJl unstructured peer-to-peer networks. In this paper we present a distributed algorithm that enables efficient un(form Sampling in large real-world networks. Specifically, we prescribe necessary conditions for Uniform Sampling in such networks and present distributed algorithms that satisfy these requiremems. We empirically evaluate the peJjormance of our algorithm in comparison to known algorithms. We also quamijy. in context of the presented algorithms, the peJjormance parameters in Uniform Sampling that are 1110St relevant in a distributed setting computational complexit..-, number of network messages, and the Uniformity of the Sampling. Detailed experimental results are used to support our claims relating to pelformance improvements of our algorithm.
Mike Shuo-wei Chen - One of the best experts on this subject based on the ideXlab platform.
-
a non Uniform Sampling adc architecture with embedded alias free asynchronous filter
Global Communications Conference, 2012Co-Authors: Dylan Hand, Mike Shuo-wei ChenAbstract:This work proposes a non-Uniform Sampling analog-to-digital converter (ADC) architecture that embeds an alias-free filter in the asynchronous digital domain to relax the requirements of the analog anti-aliasing filter, improve the overall signal dynamic range, and interface directly with synchronous digital circuitry. Both event-driven voltage and time quantizers are used in the conversion process. Furthermore, an analytical model for estimating their quantization noise power is derived, which matches the numerical simulation with less than 4% deviation within the region of interest. A signal to noise ratio (SNR) improvement of 27dB over conventional, Uniformly sampled Nyquist ADCs is obtained given the same 10-bit quantizer.
-
GLOBECOM - A non-Uniform Sampling ADC architecture with embedded alias-free asynchronous filter
2012 IEEE Global Communications Conference (GLOBECOM), 2012Co-Authors: Dylan Hand, Mike Shuo-wei ChenAbstract:This work proposes a non-Uniform Sampling analog-to-digital converter (ADC) architecture that embeds an alias-free filter in the asynchronous digital domain to relax the requirements of the analog anti-aliasing filter, improve the overall signal dynamic range, and interface directly with synchronous digital circuitry. Both event-driven voltage and time quantizers are used in the conversion process. Furthermore, an analytical model for estimating their quantization noise power is derived, which matches the numerical simulation with less than 4% deviation within the region of interest. A signal to noise ratio (SNR) improvement of 27dB over conventional, Uniformly sampled Nyquist ADCs is obtained given the same 10-bit quantizer.
