The Experts below are selected from a list of 1182 Experts worldwide ranked by ideXlab platform
Stefanie Jegelka - One of the best experts on this subject based on the ideXlab platform.
-
resnet with one neuron hidden layers is a universal approximator
arXiv: Learning, 2018Co-Authors: Hongzhou Lin, Stefanie JegelkaAbstract:We demonstrate that a very deep ResNet with stacked modules with one neuron per hidden layer and ReLU activation Functions can uniformly approximate any Lebesgue Integrable Function in $d$ dimensions, i.e. $\ell_1(\mathbb{R}^d)$. Because of the identity mapping inherent to ResNets, our network has alternating layers of dimension one and $d$. This stands in sharp contrast to fully connected networks, which are not universal approximators if their width is the input dimension $d$ [Lu et al, 2017; Hanin and Sellke, 2017]. Hence, our result implies an increase in representational power for narrow deep networks by the ResNet architecture.
-
resnet with one neuron hidden layers is a universal approximator
Neural Information Processing Systems, 2018Co-Authors: Hongzhou Lin, Stefanie JegelkaAbstract:We demonstrate that a very deep ResNet with stacked modules that have one neuron per hidden layer and ReLU activation Functions can uniformly approximate any Lebesgue Integrable Function in d dimensions, i.e. \ell_1(R^d). Due to the identity mapping inherent to ResNets, our network has alternating layers of dimension one and d. This stands in sharp contrast to fully connected networks, which are not universal approximators if their width is the input dimension d [21,11]. Hence, our result implies an increase in representational power for narrow deep networks by the ResNet architecture.
Hongzhou Lin - One of the best experts on this subject based on the ideXlab platform.
-
resnet with one neuron hidden layers is a universal approximator
arXiv: Learning, 2018Co-Authors: Hongzhou Lin, Stefanie JegelkaAbstract:We demonstrate that a very deep ResNet with stacked modules with one neuron per hidden layer and ReLU activation Functions can uniformly approximate any Lebesgue Integrable Function in $d$ dimensions, i.e. $\ell_1(\mathbb{R}^d)$. Because of the identity mapping inherent to ResNets, our network has alternating layers of dimension one and $d$. This stands in sharp contrast to fully connected networks, which are not universal approximators if their width is the input dimension $d$ [Lu et al, 2017; Hanin and Sellke, 2017]. Hence, our result implies an increase in representational power for narrow deep networks by the ResNet architecture.
-
resnet with one neuron hidden layers is a universal approximator
Neural Information Processing Systems, 2018Co-Authors: Hongzhou Lin, Stefanie JegelkaAbstract:We demonstrate that a very deep ResNet with stacked modules that have one neuron per hidden layer and ReLU activation Functions can uniformly approximate any Lebesgue Integrable Function in d dimensions, i.e. \ell_1(R^d). Due to the identity mapping inherent to ResNets, our network has alternating layers of dimension one and d. This stands in sharp contrast to fully connected networks, which are not universal approximators if their width is the input dimension d [21,11]. Hence, our result implies an increase in representational power for narrow deep networks by the ResNet architecture.
Serra-capizzano Stefano - One of the best experts on this subject based on the ideXlab platform.
-
Multilevel symmetrized Toeplitz structures and spectral distribution results for the related matrix-sequences
2020Co-Authors: Ferrari Paola, Furci Isabella, Serra-capizzano StefanoAbstract:In recent years, motivated by computational purposes, the singular value and spectral features of the symmetrization of Toeplitz matrices generated by a Lebesgue Integrable Function have been studied. Indeed, under the assumptions that $f$ belongs to $L^1([-\pi,\pi])$ and it has real Fourier coefficients, the spectral and singular value distribution of the matrix-sequence $\{Y_nT_n[f]\}_n$ has been identified, where $n$ is the matrix-size, $Y_n$ is the anti-identity matrix, and $T_n[f]$ is the Toeplitz matrix generated by $f$. In this note, we consider the multilevel Toeplitz matrix $T_{\bf n}[f]$ generated by $f\in L^1([-\pi,\pi]^k)$, $\bf n$ being a multi-index identifying the matrix-size, and we prove spectral and singular value distribution results for the matrix-sequence $\{Y_{\bf n}T_{\bf n}[f]\}_{\bf n}$ with $Y_{\bf n}$ being the corresponding tensorization of the anti-identity matrix
-
The eigenvalue distribution of special $2$-by-$2$ block matrix sequences, with applications to the case of symmetrized Toeplitz structures
2018Co-Authors: Ferrari Paola, Furci Isabella, Hon Sean, Mursaleen, Mohammad Ayman, Serra-capizzano StefanoAbstract:Given a Lebesgue Integrable Function $f$ over $[0,2\pi]$, we consider the sequence of matrices $\{Y_nT_n[f]\}_n$, where $T_n[f]$ is the $n$-by-$n$ Toeplitz matrix generated by $f$ and $Y_n$ is the flip permutation matrix, also called the anti-identity matrix. Because of the unitary character of $Y_n$, the singular values of $T_n[f]$ and $Y_n T_n[f]$ coincide. However, the eigenvalues are affected substantially by the action of the matrix $Y_n$. Under the assumption that the Fourier coefficients are real, we prove that $\{Y_nT_n[f]\}_n$ is distributed in the eigenvalue sense as \[ \phi_g(\theta)=\left\{ \begin{array}{cc} g(\theta), & \theta\in [0,2\pi], -g(-\theta), & \theta\in [-2\pi,0), \end{array} \right.\, \] with $g(\theta)=|f(\theta)|$. We also consider the preconditioning introduced by Pestana and Wathen and, by using the same arguments, we prove that the preconditioned sequence is distributed in the eigenvalue sense as $\phi_1$, under the mild assumption that $f$ is sparsely vanishing. We emphasize that the mathematical tools introduced in this setting have a general character and in fact can be potentially used in different contexts. A number of numerical experiments are provided and critically discussed
Ferrari Paola - One of the best experts on this subject based on the ideXlab platform.
-
Multilevel symmetrized Toeplitz structures and spectral distribution results for the related matrix-sequences
2020Co-Authors: Ferrari Paola, Furci Isabella, Serra-capizzano StefanoAbstract:In recent years, motivated by computational purposes, the singular value and spectral features of the symmetrization of Toeplitz matrices generated by a Lebesgue Integrable Function have been studied. Indeed, under the assumptions that $f$ belongs to $L^1([-\pi,\pi])$ and it has real Fourier coefficients, the spectral and singular value distribution of the matrix-sequence $\{Y_nT_n[f]\}_n$ has been identified, where $n$ is the matrix-size, $Y_n$ is the anti-identity matrix, and $T_n[f]$ is the Toeplitz matrix generated by $f$. In this note, we consider the multilevel Toeplitz matrix $T_{\bf n}[f]$ generated by $f\in L^1([-\pi,\pi]^k)$, $\bf n$ being a multi-index identifying the matrix-size, and we prove spectral and singular value distribution results for the matrix-sequence $\{Y_{\bf n}T_{\bf n}[f]\}_{\bf n}$ with $Y_{\bf n}$ being the corresponding tensorization of the anti-identity matrix
-
The eigenvalue distribution of special $2$-by-$2$ block matrix sequences, with applications to the case of symmetrized Toeplitz structures
2018Co-Authors: Ferrari Paola, Furci Isabella, Hon Sean, Mursaleen, Mohammad Ayman, Serra-capizzano StefanoAbstract:Given a Lebesgue Integrable Function $f$ over $[0,2\pi]$, we consider the sequence of matrices $\{Y_nT_n[f]\}_n$, where $T_n[f]$ is the $n$-by-$n$ Toeplitz matrix generated by $f$ and $Y_n$ is the flip permutation matrix, also called the anti-identity matrix. Because of the unitary character of $Y_n$, the singular values of $T_n[f]$ and $Y_n T_n[f]$ coincide. However, the eigenvalues are affected substantially by the action of the matrix $Y_n$. Under the assumption that the Fourier coefficients are real, we prove that $\{Y_nT_n[f]\}_n$ is distributed in the eigenvalue sense as \[ \phi_g(\theta)=\left\{ \begin{array}{cc} g(\theta), & \theta\in [0,2\pi], -g(-\theta), & \theta\in [-2\pi,0), \end{array} \right.\, \] with $g(\theta)=|f(\theta)|$. We also consider the preconditioning introduced by Pestana and Wathen and, by using the same arguments, we prove that the preconditioned sequence is distributed in the eigenvalue sense as $\phi_1$, under the mild assumption that $f$ is sparsely vanishing. We emphasize that the mathematical tools introduced in this setting have a general character and in fact can be potentially used in different contexts. A number of numerical experiments are provided and critically discussed
Florin Popovici - One of the best experts on this subject based on the ideXlab platform.
-
THE ASYMPTOTIC BEHAVIOR OF Integrable FunctionS
2016Co-Authors: Constantin P. Niculescu, Florin PopoviciAbstract:Abstract. Given a density d de\u85ned on the Borel subsets of [0;1); the limit in density of a Function f: [0;1) ! R is zero (abbreviated, (d)-limx!1 f(x) = 0) if there exists a set S of zero density such that f(x) ! 0 as x runs to 1 outside S. It is proved that the behavior at in\u85nity of every Lebesgue Integrable Function f 2 L1(0;1) satis\u85es the relations (d(n))-limx!1 Qn k=0 ln (k) x f(x) = 0, where (d(n))n is a scale of densities includ-ing the usual one, d(0)(A) = limr!1 m(A\[0;r))r: The analogy between convergent series and integrals over the positive semi-axis is an interesting topic from classical real analysis that ows continuously from the old days of mathematics to contemporary research. However, there is a fundamental property of convergent series in regard to which this analogy fails. Precisely, if
-
THE BEHAVIOR AT INFINITY OF AN Integrable Function
2016Co-Authors: Constantin P. Niculescu, Florin PopoviciAbstract:Abstract. Given a density d de\u85ned on the Borel subsets of [0;1); the limit at in\u85nity in density of a Function f: [0;1) ! R is zero if each of the sets ft: jf(t)j "g has zero density whenever "> 0: It is proved that every Lebesgue Integrable Function f: [0;1) ! R veri\u85es this type of be-havior at in\u85nity with respect to a scale of densities including the usual one, d(A) = limr!1 m(A\[0;r))r: The analogy between convergent series and integrals over the positive semi-axis was an elegant and fruitful subject present in all major treatises of mathematical analysis published during the 20th Century. As was noted by G. H. Hardy in his Course of Pure Mathematics [4], p. 324, there is one fundamental property of a convergent in\u85nite series in regard to which the analogy between in\u85nite series and in nite integrals breaks down. If P an is convergent then an! 0; but it is not always true, even when f: [0;1) ! R is positive, that if R1 0 f(x)dx is convergent then f(x) ! 0 as x!1. Due to the prominent role played by negligible sets one might expect that a conclusion of the type f(x) ! 0 as x runs to 1 outside a negligible set must be working. That this is not the case is shown by the Integrable Function f(x) = 1X n=1 [n;n+2n)(x); x 2 [0;1): Surprisingly, the analogy can be re-established if the usual limit is replaced by limit in density. This fact is implicit in a famous paper by B. O. Koopman and J. von Neumann [6] dedicated to weakly mixing transformations, and was recently made explicit and extended by us [10] to a scale of densities measuring how thin are the various Borel subsets of R. The aim of the present note is to provide a short argument for this general result along Koopman-von Neumanns ideas. In what follows we shall adopt the convention used in dynamical system theory for the iterates of a Function f = f(x); f (0)(x) = x and f (n)(x) = (f f f | {z}) n times (x) for n 1: Note that f (n)(x) does not mean the nth derivative of f(x); a Function that we never use in this paper