The Experts below are selected from a list of 17391 Experts worldwide ranked by ideXlab platform

Katsuyuki Hagiwara - One of the best experts on this subject based on the ideXlab platform.

  • on a fitting of a Heaviside function by deep relu neural networks
    International Conference on Neural Information Processing, 2018
    Co-Authors: Katsuyuki Hagiwara
    Abstract:

    A recent research interest on deep neural networks is to understand why deep networks are preferred to shallow networks. In this article, we considered an advantage of a deep structure in realizing a Heaviside function in training. This is significant not only as simple classification problems but also as a basis in constructing general non-smooth functions. A Heaviside function can be well approximated by a difference of ReLUs if we can set extremely large weight values. However, it is not so easy to attain them in training. We showed that a Heaviside function can be well represented without large weight values if we employ a deep structure. We also showed that update terms of weights at input side can be necessarily large if a network is trained to realize a Heaviside function. Therefore, apparent acceleration of training is brought about by setting a small learning rate. As a result, we can say that, by employing a deep structure, a good fitting of Heaviside function can be obtained within a reasonable training time under a moderate small learning rate. Our results suggest that a deep structure is effective in a practical training that requires a discontinuous output.

  • ICONIP (1) - On a Fitting of a Heaviside Function by Deep ReLU Neural Networks
    Neural Information Processing, 2018
    Co-Authors: Katsuyuki Hagiwara
    Abstract:

    A recent research interest on deep neural networks is to understand why deep networks are preferred to shallow networks. In this article, we considered an advantage of a deep structure in realizing a Heaviside function in training. This is significant not only as simple classification problems but also as a basis in constructing general non-smooth functions. A Heaviside function can be well approximated by a difference of ReLUs if we can set extremely large weight values. However, it is not so easy to attain them in training. We showed that a Heaviside function can be well represented without large weight values if we employ a deep structure. We also showed that update terms of weights at input side can be necessarily large if a network is trained to realize a Heaviside function. Therefore, apparent acceleration of training is brought about by setting a small learning rate. As a result, we can say that, by employing a deep structure, a good fitting of Heaviside function can be obtained within a reasonable training time under a moderate small learning rate. Our results suggest that a deep structure is effective in a practical training that requires a discontinuous output.

Shinji Nishiwaki - One of the best experts on this subject based on the ideXlab platform.

  • Heaviside projection based topology optimization by a pde filtered scalar function
    Structural and Multidisciplinary Optimization, 2011
    Co-Authors: Atsushi Kawamoto, Tsuyoshi Nomura, Shintaro Yamasaki, Tadayoshi Matsumori, Tsuguo Kondoh, Shinji Nishiwaki
    Abstract:

    This paper deals with topology optimization based on the Heaviside projection method using a scalar function as design variables. The scalar function is then regularized by a PDE based filter. Several image-processing based filtering techniques have so far been proposed for regularization or restricting the minimum length scale. They are conventionally applied to the design sensitivities rather than the design variables themselves. However, it causes discrepancies between the filtered sensitivities and the actual sensitivities that may confuse the optimization process and disturb the convergence. In this paper, we propose a Heaviside projection based topology optimization method with a scalar function that is filtered by a Helmholtz type partial differential equation. Therefore, the optimality can be strictly discussed in terms of the KKT condition. In order to demonstrate the effectiveness of the proposed method, a minimum compliance problem is solved.

A.m. Davis - One of the best experts on this subject based on the ideXlab platform.

  • the Heaviside theory of lumped circuits and differential systems based an causality
    International Symposium on Circuits and Systems, 1994
    Co-Authors: A.m. Davis
    Abstract:

    Due to an accident of personality and history, the Laplace transform superseded the Heaviside theory of circuits and systems. The main reason was the latter's lack of rigor-even though it is more general and directly applicable. This paper shows that the Heaviside theory can be made entirely rigorous in an elementary fashion. Furthermore, it exhibits all of the calculational machinery of the Laplace transform. The Laplace transform is shown to fit into the Heaviside theory as a special tool, primarily useful in developing the idea of a spectrum. >

  • A unified theory of lumped circuits and differential systems based on Heaviside operators and causality
    IEEE Transactions on Circuits and Systems I: Fundamental Theory and Applications, 1994
    Co-Authors: A.m. Davis
    Abstract:

    The paper demonstrates that a coherent and logical foundation for circuits and systems can, in an entirely elementary manner, be based upon the Heaviside operational calculus and causality. It is commonly believed that the operational calculus is both difficult and nonrigorous; in fact, it is neither. Further, it is more general than the Laplace transform in that it involves integration over only a finite interval and thus introduces no convergence questions as does the latter. Since it analyzes circuits and systems directly in the time domain, the Heaviside method is more intuitive and direct to apply. Furthermore, it provides a theme, a motif, linking all of the major concepts of circuits and systems. It is argued here that circuit analysis and system theory are currently taught as disparate disciplines, both being presented as a collection of isolated topics. The Heaviside theory, on the other hand, permits the two to be taught in an integrated fashion-with circuits providing concrete examples and system theory the abstract and general mathematical methodology. >

  • ISCAS - The Heaviside theory of lumped circuits and differential systems based an causality
    Proceedings of IEEE International Symposium on Circuits and Systems - ISCAS '94, 1
    Co-Authors: A.m. Davis
    Abstract:

    Due to an accident of personality and history, the Laplace transform superseded the Heaviside theory of circuits and systems. The main reason was the latter's lack of rigor-even though it is more general and directly applicable. This paper shows that the Heaviside theory can be made entirely rigorous in an elementary fashion. Furthermore, it exhibits all of the calculational machinery of the Laplace transform. The Laplace transform is shown to fit into the Heaviside theory as a special tool, primarily useful in developing the idea of a spectrum. >

Shu-cherng Fang - One of the best experts on this subject based on the ideXlab platform.

  • Best L 1 approximation of Heaviside-type functions from Chebyshev and weak-Chebyshev spaces
    Numerical Algorithms, 2016
    Co-Authors: Laurent Gajny, Eric Nyiri, Olivier Gibaru, Shu-cherng Fang
    Abstract:

    In this article, we study the problem of best L 1 approximation of Heaviside-type functions from Chebyshev and weak-Chebyshev spaces. We extend the Hobby-Rice theorem (Proc. Am. Math. Soc., 16, 665–670, 1965) into an appropriate framework and prove the unicity of best L 1 approximation of Heaviside-type functions from an even-dimensional Chebyshev space under some assumptions on the dimension of the subspaces composed of the odd and even functions. We also apply the results to compute best L 1 approximations of Heaviside-type functions by polynomials and Hermite polynomial splines with fixed knots.

  • Best $L_1$ approximation of Heaviside-type functions in Chebyshev and weak-Chebyshev spaces
    arXiv: Functional Analysis, 2014
    Co-Authors: Laurent Gajny, Eric Nyiri, Olivier Gibaru, Shu-cherng Fang
    Abstract:

    In this article, we study the problem of best $L_1$ approximation of Heaviside-type functions in Chebyshev and weak-Chebyshev spaces. We extend the Hobby-Rice theorem into an appropriate framework and prove the unicity of best $L_1$ approximation of Heaviside-type functions in an even-dimensional Chebyshev space under the condition that the dimension of the subspace composed of the even functions is half the dimension of the whole space. We also apply the results to compute best $L_1$ approximations of Heaviside-type functions by polynomials and Hermite polynomial splines with fixed knots.

Qiaoheting - One of the best experts on this subject based on the ideXlab platform.