The Experts below are selected from a list of 27291 Experts worldwide ranked by ideXlab platform

Kazuyuki Hara - One of the best experts on this subject based on the ideXlab platform.

  • ICANN - Improving the Convergence Property of Soft Committee Machines by Replacing Derivative with Truncated Gaussian Function
    Artificial Neural Networks and Machine Learning – ICANN 2014, 2014
    Co-Authors: Kazuyuki Hara, Kentaro Katahira
    Abstract:

    In online gradient descent learning, the local property of the Derivative of the output function can cause slow convergence. This phenomenon, called a plateau, occurs in the learning process of a multilayer network. Improving the Derivative Term, we propose a simple method replacing the Derivative Term with a truncated Gaussian function that greatly increases the convergence speed. We then analyze a soft committee machine trained by proposed method, and show how proposed method breaks a plateau. Results showed that the proposed method eventually led to break the symmetry between hidden units.

  • soft committee machine using simple Derivative Term
    International Conference on Artificial Intelligence and Soft Computing, 2014
    Co-Authors: Kazuyuki Hara, Kentaro Katahira
    Abstract:

    In on-line gradient descent learning, the local property of the Derivative of the output function can cause slow convergence. This phenomenon, called a plateau, occurs in the learning process of the multilayer network. Improving the Derivative Term, we employ the proposed method replacing the Derivative Term with a constant that greatly increases the relaxation speed. Moreover, we replace the Derivative Term with the 2nd order of expansion of the Derivative, and it beaks a plateau faster than the original method.

  • ICAISC (1) - Soft Committee Machine Using Simple Derivative Term
    Artificial Intelligence and Soft Computing, 2014
    Co-Authors: Kazuyuki Hara, Kentaro Katahira
    Abstract:

    In on-line gradient descent learning, the local property of the Derivative of the output function can cause slow convergence. This phenomenon, called a plateau, occurs in the learning process of the multilayer network. Improving the Derivative Term, we employ the proposed method replacing the Derivative Term with a constant that greatly increases the relaxation speed. Moreover, we replace the Derivative Term with the 2nd order of expansion of the Derivative, and it beaks a plateau faster than the original method.

  • theoretical analysis of function of Derivative Term in on line gradient descent learning
    International Conference on Artificial Neural Networks, 2012
    Co-Authors: Kazuyuki Hara, Kentaro Katahira, Kazuo Okanoya, Masato Okada
    Abstract:

    In on-line gradient descent learning, the local property of the Derivative Term of the output can slow convergence. Improving the Derivative Term, such as by using the natural gradient, has been proposed for speeding up the convergence. Beside this sophisticated method, "simple method" that replace the Derivative Term with a constant has proposed and showed that this greatly increases convergence speed. Although this phenomenon has been analyzed empirically, however, theoretical analysis is required to show its generality. In this paper, we theoretically analyze the effect of using the simple method. Our results show that, with the simple method, the generalization error decreases faster than with the true gradient descent method when the learning step is smaller than optimum value ηopt. When it is larger than ηopt, it decreases slower with the simple method, and the residual error is larger than with the true gradient descent method. Moreover, when there is output noise, ηopt is no longer optimum; thus, the simple method is not robust in noisy circumstances.

  • ICANN (2) - Theoretical analysis of function of Derivative Term in on-line gradient descent learning
    Artificial Neural Networks and Machine Learning – ICANN 2012, 2012
    Co-Authors: Kazuyuki Hara, Kentaro Katahira, Kazuo Okanoya, Masato Okada
    Abstract:

    In on-line gradient descent learning, the local property of the Derivative Term of the output can slow convergence. Improving the Derivative Term, such as by using the natural gradient, has been proposed for speeding up the convergence. Beside this sophisticated method, "simple method" that replace the Derivative Term with a constant has proposed and showed that this greatly increases convergence speed. Although this phenomenon has been analyzed empirically, however, theoretical analysis is required to show its generality. In this paper, we theoretically analyze the effect of using the simple method. Our results show that, with the simple method, the generalization error decreases faster than with the true gradient descent method when the learning step is smaller than optimum value ηopt. When it is larger than ηopt, it decreases slower with the simple method, and the residual error is larger than with the true gradient descent method. Moreover, when there is output noise, ηopt is no longer optimum; thus, the simple method is not robust in noisy circumstances.

Kentaro Katahira - One of the best experts on this subject based on the ideXlab platform.

  • ICANN - Improving the Convergence Property of Soft Committee Machines by Replacing Derivative with Truncated Gaussian Function
    Artificial Neural Networks and Machine Learning – ICANN 2014, 2014
    Co-Authors: Kazuyuki Hara, Kentaro Katahira
    Abstract:

    In online gradient descent learning, the local property of the Derivative of the output function can cause slow convergence. This phenomenon, called a plateau, occurs in the learning process of a multilayer network. Improving the Derivative Term, we propose a simple method replacing the Derivative Term with a truncated Gaussian function that greatly increases the convergence speed. We then analyze a soft committee machine trained by proposed method, and show how proposed method breaks a plateau. Results showed that the proposed method eventually led to break the symmetry between hidden units.

  • soft committee machine using simple Derivative Term
    International Conference on Artificial Intelligence and Soft Computing, 2014
    Co-Authors: Kazuyuki Hara, Kentaro Katahira
    Abstract:

    In on-line gradient descent learning, the local property of the Derivative of the output function can cause slow convergence. This phenomenon, called a plateau, occurs in the learning process of the multilayer network. Improving the Derivative Term, we employ the proposed method replacing the Derivative Term with a constant that greatly increases the relaxation speed. Moreover, we replace the Derivative Term with the 2nd order of expansion of the Derivative, and it beaks a plateau faster than the original method.

  • ICAISC (1) - Soft Committee Machine Using Simple Derivative Term
    Artificial Intelligence and Soft Computing, 2014
    Co-Authors: Kazuyuki Hara, Kentaro Katahira
    Abstract:

    In on-line gradient descent learning, the local property of the Derivative of the output function can cause slow convergence. This phenomenon, called a plateau, occurs in the learning process of the multilayer network. Improving the Derivative Term, we employ the proposed method replacing the Derivative Term with a constant that greatly increases the relaxation speed. Moreover, we replace the Derivative Term with the 2nd order of expansion of the Derivative, and it beaks a plateau faster than the original method.

  • theoretical analysis of function of Derivative Term in on line gradient descent learning
    International Conference on Artificial Neural Networks, 2012
    Co-Authors: Kazuyuki Hara, Kentaro Katahira, Kazuo Okanoya, Masato Okada
    Abstract:

    In on-line gradient descent learning, the local property of the Derivative Term of the output can slow convergence. Improving the Derivative Term, such as by using the natural gradient, has been proposed for speeding up the convergence. Beside this sophisticated method, "simple method" that replace the Derivative Term with a constant has proposed and showed that this greatly increases convergence speed. Although this phenomenon has been analyzed empirically, however, theoretical analysis is required to show its generality. In this paper, we theoretically analyze the effect of using the simple method. Our results show that, with the simple method, the generalization error decreases faster than with the true gradient descent method when the learning step is smaller than optimum value ηopt. When it is larger than ηopt, it decreases slower with the simple method, and the residual error is larger than with the true gradient descent method. Moreover, when there is output noise, ηopt is no longer optimum; thus, the simple method is not robust in noisy circumstances.

  • ICANN (2) - Theoretical analysis of function of Derivative Term in on-line gradient descent learning
    Artificial Neural Networks and Machine Learning – ICANN 2012, 2012
    Co-Authors: Kazuyuki Hara, Kentaro Katahira, Kazuo Okanoya, Masato Okada
    Abstract:

    In on-line gradient descent learning, the local property of the Derivative Term of the output can slow convergence. Improving the Derivative Term, such as by using the natural gradient, has been proposed for speeding up the convergence. Beside this sophisticated method, "simple method" that replace the Derivative Term with a constant has proposed and showed that this greatly increases convergence speed. Although this phenomenon has been analyzed empirically, however, theoretical analysis is required to show its generality. In this paper, we theoretically analyze the effect of using the simple method. Our results show that, with the simple method, the generalization error decreases faster than with the true gradient descent method when the learning step is smaller than optimum value ηopt. When it is larger than ηopt, it decreases slower with the simple method, and the residual error is larger than with the true gradient descent method. Moreover, when there is output noise, ηopt is no longer optimum; thus, the simple method is not robust in noisy circumstances.

Muneto Nitta - One of the best experts on this subject based on the ideXlab platform.

  • spatially modulated vacua in a lorentz invariant scalar field theory
    European Physical Journal C, 2018
    Co-Authors: Muneto Nitta, Shin Sasaki, Ryo Yokokura
    Abstract:

    Spatial modulation has been studied for a long time in condensed matter, nuclear matter and quark matter, where the manifest Lorentz invariance is lost due to the finite density/temperature effects and so on. In this paper, spatially modulated vacua at zero temperature and zero density are studied in Lorentz invariant field theories. We first propose an adaptation of the Nambu–Goldstone theorem to higher Derivative theories under the assumption of the absence of ghosts: when a global symmetry is spontaneously broken due to vacuum expectation values of space-time Derivatives of fields, a Nambu–Goldstone (NG) boson appears without a canonical kinetic (quadratic Derivative) Term with a quartic Derivative Term in the modulated direction while a Higgs boson appears with a canonical kinetic Term. We demonstrate this in a simple model allowing (meta)stable modulated vacuum of a phase modulation (Fulde–Ferrell state), where an NG mode associated with spontaneously broken translational and U(1) symmetries appears.

  • Spatially Modulated Vacua in Relativistic Field Theories
    arXiv: High Energy Physics - Theory, 2017
    Co-Authors: Muneto Nitta, Shin Sasaki, Ryo Yokokura
    Abstract:

    Spatial modulation has been studied for a long time in condensed matter, nuclear matter and quark matter, so far in non-relativistic field theories. In this paper, spatially modulated vacua at zero temperature and zero density are studied in relativistic field theories. We first propose a generalization of the Nambu-Goldstone theorem under the assumption of the absence of ghosts: when a global symmetry is spontaneously broken due to vacuum expectation values of space-time Derivatives of fields, a Nambu-Goldstone (NG) boson appears without a canonical kinetic (quadratic Derivative) Term with a quartic Derivative Term in the modulated direction while a Higgs boson appears as a gapless mode. We demonstrate this in a simple model allowing (meta)stable modulated vacuum of a phase modulation (Fulde-Ferrell state), where an NG mode associated with spontaneously broken translational and U(1) symmetries appears.

  • Black hole Skyrmion in a generalized Skyrme model
    Journal of High Energy Physics, 2016
    Co-Authors: Sven Bjarke Gudnason, Muneto Nitta, Nobuyuki Sawado
    Abstract:

    We study a Skyrme-like model with the Skyrme Term and a sixth-order Derivative Term as higher-order Terms, coupled to gravity and we construct Schwarzschild black hole Skyrme hair. We find, surprisingly, that the sixth-order Derivative Term alone cannot stabilize the black hole hair solutions; the Skyrme Term with a large enough coefficient is a necessity.

  • Baryonic sphere : A spherical domain wall carrying baryon number
    Physical Review D, 2014
    Co-Authors: Sven Bjarke Gudnason, Muneto Nitta
    Abstract:

    We construct a spherical domain wall which has baryon charge distributed on a sphere of finite radius in a Skyrme model with a sixth-order Derivative Term and a modified mass Term. Its distribution of energy density likewise takes the form of a sphere. In order to localize the domain wall at a finite radius we need a negative coefficient in front of the Skyrme Term and a positive coefficient of the sixth order Derivative Term to stabilize the soliton. Increasing the pion mass pronounces the shell-like structure of the configuration.

  • Gauge symmetry breaking in ten-dimensional Yang-Mills theory dynamically compactified on S 6
    Physical Review D, 2010
    Co-Authors: Pravabati Chingangbam, Hironobu Kihara, Muneto Nitta
    Abstract:

    We study fluctuation modes in ten-dimensional Yang-Mills theory with a higher Derivative Term for the gauge field. We consider the ten-dimensional space-time to be a product of a four-dimensional space-time and six-dimensional sphere which exhibits dynamical compactification. Because of the isometry on S^6, there are flat directions corresponding to the Nambu-Goldstone zero modes in the effective theory on the solution. The zero modes are absorbed into gauge fields and form massive vector fields as a consequence of the Higgs-Kibble mechanism. The mass of the vector fields is proportional to the inverse of the radius of the sphere and larger than the mass scale set by the radius because of the higher Derivative Term.

Masato Okada - One of the best experts on this subject based on the ideXlab platform.

  • theoretical analysis of function of Derivative Term in on line gradient descent learning
    International Conference on Artificial Neural Networks, 2012
    Co-Authors: Kazuyuki Hara, Kentaro Katahira, Kazuo Okanoya, Masato Okada
    Abstract:

    In on-line gradient descent learning, the local property of the Derivative Term of the output can slow convergence. Improving the Derivative Term, such as by using the natural gradient, has been proposed for speeding up the convergence. Beside this sophisticated method, "simple method" that replace the Derivative Term with a constant has proposed and showed that this greatly increases convergence speed. Although this phenomenon has been analyzed empirically, however, theoretical analysis is required to show its generality. In this paper, we theoretically analyze the effect of using the simple method. Our results show that, with the simple method, the generalization error decreases faster than with the true gradient descent method when the learning step is smaller than optimum value ηopt. When it is larger than ηopt, it decreases slower with the simple method, and the residual error is larger than with the true gradient descent method. Moreover, when there is output noise, ηopt is no longer optimum; thus, the simple method is not robust in noisy circumstances.

  • ICANN (2) - Theoretical analysis of function of Derivative Term in on-line gradient descent learning
    Artificial Neural Networks and Machine Learning – ICANN 2012, 2012
    Co-Authors: Kazuyuki Hara, Kentaro Katahira, Kazuo Okanoya, Masato Okada
    Abstract:

    In on-line gradient descent learning, the local property of the Derivative Term of the output can slow convergence. Improving the Derivative Term, such as by using the natural gradient, has been proposed for speeding up the convergence. Beside this sophisticated method, "simple method" that replace the Derivative Term with a constant has proposed and showed that this greatly increases convergence speed. Although this phenomenon has been analyzed empirically, however, theoretical analysis is required to show its generality. In this paper, we theoretically analyze the effect of using the simple method. Our results show that, with the simple method, the generalization error decreases faster than with the true gradient descent method when the learning step is smaller than optimum value ηopt. When it is larger than ηopt, it decreases slower with the simple method, and the residual error is larger than with the true gradient descent method. Moreover, when there is output noise, ηopt is no longer optimum; thus, the simple method is not robust in noisy circumstances.

Shingo Tanaka - One of the best experts on this subject based on the ideXlab platform.

  • HOPF SOLITON SOLUTIONS FROM LOW ENERGY EFFECTIVE ACTION OF SU(2) YANG–MILLS THEORY
    Modern Physics Letters A, 2006
    Co-Authors: Nobuyuki Sawado, N. Shiiki, Shingo Tanaka
    Abstract:

    The Skyrme–Faddeev–Niemi (SFN) model which is an O(3) σ-model in three-dimensional space up to fourth-order in the first Derivative is regarded as a low-energy effective theory of SU(2) Yang–Mills theory. One can show from the Wilsonian renormalization group argument that the effective action of Yang–Mills theory recovers the SFN in the infrared region. However, the theory contains another fourth-order Term which destabilizes soliton solutions. We find the stable soliton solutions in this extended action, introducing a second Derivative Term as a stabilizer. A perturbative technique for the second Derivative Term is applied to exclude (or reduce) the ill behavior of the action. A new topological energy bound formula is inferred for the action.

  • Soliton solutions in an effective action for SU(2) Yang—Mills theory: including effects of higher-Derivative Term
    Czechoslovak Journal of Physics, 2005
    Co-Authors: N. Sawado, N. Shiiki, Shingo Tanaka
    Abstract:

    The Skyrme-Faddeev-Niemi (SFN) model, which is an O(3) σ model in three dimensional space, upto fourth-order in the first Derivative is regarded as a low-energy effective theory of SU(2) Yang-Mills theory. One can show from the Wilsonian renormalization group argument that the effective action of Yang-Mills theory recovers the SFN in the infrared region. However, the theory contains an additional fourth-order Term which destabilizes the soliton solution. In this paper, we derive the second-Derivative Term perturbatively and show that the SFN model with the second-Derivative Term possesses soliton solutions.