The Experts below are selected from a list of 179154 Experts worldwide ranked by ideXlab platform
Lixin Ji - One of the best experts on this subject based on the ideXlab platform.
-
natural logarithm rectified activation function in Convolutional Neural Networks
IEEE International Conference Computer and Communications, 2019Co-Authors: Jianpeng Zhang, Jinghua Qu, Lixin JiAbstract:Activation functions play a key role in providing remarkable performance in deep Neural Networks, and the rectified linear unit (ReLU) is one of the most widely used activation functions. Various new activation functions and improvements on ReLU have been proposed, but each carry performance drawbacks. In this paper, we propose an improved activation function, which we name the natural-logarithm-rectified linear unit (NLReLU). NLReLU uses the parametric natural logarithmic transform to improve ReLU, and is defined as $f(x)=\ln (\beta \cdot \max (0,x)+1.0)$. NLReLU not only retains the sparse activation characteristic of ReLU, but it also alleviates the “dying $\mathrm {R}\mathrm {e}\mathrm {L}\mathrm {U}$” and vanishing gradient problems to some extent. It also reduces the bias shift effect and heteroscedasticity of neuron data distributions among network layers in order to accelerate the learning process. The proposed method was verified across ten Convolutional Neural Networks with different depths for two essential datasets. Experiments illustrate that Convolutional Neural Networks with NLReLU exhibit higher accuracy than those with ReLU, and that NLReLU is comparable to other well-known activation functions. NLReLU provides 0.16% and 2.04% higher classification accuracy on average compared to ReLU when used in shallow Convolutional Neural Networks with the MNIST and CIFAR-10 datasets, respectively. The average accuracy of deep Convolutional Neural Networks with NLReLU is 1.35% higher on average with the CIFAR-10 dataset.
-
natural logarithm rectified activation function in Convolutional Neural Networks
arXiv: Learning, 2019Co-Authors: Jianpeng Zhang, Jinghua Qu, Lixin JiAbstract:Activation functions play a key role in providing remarkable performance in deep Neural Networks, and the rectified linear unit (ReLU) is one of the most widely used activation functions. Various new activation functions and improvements on ReLU have been proposed, but each carry performance drawbacks. In this paper, we propose an improved activation function, which we name the natural-logarithm-rectified linear unit (NLReLU). This activation function uses the parametric natural logarithmic transform to improve ReLU and is simply defined as. NLReLU not only retains the sparse activation characteristic of ReLU, but it also alleviates the "dying ReLU" and vanishing gradient problems to some extent. It also reduces the bias shift effect and heteroscedasticity of neuron data distributions among network layers in order to accelerate the learning process. The proposed method was verified across ten Convolutional Neural Networks with different depths for two essential datasets. Experiments illustrate that Convolutional Neural Networks with NLReLU exhibit higher accuracy than those with ReLU, and that NLReLU is comparable to other well-known activation functions. NLReLU provides 0.16% and 2.04% higher classification accuracy on average compared to ReLU when used in shallow Convolutional Neural Networks with the MNIST and CIFAR-10 datasets, respectively. The average accuracy of deep Convolutional Neural Networks with NLReLU is 1.35% higher on average with the CIFAR-10 dataset.
-
Natural-Logarithm-Rectified Activation Function in Convolutional Neural Networks
2019 IEEE 5th International Conference on Computer and Communications (ICCC), 2019Co-Authors: Jianpeng Zhang, Jinghua Qu, Lixin JiAbstract:Activation functions playa key role in providing remarkable performance in deep Neural Networks, and the rectified linear unit (ReLU) is one of the most widely used activation functions. Various new activation functions and improvements on ReLU have been proposed, but each carry performance drawbacks. In this paper, we propose an improved activation function, which we name the naturalIngarithnι-rectified linear unit (NLReLU). NLReLU uses the parametric natural logarithmic transform to improve ReLU, and is defined as f(x) = In(β max(0,x) + 1.0). NLReLU not only retains the sparse activation characteristic of ReLU, but it also alleviates the 'd'ing ReLU" and vanishing gradient problems to some extent. It also reduces the bias shift effect and heteroscedasticity of neuron data distributions among network layers in order to accelerate the learning process. The proposed method was verified across ten Convolutional Neural Networks with different depths for two essential datasets. Experiments illustrate that Convolutional Neural Networks with NLReLU exhibit higher accuracy than those with ReLU, and that NLReLU is comparable to other well-known activation functions. NLReLU provides 0.16% and 2.04% higher classification accuracy on average compared to ReLU when used in shallow Convolutional Neural Networks with the MNIST and CIFAR-10 datasets, respectively. The average accuracy of deep Convolutional Neural Networks with NLReLU is 1.35% higher oon average with the CIFAR-10 dataset.
Yaron Lipman - One of the best experts on this subject based on the ideXlab platform.
-
Point Convolutional Neural Networks by Extension Operators
ACM Transactions on Graphics, 2018Co-Authors: Matan Atzmon, Haggai Maron, Yaron LipmanAbstract:This paper presents Point Convolutional Neural Networks (PCNN): a novel framework for applying Convolutional Neural Networks to point clouds.
-
Point Convolutional Neural Networks by extension operators
ACM Transactions on Graphics, 2018Co-Authors: Matan Atzmon, Haggai Maron, Yaron LipmanAbstract:This paper presents Point Convolutional Neural Networks (PCNN): a novel framework for applying Convolutional Neural Networks to point clouds. The framework consists of two operators: extension and restriction, mapping point cloud functions to volumetric functions and vise-versa. A point cloud convolution is defined by pull-back of the Euclidean volumetric convolution via an extension-restriction mechanism. The point cloud convolution is computationally efficient, invariant to the order of points in the point cloud, robust to different samplings and varying densities, and translation invariant, that is the same convolution kernel is used at all points. PCNN generalizes image CNNs and allows readily adapting their architectures to the point cloud setting. Evaluation of PCNN on three central point cloud learning benchmarks convincingly outperform competing point cloud learning methods, and the vast majority of methods working with more informative shape representations such as surfaces and/or normals.
Jianpeng Zhang - One of the best experts on this subject based on the ideXlab platform.
-
natural logarithm rectified activation function in Convolutional Neural Networks
IEEE International Conference Computer and Communications, 2019Co-Authors: Jianpeng Zhang, Jinghua Qu, Lixin JiAbstract:Activation functions play a key role in providing remarkable performance in deep Neural Networks, and the rectified linear unit (ReLU) is one of the most widely used activation functions. Various new activation functions and improvements on ReLU have been proposed, but each carry performance drawbacks. In this paper, we propose an improved activation function, which we name the natural-logarithm-rectified linear unit (NLReLU). NLReLU uses the parametric natural logarithmic transform to improve ReLU, and is defined as $f(x)=\ln (\beta \cdot \max (0,x)+1.0)$. NLReLU not only retains the sparse activation characteristic of ReLU, but it also alleviates the “dying $\mathrm {R}\mathrm {e}\mathrm {L}\mathrm {U}$” and vanishing gradient problems to some extent. It also reduces the bias shift effect and heteroscedasticity of neuron data distributions among network layers in order to accelerate the learning process. The proposed method was verified across ten Convolutional Neural Networks with different depths for two essential datasets. Experiments illustrate that Convolutional Neural Networks with NLReLU exhibit higher accuracy than those with ReLU, and that NLReLU is comparable to other well-known activation functions. NLReLU provides 0.16% and 2.04% higher classification accuracy on average compared to ReLU when used in shallow Convolutional Neural Networks with the MNIST and CIFAR-10 datasets, respectively. The average accuracy of deep Convolutional Neural Networks with NLReLU is 1.35% higher on average with the CIFAR-10 dataset.
-
natural logarithm rectified activation function in Convolutional Neural Networks
arXiv: Learning, 2019Co-Authors: Jianpeng Zhang, Jinghua Qu, Lixin JiAbstract:Activation functions play a key role in providing remarkable performance in deep Neural Networks, and the rectified linear unit (ReLU) is one of the most widely used activation functions. Various new activation functions and improvements on ReLU have been proposed, but each carry performance drawbacks. In this paper, we propose an improved activation function, which we name the natural-logarithm-rectified linear unit (NLReLU). This activation function uses the parametric natural logarithmic transform to improve ReLU and is simply defined as. NLReLU not only retains the sparse activation characteristic of ReLU, but it also alleviates the "dying ReLU" and vanishing gradient problems to some extent. It also reduces the bias shift effect and heteroscedasticity of neuron data distributions among network layers in order to accelerate the learning process. The proposed method was verified across ten Convolutional Neural Networks with different depths for two essential datasets. Experiments illustrate that Convolutional Neural Networks with NLReLU exhibit higher accuracy than those with ReLU, and that NLReLU is comparable to other well-known activation functions. NLReLU provides 0.16% and 2.04% higher classification accuracy on average compared to ReLU when used in shallow Convolutional Neural Networks with the MNIST and CIFAR-10 datasets, respectively. The average accuracy of deep Convolutional Neural Networks with NLReLU is 1.35% higher on average with the CIFAR-10 dataset.
-
Natural-Logarithm-Rectified Activation Function in Convolutional Neural Networks
2019 IEEE 5th International Conference on Computer and Communications (ICCC), 2019Co-Authors: Jianpeng Zhang, Jinghua Qu, Lixin JiAbstract:Activation functions playa key role in providing remarkable performance in deep Neural Networks, and the rectified linear unit (ReLU) is one of the most widely used activation functions. Various new activation functions and improvements on ReLU have been proposed, but each carry performance drawbacks. In this paper, we propose an improved activation function, which we name the naturalIngarithnι-rectified linear unit (NLReLU). NLReLU uses the parametric natural logarithmic transform to improve ReLU, and is defined as f(x) = In(β max(0,x) + 1.0). NLReLU not only retains the sparse activation characteristic of ReLU, but it also alleviates the 'd'ing ReLU" and vanishing gradient problems to some extent. It also reduces the bias shift effect and heteroscedasticity of neuron data distributions among network layers in order to accelerate the learning process. The proposed method was verified across ten Convolutional Neural Networks with different depths for two essential datasets. Experiments illustrate that Convolutional Neural Networks with NLReLU exhibit higher accuracy than those with ReLU, and that NLReLU is comparable to other well-known activation functions. NLReLU provides 0.16% and 2.04% higher classification accuracy on average compared to ReLU when used in shallow Convolutional Neural Networks with the MNIST and CIFAR-10 datasets, respectively. The average accuracy of deep Convolutional Neural Networks with NLReLU is 1.35% higher oon average with the CIFAR-10 dataset.
Matthias Bethge - One of the best experts on this subject based on the ideXlab platform.
-
image style transfer using Convolutional Neural Networks
Computer Vision and Pattern Recognition, 2016Co-Authors: Leon A. Gatys, Alexander S. Ecker, Matthias BethgeAbstract:Rendering the semantic content of an image in different styles is a difficult image processing task. Arguably, a major limiting factor for previous approaches has been the lack of image representations that explicitly represent semantic information and, thus, allow to separate image content from style. Here we use image representations derived from Convolutional Neural Networks optimised for object recognition, which make high level image information explicit. We introduce A Neural Algorithm of Artistic Style that can separate and recombine the image content and style of natural images. The algorithm allows us to produce new images of high perceptual quality that combine the content of an arbitrary photograph with the appearance of numerous wellknown artworks. Our results provide new insights into the deep image representations learned by Convolutional Neural Networks and demonstrate their potential for high level image synthesis and manipulation.
-
Image Style Transfer Using Convolutional Neural Networks
IEEE Conference on Computer Vision and Pattern Recognition, 2015Co-Authors: Leon A. Gatys, Alexander S. Ecker, Matthias BethgeAbstract:Here we introduce a new model of natural textures based on the feature spaces of Convolutional Neural Networks optimised for object recognition. Samples from the model are of high perceptual quality demonstrating the generative power of Neural Networks trained in a purely discriminative fashion. Within the model, textures are represented by the correlations between feature maps in several layers of the network. We show that across layers the texture representations increasingly capture the statistical properties of natural images while making object information more and more explicit. The model provides a new tool to generate stimuli for neuroscience and might offer insights into the deep representations learned by Convolutional Neural Networks.
Georges Linarès - One of the best experts on this subject based on the ideXlab platform.
-
Quaternion Convolutional Neural Networks for heterogeneous image processing
2019Co-Authors: Titouan Parcollet, Mohamed Morchid, Georges LinarèsAbstract:Convolutional Neural Networks (CNN) have recently achieved state-of-the-art results in various applications. In the case of image recognition, an ideal model has to learn independently of the training data, both local dependencies between the three components (R,G,B) of a pixel, and the global relations describing edges or shapes, making it efficient with small or heterogeneous datasets. Quaternion-valued convo-lutional Neural Networks (QCNN) solved this problematic by introducing multidimensional algebra to CNN. This paper proposes to explore the fundamental reason of the success of QCNN over CNN, by investigating the impact of the Hamilton product on a color image reconstruction task performed from a gray-scale only training. By learning independently both internal and external relations and with less parameters than real valued Convolutional encoder-decoder (CAE), quaternion Convolutional encoder-decoders (QCAE) perfectly reconstructed unseen color images while CAE produced worst and gray-scale versions. Index Terms-Quaternion Convolutional encoder-decoder, Convolutional Neural Networks, heterogeneous image processing
-
Quaternion Convolutional Neural Networks for Heterogeneous Image Processing
ICASSP 2019 - 2019 IEEE International Conference on Acoustics Speech and Signal Processing (ICASSP), 2019Co-Authors: Titouan Parcollet, Mohamed Morchid, Georges LinarèsAbstract:Convolutional Neural Networks (CNN) have recently achieved state-of-the-art results in various applications. In the case of image recognition, an ideal model has to learn independently of the training data, both local dependencies between the three components (R,G,B) of a pixel, and the global relations describing edges or shapes, making it efficient with small or heterogeneous datasets. Quaternion-valued Convolutional Neural Networks (QCNN) solved this problematic by introducing multidimensional algebra to CNN. This paper proposes to explore the fundamental reason of the success of QCNN over CNN, by investigating the impact of the Hamilton product on a color image reconstruction task performed from a gray-scale only training. By learning independently both internal and external relations and with less parameters than real valued Convolutional encoder-decoder (CAE), quaternion Convolutional encoder-decoders (QCAE) perfectly reconstructed unseen color images while CAE produced worst and gray-scale versions.