The Experts below are selected from a list of 81960 Experts worldwide ranked by ideXlab platform
Dacheng Tao - One of the best experts on this subject based on the ideXlab platform.
-
perceptual adversarial networks for Image to Image Transformation
IEEE Transactions on Image Processing, 2018Co-Authors: Chaoyue Wang, Chaohui Wang, Dacheng TaoAbstract:In this paper, we propose Perceptual Adversarial Networks (PAN) for Image-to-Image Transformations. Different from existing application driven algorithms, PAN provides a generic framework of learning to map from input Images to desired Images (Fig. 1), such as a rainy Image to its de-rained counterpart, object edges to photos, semantic labels to a scenes Image, etc. The proposed PAN consists of two feed-forward convolutional neural networks (CNNs): the Image Transformation network T and the discriminative network D. Besides the generative adversarial loss widely used in GANs, we propose the perceptual adversarial loss, which undergoes an adversarial training process between the Image Transformation network T and the hidden layers of the discriminative network D. The hidden layers and the output of the discriminative network D are upgraded to constantly and automatically discover the discrepancy between the transformed Image and the corresponding ground-truth, while the Image Transformation network T is trained to minimize the discrepancy explored by the discriminative network D. Through integrating the generative adversarial loss and the perceptual adversarial loss, D and T can be trained alternately to solve Image-to-Image Transformation tasks. Experiments evaluated on several Image-to-Image Transformation tasks (e.g., Image de-raining, Image inpainting, etc) demonstrate the effectiveness of the proposed PAN and its advantages over many existing works.
-
perceptual adversarial networks for Image to Image Transformation
arXiv: Computer Vision and Pattern Recognition, 2017Co-Authors: Chaoyue Wang, Chaohui Wang, Dacheng TaoAbstract:In this paper, we propose a principled Perceptual Adversarial Networks (PAN) for Image-to-Image Transformation tasks. Unlike existing application-specific algorithms, PAN provides a generic framework of learning mapping relationship between paired Images (Fig. 1), such as mapping a rainy Image to its de-rained counterpart, object edges to its photo, semantic labels to a scenes Image, etc. The proposed PAN consists of two feed-forward convolutional neural networks (CNNs), the Image Transformation network T and the discriminative network D. Through combining the generative adversarial loss and the proposed perceptual adversarial loss, these two networks can be trained alternately to solve Image-to-Image Transformation tasks. Among them, the hidden layers and output of the discriminative network D are upgraded to continually and automatically discover the discrepancy between the transformed Image and the corresponding ground-truth. Simultaneously, the Image Transformation network T is trained to minimize the discrepancy explored by the discriminative network D. Through the adversarial training process, the Image Transformation network T will continually narrow the gap between transformed Images and ground-truth Images. Experiments evaluated on several Image-to-Image Transformation tasks (e.g., Image de-raining, Image inpainting, etc.) show that the proposed PAN outperforms many related state-of-the-art methods.
-
heterogeneous Image Transformation
Pattern Recognition Letters, 2013Co-Authors: Nannan Wang, Dacheng Tao, Xinbo GaoAbstract:Heterogeneous Image Transformation (HIT) plays an important role in both law enforcements and digital entertainment. Some available popular Transformation methods, like locally linear embedding based, usually generate Images with lower definition and blurred details mainly due to two defects: (1) these approaches use a fixed number of nearest neighbors (NN) to model the Transformation process, i.e., K-NN-based methods; (2) with overlapping areas averaged, the transformed Image is approximately equivalent to be filtered by a low pass filter, which filters the high frequency or detail information. These drawbacks reduce the visual quality and the recognition rate across heterogeneous Images. In order to overcome these two disadvantages, a two step framework is constructed based on sparse feature selection (SFS) and support vector regression (SVR). In the proposed model, SFS selects nearest neighbors adaptively based on sparse representation to implement an initial Transformation, and subsequently the SVR model is applied to estimate the lost high frequency information or detail information. Finally, by linear superimposing these two parts, the ultimate transformed Image is obtained. Extensive experiments on both sketch-photo database and near infrared-visible Image database illustrates the effectiveness of the proposed heterogeneous Image Transformation method.
Lihi Zelnikmanor - One of the best experts on this subject based on the ideXlab platform.
-
the contextual loss for Image Transformation with non aligned data
European Conference on Computer Vision, 2018Co-Authors: Roey Mechrez, Itamar Talmi, Lihi ZelnikmanorAbstract:Feed-forward CNNs trained for Image Transformation problems rely on loss functions that measure the similarity between the generated Image and a target Image. Most of the common loss functions assume that these Images are spatially aligned and compare pixels at corresponding locations. However, for many tasks, aligned training pairs of Images will not be available. We present an alternative loss function that does not require alignment, thus providing an effective and simple solution for a new space of problems. Our loss is based on both context and semantics – it compares regions with similar semantic meaning, while considering the context of the entire Image. Hence, for example, when transferring the style of one face to another, it will translate eyes-to-eyes and mouth-to-mouth. Our code can be found at https://www.github.com/roimehrez/contextualLoss.
-
the contextual loss for Image Transformation with non aligned data
arXiv: Computer Vision and Pattern Recognition, 2018Co-Authors: Roey Mechrez, Itamar Talmi, Lihi ZelnikmanorAbstract:Feed-forward CNNs trained for Image Transformation problems rely on loss functions that measure the similarity between the generated Image and a target Image. Most of the common loss functions assume that these Images are spatially aligned and compare pixels at corresponding locations. However, for many tasks, aligned training pairs of Images will not be available. We present an alternative loss function that does not require alignment, thus providing an effective and simple solution for a new space of problems. Our loss is based on both context and semantics -- it compares regions with similar semantic meaning, while considering the context of the entire Image. Hence, for example, when transferring the style of one face to another, it will translate eyes-to-eyes and mouth-to-mouth.
-
the contextual loss for Image Transformation with non aligned data
arXiv: Computer Vision and Pattern Recognition, 2018Co-Authors: Roey Mechrez, Itamar Talmi, Lihi ZelnikmanorAbstract:Feed-forward CNNs trained for Image Transformation problems rely on loss functions that measure the similarity between the generated Image and a target Image. Most of the common loss functions assume that these Images are spatially aligned and compare pixels at corresponding locations. However, for many tasks, aligned training pairs of Images will not be available. We present an alternative loss function that does not require alignment, thus providing an effective and simple solution for a new space of problems. Our loss is based on both context and semantics -- it compares regions with similar semantic meaning, while considering the context of the entire Image. Hence, for example, when transferring the style of one face to another, it will translate eyes-to-eyes and mouth-to-mouth. Our code can be found at this https URL
Li Feifei - One of the best experts on this subject based on the ideXlab platform.
-
perceptual losses for real time style transfer and super resolution
European Conference on Computer Vision, 2016Co-Authors: Justin Johnson, Alexandre Alahi, Li FeifeiAbstract:We consider Image Transformation problems, where an input Image is transformed into an output Image. Recent methods for such problems typically train feed-forward convolutional neural networks using a per-pixel loss between the output and ground-truth Images. Parallel work has shown that high-quality Images can be generated by defining and optimizing perceptual loss functions based on high-level features extracted from pretrained networks. We combine the benefits of both approaches, and propose the use of perceptual loss functions for training feed-forward networks for Image Transformation tasks. We show results on Image style transfer, where a feed-forward network is trained to solve the optimization problem proposed by Gatys et al. in real-time. Compared to the optimization-based method, our network gives similar qualitative results but is three orders of magnitude faster. We also experiment with single-Image super-resolution, where replacing a per-pixel loss with a perceptual loss gives visually pleasing results.
Justin Johnson - One of the best experts on this subject based on the ideXlab platform.
-
perceptual losses for real time style transfer and super resolution
European Conference on Computer Vision, 2016Co-Authors: Justin Johnson, Alexandre Alahi, Li FeifeiAbstract:We consider Image Transformation problems, where an input Image is transformed into an output Image. Recent methods for such problems typically train feed-forward convolutional neural networks using a per-pixel loss between the output and ground-truth Images. Parallel work has shown that high-quality Images can be generated by defining and optimizing perceptual loss functions based on high-level features extracted from pretrained networks. We combine the benefits of both approaches, and propose the use of perceptual loss functions for training feed-forward networks for Image Transformation tasks. We show results on Image style transfer, where a feed-forward network is trained to solve the optimization problem proposed by Gatys et al. in real-time. Compared to the optimization-based method, our network gives similar qualitative results but is three orders of magnitude faster. We also experiment with single-Image super-resolution, where replacing a per-pixel loss with a perceptual loss gives visually pleasing results.
-
Perceptual losses for real-time style transfer and super-resolution
Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2016Co-Authors: Justin Johnson, Alexandre Alahi, Li Fei-feiAbstract:We consider Image Transformation problems, where an input Image is transformed into an output Image. Recent methods for such problems typically train feed-forward convolutional neural networks using a \emph{per-pixel} loss between the output and ground-truth Images. Parallel work has shown that high-quality Images can be generated by defining and optimizing \emph{perceptual} loss functions based on high-level features extracted from pretrained networks. We combine the benefits of both approaches, and propose the use of perceptual loss functions for training feed-forward networks for Image Transformation tasks. We show results on Image style transfer, where a feed-forward network is trained to solve the optimization problem proposed by Gatys et al in real-time. Compared to the optimization-based method, our network gives similar qualitative results but is three orders of magnitude faster. We also experiment with single-Image super-resolution, where replacing a per-pixel loss with a perceptual loss gives visually pleasing results.
Qionghai Dai - One of the best experts on this subject based on the ideXlab platform.
-
unsupervised content preserving Transformation for optical microscopy
Light-Science & Applications, 2021Co-Authors: Guoxun Zhang, Hui Qiao, Feng Bao, Yue Deng, Jingping Yun, Xing Lin, Hao Xie, Haoqian Wang, Qionghai DaiAbstract:The development of deep learning and open access to a substantial collection of imaging data together provide a potential solution for computational Image Transformation, which is gradually changing the landscape of optical imaging and biomedical research. However, current implementations of deep learning usually operate in a supervised manner, and their reliance on laborious and error-prone data annotation procedures remains a barrier to more general applicability. Here, we propose an unsupervised Image Transformation to facilitate the utilization of deep learning for optical microscopy, even in some cases in which supervised models cannot be applied. Through the introduction of a saliency constraint, the unsupervised model, named Unsupervised content-preserving Transformation for Optical Microscopy (UTOM), can learn the mapping between two Image domains without requiring paired training data while avoiding distortions of the Image content. UTOM shows promising performance in a wide range of biomedical Image Transformation tasks, including in silico histological staining, fluorescence Image restoration, and virtual fluorescence labeling. Quantitative evaluations reveal that UTOM achieves stable and high-fidelity Image Transformations across different imaging conditions and modalities. We anticipate that our framework will encourage a paradigm shift in training neural networks and enable more applications of artificial intelligence in biomedical imaging.
-
unsupervised content preserving Transformation for optical microscopy
bioRxiv, 2019Co-Authors: Guoxun Zhang, Hui Qiao, Xing Lin, Hao Xie, Haoqian Wang, Qionghai DaiAbstract:The advent of deep learning and the open access to a substantial collection of imaging data provide a potential solution to computational Image Transformation, which is gradually changing the landscape of optical imaging and biomedical research. However, current deep-learning implementations usually operate in a supervised manner, and the reliance on a laborious and error-prone data annotation procedure remains a barrier towards more general applicability. Here, we propose an unsupervised Image Transformation enlightened by cycle-consistent generative adversarial networks (cycleGANs) to facilitate the utilization of deep learning in optical microscopy. By incorporating the saliency constraint into cycleGAN, the unsupervised approach, dubbed as content-preserving cycleGAN (c2GAN), can learn the mapping between two Image domains and avoid the misalignment of salient objects without paired training data. We demonstrate several Image Transformation tasks such as fluorescence Image restoration, whole-slide histological coloration, and virtual fluorescent labeling. Quantitative evaluations prove that c2GAN achieves robust and high-fidelity Image Transformation across different imaging modalities and various data configurations. We anticipate that our framework will encourage a paradigm shift in training neural networks and democratize deep learning algorithms for optical society.
