The Experts below are selected from a list of 3366 Experts worldwide ranked by ideXlab platform
Shih-fu Chang - One of the best experts on this subject based on the ideXlab platform.
-
Unsupervised Embedding Learning via Invariant and Spreading Instance Feature.
arXiv: Computer Vision and Pattern Recognition, 2019Co-Authors: Xu Zhang, Pong C. Yuen, Shih-fu ChangAbstract:This paper studies the unsupervised embedding learning problem, which requires an effective similarity measurement between samples in low-dimensional embedding space. Motivated by the positive concentrated and negative separated properties observed from category-wise supervised learning, we propose to utilize the instance-wise supervision to approximate these properties, which aims at learning data augmentation invariant and instance spread-out features. To achieve this goal, we propose a novel instance based Softmax embedding method, which directly optimizes the `real' instance features on top of the Softmax Function. It achieves significantly faster learning speed and higher accuracy than all existing methods. The proposed method performs well for both seen and unseen testing categories with cosine similarity. It also achieves competitive performance even without pre-trained network over samples from fine-grained categories.
-
CVPR - Unsupervised Embedding Learning via Invariant and Spreading Instance Feature
2019 IEEE CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019Co-Authors: Xu Zhang, Pong C. Yuen, Shih-fu ChangAbstract:This paper studies the unsupervised embedding learning problem, which requires an effective similarity measurement between samples in low-dimensional embedding space. Motivated by the positive concentrated and negative separated properties observed from category-wise supervised learning, we propose to utilize the instance-wise supervision to approximate these properties, which aims at learning data augmentation invariant and instance spread-out features. To achieve this goal, we propose a novel instance based Softmax embedding method, which directly optimizes the `real' instance features on top of the Softmax Function. It achieves significantly faster learning speed and higher accuracy than all existing methods. The proposed method performs well for both seen and unseen testing categories with cosine similarity. It also achieves competitive performance even without pre-trained network over samples from fine-grained categories.
-
Heated-Up Softmax Embedding.
arXiv: Learning, 2018Co-Authors: Xu Zhang, Svebor Karaman, Wei Zhang, Shih-fu ChangAbstract:Metric learning aims at learning a distance which is consistent with the semantic meaning of the samples. The problem is generally solved by learning an embedding for each sample such that the embeddings of samples of the same category are compact while the embeddings of samples of different categories are spread-out in the feature space. We study the features extracted from the second last layer of a deep neural network based classifier trained with the cross entropy loss on top of the Softmax layer. We show that training classifiers with different temperature values of Softmax Function leads to features with different levels of compactness. Leveraging these insights, we propose a "heating-up" strategy to train a classifier with increasing temperatures, leading the corresponding embeddings to achieve state-of-the-art performance on a variety of metric learning benchmarks.
Xu Zhang - One of the best experts on this subject based on the ideXlab platform.
-
Unsupervised Embedding Learning via Invariant and Spreading Instance Feature.
arXiv: Computer Vision and Pattern Recognition, 2019Co-Authors: Xu Zhang, Pong C. Yuen, Shih-fu ChangAbstract:This paper studies the unsupervised embedding learning problem, which requires an effective similarity measurement between samples in low-dimensional embedding space. Motivated by the positive concentrated and negative separated properties observed from category-wise supervised learning, we propose to utilize the instance-wise supervision to approximate these properties, which aims at learning data augmentation invariant and instance spread-out features. To achieve this goal, we propose a novel instance based Softmax embedding method, which directly optimizes the `real' instance features on top of the Softmax Function. It achieves significantly faster learning speed and higher accuracy than all existing methods. The proposed method performs well for both seen and unseen testing categories with cosine similarity. It also achieves competitive performance even without pre-trained network over samples from fine-grained categories.
-
CVPR - Unsupervised Embedding Learning via Invariant and Spreading Instance Feature
2019 IEEE CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019Co-Authors: Xu Zhang, Pong C. Yuen, Shih-fu ChangAbstract:This paper studies the unsupervised embedding learning problem, which requires an effective similarity measurement between samples in low-dimensional embedding space. Motivated by the positive concentrated and negative separated properties observed from category-wise supervised learning, we propose to utilize the instance-wise supervision to approximate these properties, which aims at learning data augmentation invariant and instance spread-out features. To achieve this goal, we propose a novel instance based Softmax embedding method, which directly optimizes the `real' instance features on top of the Softmax Function. It achieves significantly faster learning speed and higher accuracy than all existing methods. The proposed method performs well for both seen and unseen testing categories with cosine similarity. It also achieves competitive performance even without pre-trained network over samples from fine-grained categories.
-
Heated-Up Softmax Embedding.
arXiv: Learning, 2018Co-Authors: Xu Zhang, Svebor Karaman, Wei Zhang, Shih-fu ChangAbstract:Metric learning aims at learning a distance which is consistent with the semantic meaning of the samples. The problem is generally solved by learning an embedding for each sample such that the embeddings of samples of the same category are compact while the embeddings of samples of different categories are spread-out in the feature space. We study the features extracted from the second last layer of a deep neural network based classifier trained with the cross entropy loss on top of the Softmax layer. We show that training classifiers with different temperature values of Softmax Function leads to features with different levels of compactness. Leveraging these insights, we propose a "heating-up" strategy to train a classifier with increasing temperatures, leading the corresponding embeddings to achieve state-of-the-art performance on a variety of metric learning benchmarks.
Joshua Zhexue Huang - One of the best experts on this subject based on the ideXlab platform.
-
Determining the optimal temperature parameter for Softmax Function in reinforcement learning
Applied Soft Computing, 2018Co-Authors: Xiaoliang Zhang, Joshua Zhexue HuangAbstract:Abstract The temperature parameter plays an important role in the action selection based on Softmax Function which is used to transform an original vector into a probability vector. An efficient method named Opti-Softmax to determine the optimal temperature parameter for Softmax Function in reinforcement learning is developed in this paper. Firstly, a new evaluation Function is designed to measure the effectiveness of temperature parameter by considering the information-loss of transformation and the diversity among probability vector elements. Secondly, an iterative updating rule is derived to determine the optimal temperature parameter by calculating the minimum of evaluation Function. Finally, the experimental results on the synthetic data and D -armed bandit problems demonstrate the feasibility and effectiveness of Opti-Softmax method.
Philip C. Woodland - One of the best experts on this subject based on the ideXlab platform.
-
ICASSP - Recurrent neural network language model training with noise contrastive estimation for speech recognition
2015 IEEE International Conference on Acoustics Speech and Signal Processing (ICASSP), 2015Co-Authors: Xie Chen, Xunying Liu, Mark J. F. Gales, Philip C. WoodlandAbstract:In recent years recurrent neural network language models (RNNLMs) have been successfully applied to a range of tasks including speech recognition. However, an important issue that limits the quantity of data used, and their possible application areas, is the computational cost in training. A signi??cant part of this cost is associated with the Softmax Function at the output layer, as this requires a normalization term to be explicitly calculated. This impacts both the training and testing speed, especially when a large output vocabulary is used. To address this problem, noise contrastive estimation (NCE) is explored in RNNLM training. NCE does not require the above normalization during both training and testing. It is insensitive to the output layer size. On a large vocabulary conversational telephone speech recognition task, a doubling in training speed on a GPU and a 56 times speed up in test time evaluation on a CPU were obtained.
Pong C. Yuen - One of the best experts on this subject based on the ideXlab platform.
-
Unsupervised Embedding Learning via Invariant and Spreading Instance Feature.
arXiv: Computer Vision and Pattern Recognition, 2019Co-Authors: Xu Zhang, Pong C. Yuen, Shih-fu ChangAbstract:This paper studies the unsupervised embedding learning problem, which requires an effective similarity measurement between samples in low-dimensional embedding space. Motivated by the positive concentrated and negative separated properties observed from category-wise supervised learning, we propose to utilize the instance-wise supervision to approximate these properties, which aims at learning data augmentation invariant and instance spread-out features. To achieve this goal, we propose a novel instance based Softmax embedding method, which directly optimizes the `real' instance features on top of the Softmax Function. It achieves significantly faster learning speed and higher accuracy than all existing methods. The proposed method performs well for both seen and unseen testing categories with cosine similarity. It also achieves competitive performance even without pre-trained network over samples from fine-grained categories.
-
CVPR - Unsupervised Embedding Learning via Invariant and Spreading Instance Feature
2019 IEEE CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019Co-Authors: Xu Zhang, Pong C. Yuen, Shih-fu ChangAbstract:This paper studies the unsupervised embedding learning problem, which requires an effective similarity measurement between samples in low-dimensional embedding space. Motivated by the positive concentrated and negative separated properties observed from category-wise supervised learning, we propose to utilize the instance-wise supervision to approximate these properties, which aims at learning data augmentation invariant and instance spread-out features. To achieve this goal, we propose a novel instance based Softmax embedding method, which directly optimizes the `real' instance features on top of the Softmax Function. It achieves significantly faster learning speed and higher accuracy than all existing methods. The proposed method performs well for both seen and unseen testing categories with cosine similarity. It also achieves competitive performance even without pre-trained network over samples from fine-grained categories.
