The Experts below are selected from a list of 21099 Experts worldwide ranked by ideXlab platform
Tatseng Chua - One of the best experts on this subject based on the ideXlab platform.
-
semantic gap oriented active learning for multilabel Image Annotation
IEEE Transactions on Image Processing, 2012Co-Authors: Jinhui Tang, Dacheng Tao, Zhengjun Zha, Tatseng ChuaAbstract:User interaction is an effective way to handle the semantic gap problem in Image Annotation. To minimize user effort in the interactions, many active learning methods were proposed. These methods treat the semantic concepts individually or correlatively. However, they still neglect the key motivation of user feedback: to tackle the semantic gap. The size of the semantic gap of each concept is an important factor that affects the performance of user feedback. User should pay more efforts to the concepts with large semantic gaps, and vice versa. In this paper, we propose a semantic-gap-oriented active learning method, which incorporates the semantic gap measure into the information-minimization-based sample selection strategy. The basic learning model used in the active learning framework is an extended multilabel version of the sparse-graph-based semisupervised learning method that incorporates the semantic correlation. Extensive experiments conducted on two benchmark Image data sets demonstrated the importance of bringing the semantic gap measure into the active learning process.
-
semantic context modeling with maximal margin conditional random fields for automatic Image Annotation
Computer Vision and Pattern Recognition, 2010Co-Authors: Yu Xiang, Xiangdong Zhou, Zuotao Liu, Tatseng Chua, Chong Wah NgoAbstract:Context modeling for Vision Recognition and Automatic Image Annotation (AIA) has attracted increasing attentions in recent years. For various contextual information and resources, semantic context has been exploited in AIA and brings promising results. However, previous works either casted the problem into structural classification or adopted multi-layer modeling, which suffer from the problems of scalability or model efficiency. In this paper, we propose a novel discriminative Conditional Random Field (CRF) model for semantic context modeling in AIA, which is built over semantic concepts and treats an Image as a whole observation without segmentation. Our model captures the interactions between semantic concepts from both semantic level and visual level in an integrated manner. Specifically, we employ graph structure to model contextual relationships between semantic concepts. The potential functions are designed based on linear discriminative models, which enables us to propose a novel decoupled hinge loss function for maximal margin parameter estimation. We train the model by solving a set of independent quadratic programming problems with our derived contextual kernel. The experiments are conducted on commonly used benchmarks: Corel and TRECVID data sets for evaluation. The experimental results show that compared with the state-of-the-art methods, our method achieves significant improvement on Annotation performance.
-
a revisit of generative model for automatic Image Annotation using markov random fields
Computer Vision and Pattern Recognition, 2009Co-Authors: Yu Xiang, Xiangdong Zhou, Tatseng Chua, Chong Wah NgoAbstract:Much research effort on automatic Image Annotation (AIA) has been focused on generative model, due to its well formed theory and competitive performance as compared with many well designed and sophisticated methods. However, when considering semantic context for Annotation, the model suffers from the weak learning ability. This is mainly due to the lack of parameter setting and appropriate learning strategy for characterizing the semantic context in the traditional generative model. In this paper, we present a new approach based on multiple Markov random fields (MRF) for semantic context modeling and learning. Differing from previous MRF related AIA approach; we explore the optimal parameter estimation and model inference systematically to leverage the learning power of traditional generative model. Specifically, we propose new potential function for site modeling based on generative model and build local graphs for each Annotation keyword. The parameter estimation and model inference is performed in local optimal sense. We conduct experiments on commonly used benchmarks. On Corel 5000 Images, we achieved 0.36 and 0.31 in recall and precision respectively on 263 keywords. This is a very significant improvement over the best reported result of the current state-of-the-art approaches.
-
automatic Image Annotation via local multi label classification
Conference on Image and Video Retrieval, 2008Co-Authors: Mei Wang, Xiangdong Zhou, Tatseng ChuaAbstract:As the consequence of semantic gap, visual similarity does not guarantee semantic similarity, which in general is conflicting with the inherent assumption of many generative-based Image Annotation methods. While discriminative learning approach had often been used to classify Images into different semantic classes, its efficiency is often impaired by the problems of multi-labeling and large scale concept space typically encountered in practical Image Annotation tasks. In this paper, we explore solutions to the problems of large scale concept space learning and mismatch between semantic and visual space. To tackle the first problem, we explore the use of higher level semantic space with lower dimension by clustering correlated keywords into topics in the local neighborhood. The topics are used as lexis for assigning multiple labels for unlabeled Images. To tackle the problem of semantic gap, we aim to reduce the bias between visual and semantic spaces by finding optimal margins in both spaces. In particular, we propose an iterative solution by alternately maximizing the sum of the margins to reduce the gap between visual similarity and semantic similarity. The experimental results on the ECCV2002 benchmark show that our method outperforms the state-of-the-art generative-based Annotation method MBRM and discriminative-based ASVM-MIL by 9% and 11% in terms of F1 measure respectively.
-
enhancing Image Annotation by integrating concept ontology and text based bayesian learning model
ACM Multimedia, 2007Co-Authors: Rui Shi, Chinhui Lee, Tatseng ChuaAbstract:Automatic Image Annotation (AIA) has been a hot research topic in recent years since it can be used to support concept-based Image retrieval. However, most existing AIA models depend heavily on the availability of a large number of labeled training samples, which require significant human labeling efforts. In this paper, we propose a novel learning framework which integrates text-based Bayesian model (TBM) and concept ontology to effectively expand the training set of each concept class without the need of additional human labeling efforts or collecting additional training Images from other data sources. The basic idea lies in exploiting the text information from training set to provide additional effective Annotations for training Images so that training data for each concept class can be augmented. In this study we employ Bayesian Hierarchical Multinomial Mixture Models (BHMMMs) as our baseline AIA model. By combining additional Annotations obtained from TBM into each concept class in the training phase, the performance of BHMMMs can be significantly improved on Corel Image dataset with 263 testing concepts as compared to the state-of-the-art AIA models under the same experimental configurations.
Songhe Feng - One of the best experts on this subject based on the ideXlab platform.
-
transductive multi instance multi label learning algorithm with application to automatic Image Annotation
Expert Systems With Applications, 2010Co-Authors: Songhe FengAbstract:Automatic Image Annotation has emerged as an important research topic due to its potential application on both Image understanding and web Image search. Due to the inherent ambiguity of Image-label mapping and the scarcity of training examples, the Annotation task has become a challenge to systematically develop robust Annotation models with better performance. From the perspective of machine learning, the Annotation task fits both multi-instance and multi-label learning framework due to the fact that an Image is usually described by multiple semantic labels (keywords) and these labels are often highly related to respective regions rather than the entire Image. In this paper, we propose an improved Transductive Multi-Instance Multi-Label (TMIML) learning framework, which aims at taking full advantage of both labeled and unlabeled data to address the Annotation problem. The experiments over the well known Corel 5000 data set demonstrate that the proposed method is beneficial in the Image Annotation task and outperforms most existing Image Annotation algorithms.
Bo Wang - One of the best experts on this subject based on the ideXlab platform.
-
multi instance multi label learning combining hierarchical context and its application to Image Annotation
IEEE Transactions on Multimedia, 2016Co-Authors: Xinmiao Ding, Weihua Xiong, Wen Guo, Bo WangAbstract:In Image Annotation, one Image is often modeled as a bag of regions (“instances”) associated with multiple labels, which is a typical application of multi-instance multi-label learning (MIML). Although lots of research has shown that the interplay embedded among instances and labels can largely boost the Image Annotation accuracy, most existing MIML methods consider none or partial context cues. In this paper, we propose a novel context-aware MIML model to integrate the instance context and label context into a general framework. Specially, the instance context is constructed with multiple graphs, while the label context is built up through a linear combination of several common latent conceptions that link low level features and high level semantic labels. Comparison with other leading methods on several benchmark datasets in terms of Image Annotation shows that our proposed method can get better performance than the state-of-the-art approaches.
Lei Zhang - One of the best experts on this subject based on the ideXlab platform.
-
multi label sparse coding for automatic Image Annotation
Computer Vision and Pattern Recognition, 2009Co-Authors: Changhu Wang, Lei Zhang, Shuicheng Yan, Hongjiang ZhangAbstract:In this paper, we present a multi-label sparse coding framework for feature extraction and classification within the context of automatic Image Annotation. First, each Image is encoded into a so-called supervector, derived from the universal Gaussian Mixture Models on orderless Image patches. Then, a label sparse coding based subspace learning algorithm is derived to effectively harness multi-label information for dimensionality reduction. Finally, the sparse coding method for multi-label data is proposed to propagate the multi-labels of the training Images to the query Image with the sparse l1 reconstruction coefficients. Extensive Image Annotation experiments on the Corel5k and Corel30k databases both show the superior performance of the proposed multi-label sparse coding framework over the state-of-the-art algorithms.
-
Image Annotation by large scale content based Image retrieval
ACM Multimedia, 2006Co-Authors: Xirong Li, Le Chen, Lei ZhangAbstract:Image Annotation has been an active research topic in recent years due to its potentially large impact on both Image understanding and Web Image search. In this paper, we target at solving the automatic Image Annotation problem in a novel search and mining framework. Given an uncaptioned Image, first in the search stage, we perform content-based Image retrieval (CBIR) facilitated by high-dimensional indexing to find a set of visually similar Images from a large-scale Image database. The database consists of Images crawled from the World Wide Web with rich Annotations, e.g. titles and surrounding text. Then in the mining stage, a search result clustering technique is utilized to find most representative keywords from the Annotations of the retrieved Image subset. These keywords, after salience ranking, are finally used to annotate the uncaptioned Image. Based on search technologies, this framework does not impose an explicit training stage, but efficiently leverages large-scale and well-annotated Images, and is potentially capable of dealing with unlimited vocabulary. Based on 2.4 million real Web Images, comprehensive evaluation of Image Annotation on Corel and U. Washington Image databases show the effectiveness and efficiency of the proposed approach.
Hongjiang Zhang - One of the best experts on this subject based on the ideXlab platform.
-
multi label sparse coding for automatic Image Annotation
Computer Vision and Pattern Recognition, 2009Co-Authors: Changhu Wang, Lei Zhang, Shuicheng Yan, Hongjiang ZhangAbstract:In this paper, we present a multi-label sparse coding framework for feature extraction and classification within the context of automatic Image Annotation. First, each Image is encoded into a so-called supervector, derived from the universal Gaussian Mixture Models on orderless Image patches. Then, a label sparse coding based subspace learning algorithm is derived to effectively harness multi-label information for dimensionality reduction. Finally, the sparse coding method for multi-label data is proposed to propagate the multi-labels of the training Images to the query Image with the sparse l1 reconstruction coefficients. Extensive Image Annotation experiments on the Corel5k and Corel30k databases both show the superior performance of the proposed multi-label sparse coding framework over the state-of-the-art algorithms.
-
a probabilistic semantic model for Image Annotation and multimodal Image retrieval
International Conference on Computer Vision, 2005Co-Authors: Ruofei Zhang, Zhongfei Zhang, Hongjiang ZhangAbstract:This paper addresses automatic Image Annotation problem and its application to multi-modal Image retrieval. The contribution of our work is three-fold. (1) We propose a probabilistic semantic model in which the visual features and the textual words are connected via a hidden layer which constitutes the semantic concepts to be discovered to explicitly exploit the synergy among the modalities. (2) The association of visual features and textual words is determined in a Bayesian framework such that the confidence of the association can be provided. (3) Extensive evaluation on a large-scale, visually and semantically diverse Image collection crawled from Web is reported to evaluate the prototype system based on the model. In the proposed probabilistic model, a hidden concept layer which connects the visual feature and the word layer is discovered by fitting a generative model to the training Image and Annotation words through an Expectation-Maximization (EM) based iterative learning procedure. The evaluation of the prototype system on 17,000 Images and 7,736 automatically extracted Annotation words from crawled Web pages for multi-modal Image retrieval has indicated that the proposed semantic model and the developed Bayesian framework are superior to a state-of-the-art peer system in the literature.
-
semi automatic Image Annotation
International Conference on Human-Computer Interaction, 2001Co-Authors: Liu Wenyin, Hongjiang Zhang, Susan T Dumais, Yanfeng Sun, Mary Czerwinski, Brent A FieldAbstract:A novel approach to semi-automatically and progressively annotating Images with keywords is presented. The progressive Annotation process is embedded in the course of integrated keyword-based and content-based Image retrieval and user feedback. When the user submits a keyword query and then provides relevance feedback, the search keywords are automatically added to the Images that receive positive feedback and can then facilitate keyword-based Image retrieval in the future. The coverage and quality of Image Annotation in such a database system is improved progressively as the cycle of search and feedback increases. The strategy of semi-automatic Image Annotation is better than manual Annotation in terms of efficiency and better than automatic Annotation in terms of accuracy. A performance study is presented which shows that high Annotation coverage can be achieved with this approach, and a preliminary user study is described showing that users view Annotations as important and will likely use them in Image retrieval. The user study also suggested user interface enhancements needed to support relevance feedback. We believe that similar approaches could also be applied to annotating and managing other forms of multimedia objects.
