The Experts below are selected from a list of 1392 Experts worldwide ranked by ideXlab platform
Haojie Li - One of the best experts on this subject based on the ideXlab platform.
-
High Quality Depth Estimation from Monocular Images Based on Depth Prediction and Enhancement Sub-Networks
2018 IEEE International Conference on Multimedia and Expo (ICME), 2018Co-Authors: Xiangyue Duan, Xinchen Ye, Yang Li, Haojie LiAbstract:This paper addresses the problem of depth estimation from a single RGB image. Previous methods mainly focus on the problems of depth Prediction accuracy and output depth resolution, but seldom of them can tackle these two problems well. Here, we present a novel depth estimation framework based on deep convolutional neural Network (CNN) to learn the mapping between monocular images and depth maps. The proposed architecture can be divided into two components, i.e., depth Prediction and depth enhancement sub-Networks. We first design a depth Prediction Network based on the ResNet architecture to infer the scene depth from color image. Then, a depth enhancement Network is concatenated to the end of the depth Prediction Network to obtain a high resolution depth map. Experimental results show that the proposed method outperforms other methods on benchmark RGB-D datasets and achieves state-of-the-art performance.
-
ICME - High Quality Depth Estimation from Monocular Images Based on Depth Prediction and Enhancement Sub-Networks
2018 IEEE International Conference on Multimedia and Expo (ICME), 2018Co-Authors: Xiangyue Duan, Xinchen Ye, Yang Li, Haojie LiAbstract:This paper addresses the problem of depth estimation from a single RGB image. Previous methods mainly focus on the problems of depth Prediction accuracy and output depth resolution, but seldom of them can tackle these two problems well. Here, we present a novel depth estimation framework based on deep convolutional neural Network (CNN) to learn the mapping between monocular images and depth maps. The proposed architecture can be divided into two components, i.e., depth Prediction and depth enhancement sub-Networks. We first design a depth Prediction Network based on the ResNet architecture to infer the scene depth from color image. Then, a depth enhancement Network is concatenated to the end of the depth Prediction Network to obtain a high resolution depth map. Experimental results show that the proposed method outperforms other methods on benchmark RGB-D datasets and achieves state-of-the-art performance.
Kwang-ting Cheng - One of the best experts on this subject based on the ideXlab platform.
-
Reactive obstacle avoidance of monocular quadrotors with online adapted depth Prediction Network
Neurocomputing, 2019Co-Authors: Xin Yang, Yuhao Wu, Chunyuan Liao, Kwang-ting ChengAbstract:Abstract Obstacle avoidance based on a monocular camera is a fundamental yet highly challenging task due to the lack of 3D information for a monocular quadrotor. Recent methods based on convolutional neural Networks (CNNs) [1] for monocular depth estimation and obstacle detection become increasingly popular due to the considerable advances in deep learning. However, depth estimation by pre-trained CNNs usually suffers from large accuracy degradation for scenes of different types from the training data which are common for obstacle avoidance of drones in unknown environments. In this paper, we present a reactive obstacle avoidance system which employs an online adaptive CNN for progressively improving depth estimation from a monocular camera in unfamiliar environments. Pairs of motion stereo images are collected on-the-fly as training data based on a direct monocular SLAM running in parallel with the CNN. Novel approaches are introduced for selecting highly reliable training samples from noisy data provided by SLAM and efficient online CNN tuning. The depth map computed from the CNN is transformed into Ego Dynamic Space (EDS) by embedding both dynamic motion constraints of a quadrotor and depth estimation errors into the spatial depth map. Traversable waypoints with consideration of the camera’s field of view constraint are automatically computed in EDS based on which appropriate control inputs for the quadcopter are produced. Experimental results on both public datasets, simulated environments and unseen cluttered indoor environments demonstrate the effectiveness of our system.
-
Real-Time Dense Monocular SLAM With Online Adapted Depth Prediction Network
IEEE Transactions on Multimedia, 2019Co-Authors: Yuhao Wu, Chunyuan Liao, Xin Yang, Kwang-ting ChengAbstract:Considerable advances have been achieved in estimating the depth map from a single image via convolutional neural Networks (CNNs) during the past few years. Combining depth Prediction from CNNs with conventional monocular simultaneous localization and mapping (SLAM) is promising for accurate and dense monocular reconstruction, in particular addressing the two long-standing challenges in conventional monocular SLAM: low map completeness and scale ambiguity. However, depth estimated by pretrained CNNs usually fails to achieve sufficient accuracy for environments of different types from the training data, which are common for certain applications such as obstacle avoidance of drones in unknown scenes. Additionally, inaccurate depth Prediction of CNN could yield large tracking errors in monocular SLAM. In this paper, we present a real-time dense monocular SLAM system, which effectively fuses direct monocular SLAM with an online-adapted depth Prediction Network for achieving accurate depth Prediction of scenes of different types from the training data and providing absolute scale information for tracking and mapping. Specifically, on one hand, tracking pose (i.e., translation and rotation) from direct SLAM is used for selecting a small set of highly effective and reliable training images, which acts as ground truth for tuning the depth Prediction Network on-the-fly toward better generalization ability for scenes of different types. A stage-wise Stochastic Gradient Descent algorithm with a selective update strategy is introduced for efficient convergence of the tuning process. On the other hand, the dense map produced by the adapted Network is applied to address scale ambiguity of direct monocular SLAM which in turn improves the accuracy of both tracking and overall reconstruction. The system with assistance of both CPUs and GPUs, can achieve real-time performance with progressively improved reconstruction accuracy. Experimental results on public datasets and live application to obstacle avoidance of drones demonstrate that our method outperforms the state-of-the-art methods with greater map completeness and accuracy, and a smaller tracking error.
Chun-ying Huang - One of the best experts on this subject based on the ideXlab platform.
-
Optimizing Fixation Prediction Using Recurrent Neural Networks for 360 $^{\circ }$ Video Streaming in Head-Mounted Virtual Reality
IEEE Transactions on Multimedia, 2020Co-Authors: Chun-ying HuangAbstract:We study the problem of predicting the viewing probability of different parts of $3\text{60}^{\circ }$ videos when streaming them to head-mounted displays. We propose a fixation Prediction Network based on recurrent neural Network, which leverages sensor and content features. The content features are derived by computer vision (CV) algorithms, which may suffer from inferior performance due to various types of distortion caused by diverse $\text{360}^{\circ }$ video projection models. We propose a unified approach with overlapping virtual viewports to eliminate such negative effects, and we evaluate our proposed solution using several CV algorithms, such as saliency detection, face detection, and object detection. We find that overlapping virtual viewports increase the performance of these existing CV algorithms that were not trained for $\text{360}^{\circ }$ videos. We next fine-tune our fixation Prediction Network with diverse design options, including: 1) with or without overlapping virtual viewports, 2) with or without future content features, and 3) different feature sampling rates. We empirically choose the best fixation Prediction Network and use it in a $\text{360}^{\circ }$ video streaming system. We conduct extensive trace-driven simulations with a large-scale dataset to quantify the performance of the $\text{360}^{\circ }$ video streaming system with different fixation Prediction algorithms. The results show that our proposed fixation Prediction Network outperforms other algorithms in several aspects, such as: 1) achieving comparable video quality (average gaps between −0.05 and 0.92 dB), 2) consuming much less bandwidth (average bandwidth reduction by up to 8 Mb/s), 3) reducing the rebuffering time (on average 40 s in bandwidth-limited 4G cellular Networks), and 4) running in real-time (at most 124 ms).
-
Optimizing Fixation Prediction Using Recurrent Neural Networks for 360$^{\circ }$ Video Streaming in Head-Mounted Virtual Reality
IEEE Transactions on Multimedia, 2020Co-Authors: Chun-ying HuangAbstract:We study the problem of predicting the viewing probability of different parts of 3600 videos when streaming them to head-mounted displays. We propose a fixation Prediction Network based on recurrent neural Network, which leverages sensor and content features. The content features are derived by computer vision (CV) algorithms, which may suffer from inferior performance due to various types of distortion caused by diverse 3600 video projection models. We propose a unified approach with overlapping virtual viewports to eliminate such negative effects, and we evaluate our proposed solution using several CV algorithms, such as saliency detection, face detection, and object detection. We find that overlapping virtual viewports increase the performance of these existing CV algorithms that were not trained for 3600 videos. We next fine-tune our fixation Prediction Network with diverse design options, including: 1) with or without overlapping virtual viewports, 2) with or without future content features, and 3) different feature sampling rates. We empirically choose the best fixation Prediction Network and use it in a 3600 video streaming system. We conduct extensive trace-driven simulations with a large-scale dataset to quantify the performance of the 3600 video streaming system with different fixation Prediction algorithms. The results show that our proposed fixation Prediction Network outperforms other algorithms in several aspects, such as: 1) achieving comparable video quality (average gaps between -0.05 and 0.92 dB), 2) consuming much less bandwidth (average bandwidth reduction by up to 8 Mb/s), 3) reducing the rebuffering time (on average 40 s in bandwidth-limited 4G cellular Networks), and 4) running in real-time (at most 124 ms).
Dong Yu - One of the best experts on this subject based on the ideXlab platform.
-
ICASSP - Prediction-adaptation-correction recurrent neural Networks for low-resource language speech recognition
2016 IEEE International Conference on Acoustics Speech and Signal Processing (ICASSP), 2016Co-Authors: Yu Zhang, Ekapol Chuangsuwanich, James Glass, Dong YuAbstract:In this paper, we investigate the use of Prediction-adaptation-correction recurrent neural Networks (PAC-RNNs) for low-resource speech recognition. A PAC-RNN is comprised of a pair of neural Networks in which a correction Network uses auxiliary information given by a Prediction Network to help estimate the state probability. The information from the correction Network is also used by the Prediction Network in a recurrent loop. Our model outperforms other state-of-the-art neural Networks (DNNs, LSTMs) on IARPA-Babel tasks. Moreover, transfer learning from a language that is similar to the target language can help improve performance further.
-
Prediction-Adaptation-Correction Recurrent Neural Networks for Low-Resource Language Speech Recognition
ICASSP 2016, 2016Co-Authors: Yu Zhang, Ekapol Chuangsuwanich, James Glass, Dong YuAbstract:In this paper, we investigate the use of Prediction-adaptation-correction recurrent neural Networks (PAC-RNNs) for low-resource speech recognition. A PAC-RNN is comprised of a pair of neural Networks in which a {\it correction} Network uses auxiliary information given by a {\it Prediction} Network to help estimate the state probability. The information from the correction Network is also used by the Prediction Network in a recurrent loop. Our model outperforms other state-of-the-art neural Networks (DNNs, LSTMs) on IARPA-Babel tasks. Moreover, transfer learning from a language that is similar to the target language can help improve performance further.
Xiangyue Duan - One of the best experts on this subject based on the ideXlab platform.
-
High Quality Depth Estimation from Monocular Images Based on Depth Prediction and Enhancement Sub-Networks
2018 IEEE International Conference on Multimedia and Expo (ICME), 2018Co-Authors: Xiangyue Duan, Xinchen Ye, Yang Li, Haojie LiAbstract:This paper addresses the problem of depth estimation from a single RGB image. Previous methods mainly focus on the problems of depth Prediction accuracy and output depth resolution, but seldom of them can tackle these two problems well. Here, we present a novel depth estimation framework based on deep convolutional neural Network (CNN) to learn the mapping between monocular images and depth maps. The proposed architecture can be divided into two components, i.e., depth Prediction and depth enhancement sub-Networks. We first design a depth Prediction Network based on the ResNet architecture to infer the scene depth from color image. Then, a depth enhancement Network is concatenated to the end of the depth Prediction Network to obtain a high resolution depth map. Experimental results show that the proposed method outperforms other methods on benchmark RGB-D datasets and achieves state-of-the-art performance.
-
ICME - High Quality Depth Estimation from Monocular Images Based on Depth Prediction and Enhancement Sub-Networks
2018 IEEE International Conference on Multimedia and Expo (ICME), 2018Co-Authors: Xiangyue Duan, Xinchen Ye, Yang Li, Haojie LiAbstract:This paper addresses the problem of depth estimation from a single RGB image. Previous methods mainly focus on the problems of depth Prediction accuracy and output depth resolution, but seldom of them can tackle these two problems well. Here, we present a novel depth estimation framework based on deep convolutional neural Network (CNN) to learn the mapping between monocular images and depth maps. The proposed architecture can be divided into two components, i.e., depth Prediction and depth enhancement sub-Networks. We first design a depth Prediction Network based on the ResNet architecture to infer the scene depth from color image. Then, a depth enhancement Network is concatenated to the end of the depth Prediction Network to obtain a high resolution depth map. Experimental results show that the proposed method outperforms other methods on benchmark RGB-D datasets and achieves state-of-the-art performance.