The Experts below are selected from a list of 288 Experts worldwide ranked by ideXlab platform
Chulho Shin - One of the best experts on this subject based on the ideXlab platform.
-
Z^2 Traversal Order: An interleaving approach for VR stereo rendering on tile-based GPUs
Computational Visual Media, 2017Co-Authors: Jae-ho Nah, Yeongkyu Lim, Chulho ShinAbstract:With increasing demands of virtual reality (VR) applications, efficient VR rendering techniques are becoming essential. Because VR stereo rendering has increased computational costs to separately render views for the left and right eyes, to reduce the rendering cost in VR applications, we present a novel Traversal Order for tile-based mobile GPU architectures: Z ^2 Traversal Order. In tile-based mobile GPU architectures, a tile Traversal Order that maximizes spatial locality can increase GPU cache efficiency. For VR applications, our approach improves upon the traditional Z Order curve. We render corresponding screen tiles in left and right views in turn, or simultaneously, and as a result, we can exploit spatial adjacency of the two tiles. To evaluate our approach, we conducted a trace-driven hardware simulation using Mesa and a hardware simulator. Our experimental results show that Z ^2 Traversal Order can reduce external memory bandwidth requirements and increase rendering performance.
-
z 2 Traversal Order for vr stereo rendering on tile based mobile gpus
International Conference on Computer Graphics and Interactive Techniques, 2016Co-Authors: Jae-ho Nah, Yeongkyu Lim, Chulho ShinAbstract:With increasing demands of virtual reality (VR) applications, efficient VR rendering techniques are becoming essential because VR stereo rendering requires increased computational costs to separately render views for the left and right eyes. To reduce the rendering cost in VR applications, we present a novel Traversal Order for tile-based mobile GPU architectures, called the Z2 Traversal Order. In tile-based mobile GPU architectures, a tile Traversal Order that maximizes spatial locality can increase the GPU cache efficiency. For VR applications, our approach improves the traditional Z-curve Order; we render two screen tiles in the left and right views by turns or simultaneously, as a result, we can exploit spatial locality between the two tiles. To evaluate our approach, we conducted a trace-driven hardware simulation using Mesa and a hardware simulator. The experimental results show that the Z2 Traversal Order can reduce external memory bandwidth requirements and can increase rendering performance.
-
SIGGRAPH Asia Technical Briefs - Z 2 Traversal Order for VR stereo rendering on tile-based mobile GPUs
SIGGRAPH ASIA 2016 Technical Briefs, 2016Co-Authors: Jae-ho Nah, Yeongkyu Lim, Chulho ShinAbstract:With increasing demands of virtual reality (VR) applications, efficient VR rendering techniques are becoming essential because VR stereo rendering requires increased computational costs to separately render views for the left and right eyes. To reduce the rendering cost in VR applications, we present a novel Traversal Order for tile-based mobile GPU architectures, called the Z2 Traversal Order. In tile-based mobile GPU architectures, a tile Traversal Order that maximizes spatial locality can increase the GPU cache efficiency. For VR applications, our approach improves the traditional Z-curve Order; we render two screen tiles in the left and right views by turns or simultaneously, as a result, we can exploit spatial locality between the two tiles. To evaluate our approach, we conducted a trace-driven hardware simulation using Mesa and a hardware simulator. The experimental results show that the Z2 Traversal Order can reduce external memory bandwidth requirements and can increase rendering performance.
Jiebo Luo - One of the best experts on this subject based on the ideXlab platform.
-
Action Recognition With Spatio–Temporal Visual Attention on Skeleton Image Sequences
IEEE Transactions on Circuits and Systems for Video Technology, 2019Co-Authors: Zhengyuan Yang, Jianchao Yang, Jiebo LuoAbstract:Action recognition with 3D skeleton sequences became popular due to its speed and robustness. The recently proposed convolutional neural networks (CNNs)-based methods show a good performance in learning spatio–temporal representations for skeleton sequences. Despite the good recognition accuracy achieved by previous CNN-based methods, there existed two problems that potentially limit the performance. First, previous skeleton representations were generated by chaining joints with a fixed Order. The corresponding semantic meaning was unclear and the structural information among the joints was lost. Second, previous models did not have an ability to focus on informative joints. The attention mechanism was important for skeleton-based action recognition because different joints contributed unequally toward the correct recognition. To solve these two problems, we proposed a novel CNN-based method for skeleton-based action recognition. We first redesigned the skeleton representations with a depth-first tree Traversal Order, which enhanced the semantic meaning of skeleton images and better preserved the associated structural information. We then proposed the general two-branch attention architecture that automatically focused on spatio–temporal key stages and filtered out unreliable joint predictions. Based on the proposed general architecture, we designed a global long-sequence attention network with refined branch structures. Furthermore, in Order to adjust the kernel’s spatio–temporal aspect ratios and better capture long-term dependencies, we proposed a sub-sequence attention network (SSAN) that took sub-image sequences as inputs. We showed that the two-branch attention architecture could be combined with the SSAN to further improve the performance. Our experiment results on the NTU RGB+D data set and the SBU kinetic interaction data set outperformed the state of the art. The model was further validated on noisy estimated poses from the subsets of the UCF101 data set and the kinetics data set.
-
Action Recognition with Visual Attention on Skeleton Images
arXiv: Computer Vision and Pattern Recognition, 2018Co-Authors: Zhengyuan Yang, Jianchao Yang, Jiebo LuoAbstract:Action recognition with 3D skeleton sequences is becoming popular due to its speed and robustness. The recently proposed Convolutional Neural Networks (CNN) based methods have shown good performance in learning spatio-temporal representations for skeleton sequences. Despite the good recognition accuracy achieved by previous CNN based methods, there exist two problems that potentially limit the performance. First, previous skeleton representations are generated by chaining joints with a fixed Order. The corresponding semantic meaning is unclear and the structural information among the joints is lost. Second, previous models do not have an ability to focus on informative joints. The attention mechanism is important for skeleton based action recognition because there exist spatio-temporal key stages and the joint predictions can be inaccurate. To solve the two problems, we propose a novel CNN based method for skeleton based action recognition. We first redesign the skeleton representations with a depth-first tree Traversal Order, which enhances the semantic meaning of skeleton images and better preserves the structural information. We then propose the idea of a two-branch attention architecture that focuses on spatio-temporal key stages and filters out unreliable joint predictions. A base attention model with the simplest structure is first introduced to illustrate the two-branch attention architecture. By improving the structures in both branches, we further propose a Global Long-sequence Attention Network (GLAN). Experiment results on the NTU RGB+D dataset and the SBU Kinetic Interaction dataset show that our proposed approach outperforms the state-of-the-art, as well as the effectiveness of each component.
-
Action Recognition with Spatio-Temporal Visual Attention on Skeleton Image Sequences
arXiv: Computer Vision and Pattern Recognition, 2018Co-Authors: Zhengyuan Yang, Jianchao Yang, Jiebo LuoAbstract:Action recognition with 3D skeleton sequences is becoming popular due to its speed and robustness. The recently proposed Convolutional Neural Networks (CNN) based methods have shown good performance in learning spatio-temporal representations for skeleton sequences. Despite the good recognition accuracy achieved by previous CNN based methods, there exist two problems that potentially limit the performance. First, previous skeleton representations are generated by chaining joints with a fixed Order. The corresponding semantic meaning is unclear and the structural information among the joints is lost. Second, previous models do not have an ability to focus on informative joints. The attention mechanism is important for skeleton based action recognition because there exist spatio-temporal key stages while the joint predictions can be inaccurate. To solve these two problems, we propose a novel CNN based method for skeleton based action recognition. We first redesign the skeleton representations with a depth-first tree Traversal Order, which enhances the semantic meaning of skeleton images and better preserves the associated structural information. We then propose the idea of a two-branch attention architecture that focuses on spatio-temporal key stages and filters out unreliable joint predictions. A base attention model with the simplest structure is first introduced. By improving the structures in both branches, we further propose a Global Long-sequence Attention Network (GLAN). Furthermore, in Order to adjust the kernel's spatio-temporal aspect ratios and better capture long term dependencies, we propose a Sub-Sequence Attention Network (SSAN) that takes sub-image sequences as inputs. Our experiment results on NTU RGB+D and SBU Kinetic Interaction outperforms the state-of-the-art. The model is further validated on noisy estimated poses from UCF101 and Kinetics.
-
ICPR - Action Recognition with Visual Attention on Skeleton Images
2018 24th International Conference on Pattern Recognition (ICPR), 2018Co-Authors: Zhengyuan Yang, Jianchao Yang, Jiebo LuoAbstract:Action recognition with 3D skeleton sequences is becoming popular due to its speed and robustness. The recently proposed Convolutional Neural Networks (CNN) based methods have shown good performance in learning spatio-temporal representations for skeleton sequences. Despite the good recognition accuracy achieved by previous CNN based methods, there exist two problems that potentially limit the performance. First, previous skeleton representations are generated by chaining joints with a fixed Order. The corresponding semantic meaning is unclear and the structural information among the joints is lost. Second, previous models do not have an ability to focus on informative joints. The attention mechanism is important for skeleton based action recognition because there exist spatio-temporal key stages and the joint predictions can be inaccurate. To solve the two problems, we propose a novel CNN based method for skeleton based action recognition. We first redesign the skeleton representations with a depth-first tree Traversal Order, which enhances the semantic meaning of skeleton images and better preserves the structural information. We then propose the idea of a two-branch attention architecture that focuses on spatio-temporal key stages and filters out unreliable joint predictions. A base attention model with the simplest structure is first introduced to illustrate the two-branch attention architecture. By improving the structures in both branches, we further propose a Global Long-sequence Attention Network (GLAN). Experiment results on the NTU RGB+D dataset and the SBU Kinetic Interaction dataset show that our proposed approach outperforms the state-of-the-art, as well as the effectiveness of each component.
Jae-ho Nah - One of the best experts on this subject based on the ideXlab platform.
-
Z^2 Traversal Order: An interleaving approach for VR stereo rendering on tile-based GPUs
Computational Visual Media, 2017Co-Authors: Jae-ho Nah, Yeongkyu Lim, Chulho ShinAbstract:With increasing demands of virtual reality (VR) applications, efficient VR rendering techniques are becoming essential. Because VR stereo rendering has increased computational costs to separately render views for the left and right eyes, to reduce the rendering cost in VR applications, we present a novel Traversal Order for tile-based mobile GPU architectures: Z ^2 Traversal Order. In tile-based mobile GPU architectures, a tile Traversal Order that maximizes spatial locality can increase GPU cache efficiency. For VR applications, our approach improves upon the traditional Z Order curve. We render corresponding screen tiles in left and right views in turn, or simultaneously, and as a result, we can exploit spatial adjacency of the two tiles. To evaluate our approach, we conducted a trace-driven hardware simulation using Mesa and a hardware simulator. Our experimental results show that Z ^2 Traversal Order can reduce external memory bandwidth requirements and increase rendering performance.
-
z 2 Traversal Order for vr stereo rendering on tile based mobile gpus
International Conference on Computer Graphics and Interactive Techniques, 2016Co-Authors: Jae-ho Nah, Yeongkyu Lim, Chulho ShinAbstract:With increasing demands of virtual reality (VR) applications, efficient VR rendering techniques are becoming essential because VR stereo rendering requires increased computational costs to separately render views for the left and right eyes. To reduce the rendering cost in VR applications, we present a novel Traversal Order for tile-based mobile GPU architectures, called the Z2 Traversal Order. In tile-based mobile GPU architectures, a tile Traversal Order that maximizes spatial locality can increase the GPU cache efficiency. For VR applications, our approach improves the traditional Z-curve Order; we render two screen tiles in the left and right views by turns or simultaneously, as a result, we can exploit spatial locality between the two tiles. To evaluate our approach, we conducted a trace-driven hardware simulation using Mesa and a hardware simulator. The experimental results show that the Z2 Traversal Order can reduce external memory bandwidth requirements and can increase rendering performance.
-
SIGGRAPH Asia Technical Briefs - Z 2 Traversal Order for VR stereo rendering on tile-based mobile GPUs
SIGGRAPH ASIA 2016 Technical Briefs, 2016Co-Authors: Jae-ho Nah, Yeongkyu Lim, Chulho ShinAbstract:With increasing demands of virtual reality (VR) applications, efficient VR rendering techniques are becoming essential because VR stereo rendering requires increased computational costs to separately render views for the left and right eyes. To reduce the rendering cost in VR applications, we present a novel Traversal Order for tile-based mobile GPU architectures, called the Z2 Traversal Order. In tile-based mobile GPU architectures, a tile Traversal Order that maximizes spatial locality can increase the GPU cache efficiency. For VR applications, our approach improves the traditional Z-curve Order; we render two screen tiles in the left and right views by turns or simultaneously, as a result, we can exploit spatial locality between the two tiles. To evaluate our approach, we conducted a trace-driven hardware simulation using Mesa and a hardware simulator. The experimental results show that the Z2 Traversal Order can reduce external memory bandwidth requirements and can increase rendering performance.
-
SATO: Surface Area Traversal Order for Shadow Ray Tracing
Computer Graphics Forum, 2014Co-Authors: Jae-ho Nah, Dinesh ManochaAbstract:We present the surface area Traversal Order SATO metric to accelerate shadow ray Traversal. Our formulation uses the surface area of each child node to compute the TO. In this metric, we give a Traversal priority to the child node with the larger surface area to quickly find occluders. Our algorithm reduces the pre-processing overhead significantly, and is much faster than other metrics. Overall, the SATO is useful for ray tracing large and complex dynamic scenes e.g. a few million triangles with shadows.
Zhengyuan Yang - One of the best experts on this subject based on the ideXlab platform.
-
Action Recognition With Spatio–Temporal Visual Attention on Skeleton Image Sequences
IEEE Transactions on Circuits and Systems for Video Technology, 2019Co-Authors: Zhengyuan Yang, Jianchao Yang, Jiebo LuoAbstract:Action recognition with 3D skeleton sequences became popular due to its speed and robustness. The recently proposed convolutional neural networks (CNNs)-based methods show a good performance in learning spatio–temporal representations for skeleton sequences. Despite the good recognition accuracy achieved by previous CNN-based methods, there existed two problems that potentially limit the performance. First, previous skeleton representations were generated by chaining joints with a fixed Order. The corresponding semantic meaning was unclear and the structural information among the joints was lost. Second, previous models did not have an ability to focus on informative joints. The attention mechanism was important for skeleton-based action recognition because different joints contributed unequally toward the correct recognition. To solve these two problems, we proposed a novel CNN-based method for skeleton-based action recognition. We first redesigned the skeleton representations with a depth-first tree Traversal Order, which enhanced the semantic meaning of skeleton images and better preserved the associated structural information. We then proposed the general two-branch attention architecture that automatically focused on spatio–temporal key stages and filtered out unreliable joint predictions. Based on the proposed general architecture, we designed a global long-sequence attention network with refined branch structures. Furthermore, in Order to adjust the kernel’s spatio–temporal aspect ratios and better capture long-term dependencies, we proposed a sub-sequence attention network (SSAN) that took sub-image sequences as inputs. We showed that the two-branch attention architecture could be combined with the SSAN to further improve the performance. Our experiment results on the NTU RGB+D data set and the SBU kinetic interaction data set outperformed the state of the art. The model was further validated on noisy estimated poses from the subsets of the UCF101 data set and the kinetics data set.
-
Action Recognition with Visual Attention on Skeleton Images
arXiv: Computer Vision and Pattern Recognition, 2018Co-Authors: Zhengyuan Yang, Jianchao Yang, Jiebo LuoAbstract:Action recognition with 3D skeleton sequences is becoming popular due to its speed and robustness. The recently proposed Convolutional Neural Networks (CNN) based methods have shown good performance in learning spatio-temporal representations for skeleton sequences. Despite the good recognition accuracy achieved by previous CNN based methods, there exist two problems that potentially limit the performance. First, previous skeleton representations are generated by chaining joints with a fixed Order. The corresponding semantic meaning is unclear and the structural information among the joints is lost. Second, previous models do not have an ability to focus on informative joints. The attention mechanism is important for skeleton based action recognition because there exist spatio-temporal key stages and the joint predictions can be inaccurate. To solve the two problems, we propose a novel CNN based method for skeleton based action recognition. We first redesign the skeleton representations with a depth-first tree Traversal Order, which enhances the semantic meaning of skeleton images and better preserves the structural information. We then propose the idea of a two-branch attention architecture that focuses on spatio-temporal key stages and filters out unreliable joint predictions. A base attention model with the simplest structure is first introduced to illustrate the two-branch attention architecture. By improving the structures in both branches, we further propose a Global Long-sequence Attention Network (GLAN). Experiment results on the NTU RGB+D dataset and the SBU Kinetic Interaction dataset show that our proposed approach outperforms the state-of-the-art, as well as the effectiveness of each component.
-
Action Recognition with Spatio-Temporal Visual Attention on Skeleton Image Sequences
arXiv: Computer Vision and Pattern Recognition, 2018Co-Authors: Zhengyuan Yang, Jianchao Yang, Jiebo LuoAbstract:Action recognition with 3D skeleton sequences is becoming popular due to its speed and robustness. The recently proposed Convolutional Neural Networks (CNN) based methods have shown good performance in learning spatio-temporal representations for skeleton sequences. Despite the good recognition accuracy achieved by previous CNN based methods, there exist two problems that potentially limit the performance. First, previous skeleton representations are generated by chaining joints with a fixed Order. The corresponding semantic meaning is unclear and the structural information among the joints is lost. Second, previous models do not have an ability to focus on informative joints. The attention mechanism is important for skeleton based action recognition because there exist spatio-temporal key stages while the joint predictions can be inaccurate. To solve these two problems, we propose a novel CNN based method for skeleton based action recognition. We first redesign the skeleton representations with a depth-first tree Traversal Order, which enhances the semantic meaning of skeleton images and better preserves the associated structural information. We then propose the idea of a two-branch attention architecture that focuses on spatio-temporal key stages and filters out unreliable joint predictions. A base attention model with the simplest structure is first introduced. By improving the structures in both branches, we further propose a Global Long-sequence Attention Network (GLAN). Furthermore, in Order to adjust the kernel's spatio-temporal aspect ratios and better capture long term dependencies, we propose a Sub-Sequence Attention Network (SSAN) that takes sub-image sequences as inputs. Our experiment results on NTU RGB+D and SBU Kinetic Interaction outperforms the state-of-the-art. The model is further validated on noisy estimated poses from UCF101 and Kinetics.
-
ICPR - Action Recognition with Visual Attention on Skeleton Images
2018 24th International Conference on Pattern Recognition (ICPR), 2018Co-Authors: Zhengyuan Yang, Jianchao Yang, Jiebo LuoAbstract:Action recognition with 3D skeleton sequences is becoming popular due to its speed and robustness. The recently proposed Convolutional Neural Networks (CNN) based methods have shown good performance in learning spatio-temporal representations for skeleton sequences. Despite the good recognition accuracy achieved by previous CNN based methods, there exist two problems that potentially limit the performance. First, previous skeleton representations are generated by chaining joints with a fixed Order. The corresponding semantic meaning is unclear and the structural information among the joints is lost. Second, previous models do not have an ability to focus on informative joints. The attention mechanism is important for skeleton based action recognition because there exist spatio-temporal key stages and the joint predictions can be inaccurate. To solve the two problems, we propose a novel CNN based method for skeleton based action recognition. We first redesign the skeleton representations with a depth-first tree Traversal Order, which enhances the semantic meaning of skeleton images and better preserves the structural information. We then propose the idea of a two-branch attention architecture that focuses on spatio-temporal key stages and filters out unreliable joint predictions. A base attention model with the simplest structure is first introduced to illustrate the two-branch attention architecture. By improving the structures in both branches, we further propose a Global Long-sequence Attention Network (GLAN). Experiment results on the NTU RGB+D dataset and the SBU Kinetic Interaction dataset show that our proposed approach outperforms the state-of-the-art, as well as the effectiveness of each component.
Yeongkyu Lim - One of the best experts on this subject based on the ideXlab platform.
-
Z^2 Traversal Order: An interleaving approach for VR stereo rendering on tile-based GPUs
Computational Visual Media, 2017Co-Authors: Jae-ho Nah, Yeongkyu Lim, Chulho ShinAbstract:With increasing demands of virtual reality (VR) applications, efficient VR rendering techniques are becoming essential. Because VR stereo rendering has increased computational costs to separately render views for the left and right eyes, to reduce the rendering cost in VR applications, we present a novel Traversal Order for tile-based mobile GPU architectures: Z ^2 Traversal Order. In tile-based mobile GPU architectures, a tile Traversal Order that maximizes spatial locality can increase GPU cache efficiency. For VR applications, our approach improves upon the traditional Z Order curve. We render corresponding screen tiles in left and right views in turn, or simultaneously, and as a result, we can exploit spatial adjacency of the two tiles. To evaluate our approach, we conducted a trace-driven hardware simulation using Mesa and a hardware simulator. Our experimental results show that Z ^2 Traversal Order can reduce external memory bandwidth requirements and increase rendering performance.
-
z 2 Traversal Order for vr stereo rendering on tile based mobile gpus
International Conference on Computer Graphics and Interactive Techniques, 2016Co-Authors: Jae-ho Nah, Yeongkyu Lim, Chulho ShinAbstract:With increasing demands of virtual reality (VR) applications, efficient VR rendering techniques are becoming essential because VR stereo rendering requires increased computational costs to separately render views for the left and right eyes. To reduce the rendering cost in VR applications, we present a novel Traversal Order for tile-based mobile GPU architectures, called the Z2 Traversal Order. In tile-based mobile GPU architectures, a tile Traversal Order that maximizes spatial locality can increase the GPU cache efficiency. For VR applications, our approach improves the traditional Z-curve Order; we render two screen tiles in the left and right views by turns or simultaneously, as a result, we can exploit spatial locality between the two tiles. To evaluate our approach, we conducted a trace-driven hardware simulation using Mesa and a hardware simulator. The experimental results show that the Z2 Traversal Order can reduce external memory bandwidth requirements and can increase rendering performance.
-
SIGGRAPH Asia Technical Briefs - Z 2 Traversal Order for VR stereo rendering on tile-based mobile GPUs
SIGGRAPH ASIA 2016 Technical Briefs, 2016Co-Authors: Jae-ho Nah, Yeongkyu Lim, Chulho ShinAbstract:With increasing demands of virtual reality (VR) applications, efficient VR rendering techniques are becoming essential because VR stereo rendering requires increased computational costs to separately render views for the left and right eyes. To reduce the rendering cost in VR applications, we present a novel Traversal Order for tile-based mobile GPU architectures, called the Z2 Traversal Order. In tile-based mobile GPU architectures, a tile Traversal Order that maximizes spatial locality can increase the GPU cache efficiency. For VR applications, our approach improves the traditional Z-curve Order; we render two screen tiles in the left and right views by turns or simultaneously, as a result, we can exploit spatial locality between the two tiles. To evaluate our approach, we conducted a trace-driven hardware simulation using Mesa and a hardware simulator. The experimental results show that the Z2 Traversal Order can reduce external memory bandwidth requirements and can increase rendering performance.
