The Experts below are selected from a list of 360 Experts worldwide ranked by ideXlab platform
Kate Saenko - One of the best experts on this subject based on the ideXlab platform.
-
ask attend and answer exploring question guided spatial attention for visual question answering
European Conference on Computer Vision, 2016Co-Authors: Kate SaenkoAbstract:We address the problem of Visual Question Answering (VQA), which requires joint image and language understanding to answer a question about a given photograph. Recent approaches have applied deep image captioning methods based on convolutional-recurrent Networks to this problem, but have failed to model spatial inference. To remedy this, we propose a model we call the Spatial Memory Network and apply it to the VQA task. Memory Networks are recurrent neural Networks with an explicit attention mechanism that selects certain parts of the information stored in Memory. Our Spatial Memory Network stores neuron activations from different spatial regions of the image in its Memory, and uses attention to choose regions relevant for computing the answer. We propose a novel question-guided spatial attention architecture that looks for regions relevant to either individual words or the entire question, repeating the process over multiple recurrent steps, or “hops”. To better understand the inference process learned by the Network, we design synthetic questions that specifically require spatial inference and visualize the Network’s attention. We evaluate our model on two available visual question answering datasets and obtain improved results.
-
ask attend and answer exploring question guided spatial attention for visual question answering
arXiv: Computer Vision and Pattern Recognition, 2015Co-Authors: Kate SaenkoAbstract:We address the problem of Visual Question Answering (VQA), which requires joint image and language understanding to answer a question about a given photograph. Recent approaches have applied deep image captioning methods based on convolutional-recurrent Networks to this problem, but have failed to model spatial inference. To remedy this, we propose a model we call the Spatial Memory Network and apply it to the VQA task. Memory Networks are recurrent neural Networks with an explicit attention mechanism that selects certain parts of the information stored in Memory. Our Spatial Memory Network stores neuron activations from different spatial regions of the image in its Memory, and uses the question to choose relevant regions for computing the answer, a process of which constitutes a single "hop" in the Network. We propose a novel spatial attention architecture that aligns words with image patches in the first hop, and obtain improved results by adding a second attention hop which considers the whole question to choose visual evidence based on the results of the first hop. To better understand the inference process learned by the Network, we design synthetic questions that specifically require spatial inference and visualize the attention weights. We evaluate our model on two published visual question answering datasets, DAQUAR [1] and VQA [2], and obtain improved results compared to a strong deep baseline model (iBOWIMG) which concatenates image and question features to predict the answer [3].
Erhardt Barth - One of the best experts on this subject based on the ideXlab platform.
-
recurrent dropout without Memory loss
International Conference on Computational Linguistics, 2016Co-Authors: Stanislau Semeniuta, Aliaksei Severyn, Erhardt BarthAbstract:This paper presents a novel approach to recurrent neural Network (RNN) regularization. Differently from the widely adopted dropout method, which is applied to forward connections of feedforward architectures or RNNs, we propose to drop neurons directly in recurrent connections in a way that does not cause loss of long-term Memory. Our approach is as easy to implement and apply as the regular feed-forward dropout and we demonstrate its effectiveness for the most effective modern recurrent Network – Long Short-Term Memory Network. Our experiments on three NLP benchmarks show consistent improvements even when combined with conventional feed-forward dropout.
-
recurrent dropout without Memory loss
arXiv: Computation and Language, 2016Co-Authors: Stanislau Semeniuta, Aliaksei Severyn, Erhardt BarthAbstract:This paper presents a novel approach to recurrent neural Network (RNN) regularization. Differently from the widely adopted dropout method, which is applied to \textit{forward} connections of feed-forward architectures or RNNs, we propose to drop neurons directly in \textit{recurrent} connections in a way that does not cause loss of long-term Memory. Our approach is as easy to implement and apply as the regular feed-forward dropout and we demonstrate its effectiveness for Long Short-Term Memory Network, the most popular type of RNN cells. Our experiments on NLP benchmarks show consistent improvements even when combined with conventional feed-forward dropout.
Zhi Yang - One of the best experts on this subject based on the ideXlab platform.
-
parietal Memory Network and default mode Network in first episode drug naive schizophrenia associations with auditory hallucination
Human Brain Mapping, 2020Co-Authors: Qian Guo, Botao Zeng, Yingying Tang, Tianhong Zhang, Jinhong Wang, Georg Northoff, Donald C Goff, Jijun Wang, Zhi YangAbstract:Atypical spontaneous activities in resting-state Networks may play a role in auditory hallucinations (AHs), but Networks relevant to AHs are not apparent. Given the debating role of the default mode Network (DMN) in AHs, a parietal Memory Network (PMN) may better echo cognitive theories of AHs in schizophrenia, because PMN is spatially adjacent to the DMN and more relevant to Memory processing or information integration. To examine whether PMN is more relevant to AHs than DMN, we characterized these intrinsic Networks in AHs with 59 first-episode, drug-naive schizophrenics (26 AH+ and 33 AH-) and 60 healthy participants in resting-state fMRI. We separated the PMN, DMN, and auditory Network (AN) using independent component analysis, and compared their functional connectivity across the three groups. We found that only AH+ patients displayed dysconnectivity in PMN, both AH+ and AH- patients exhibited dysfunctions of AN, but neither patient group showed abnormal connectivity within DMN. The connectivity of PMN significantly correlated with Memory performance of the patients. Further region-of-interest analyses confirmed that the connectivity between the core regions of PMN, the left posterior cingulate gyrus and the left precuneus, was significantly lower only in the AH+ group. In exploratory correlation analysis, this functional connectivity metric significantly correlated with the severity of AH symptoms. The results implicate that compared to the DMN, the PMN is more relevant to the AH symptoms in schizophrenia, and further provides a more precise potential brain modulation target for the intervention of AH symptoms.
-
Data_Sheet_1_Loss of Parietal Memory Network Integrity in Alzheimer’s Disease.docx
2019Co-Authors: Yiwen Zhang, Ying Han, Zhi YangAbstract:A functional brain Network, termed the parietal Memory Network (PMN), has been shown to reflect the familiarity of stimuli in both Memory encoding and retrieval. The function of this Network has been separated from the commonly investigated default mode Network (DMN) in both resting-state fMRI and task-activations. This study examined the deficit of the PMN in Alzheimer’s disease (AD) patients using resting-state fMRI and independent component analysis (ICA) and investigated its diagnostic value in identifying AD patients. The DMN was also examined as a reference Network. In addition, the robustness of the findings was examined using different types of analysis methods and parameters. Our results showed that the integrity as an intrinsic connectivity Network for the PMN was significantly decreased in AD and this feature showed at least equivalent predictive ability to that for the DMN. These findings were robust to varied methods and parameters. Our findings suggest that the intrinsic connectivity of the PMN is disrupted in AD and further call for considering the PMN and the DMN separately in clinical neuroimaging studies.
-
deficit in parietal Memory Network underlies auditory hallucination a longitudinal study
bioRxiv, 2017Co-Authors: Qian Guo, Botao Zeng, Yingying Tang, Tianhong Zhang, Jinhong Wang, Georg Northoff, Donald C Goff, Jijun Wang, Zhi YangAbstract:Auditory hallucination is a prominent and common symptom in schizophrenia. Previous neuroimaging studies have yielded mixed results of its brain Network deficits. We proposed a novel hypothesis that parietal Memory Network, centered at the precuneus, plays a critical role in auditory hallucination. This Network is adjacent and partially overlaps with the default mode Network, and has been associated with brain function of familiarity labelling in Memory processing. Using a longitudinal design and a large cohort of first-episode, drug-naive schizophrenia patients, we examined this hypothesis and further investigated whether the functional connectivity patterns of the parietal Memory Network can serve as a neuroimaging marker for auditory hallucination and help to predict future treatment effects. Resting-state scans from 59 first-episode drug-naive schizophrenic patients (27 with and 32 without hallucination) and 53 healthy control subjects were acquired at the baseline test, and 56 of them were scanned again after two months. Functional connectivity strength within the parietal Memory Network and between this Network and Memory hubs was across the three groups at baseline and follow-up scans. Results showed that decreased functional connectivity strength within the parietal Memory Network was specific to the auditory hallucination group (p = 0.009, compare to the healthy subjects; p = 0.029, compare to the patients without hallucination), with the precuneus representing the largest group difference. The intra-Network connectivity strength of the precuneus negatively correlated with the severity of hallucination at the baseline scan (r = -0.437, p = 0.029), and it was significantly increased after two-month medication (p = 0.039). Logistic regression analysis and cross-validation test demonstrated that the functional connectivity strength of the precuneus and precuneus-hippocampus connectivity could differentiate patients with or without auditory hallucination with a sensitivity of 0.750 and a specificity of 0.708. Moreover, cross-validation test showed that these imaging features at the baseline scan well predicted the extents of positive symptom improvement in the hallucination group after the two-month medication (R2 = 0.433, p = 0.022). Our results provide evidence for a critical role of the parietal Memory Network underlying auditory hallucination, and further propose a novel neuroimaging marker for identifying patients, accessing severity, and prognosis of treatment effect for auditory hallucination.
-
segregation between the parietal Memory Network and the default mode Network effects of spatial smoothing and model order in ica
Chinese Science Bulletin, 2016Co-Authors: Yang Hu, Jijun Wang, Zhi Yang, Chunbo Li, Yinshan WangAbstract:A brain Network consisting of two key parietal nodes, the precuneus and the posterior cingulate cortex, has emerged from recent fMRI studies. Though it is anatomically adjacent to and spatially overlaps with the default mode Network (DMN), its function has been associated with Memory processing, and it has been referred to as the parietal Memory Network (PMN). Independent component analysis (ICA) is the most common data-driven method used to extract PMN and DMN simultaneously. However, the effects of data preprocessing and parameter determination in ICA on PMN–DMN segregation are completely unknown. Here, we employ three typical algorithms of group ICA to assess how spatial smoothing and model order influence the degree of PMN–DMN segregation. Our findings indicate that PMN and DMN can only be stably separated using a combination of low-level spatial smoothing and high model order across the three ICA algorithms. We thus argue for more considerations on parametric settings for interpreting DMN data.
-
segregation between the parietal Memory Network and the default mode Network effects of spatial smoothing and model order in ica
bioRxiv, 2016Co-Authors: Yang Hu, Jijun Wang, Chunbo Li, Yinshan Wang, Zhi YangAbstract:A brain Network consisting of two key parietal nodes, the precuneus and the posterior cingulate cortex, has emerged from recent fMRI studies. Though it is anatomically adjacent to and spatially overlaps with the default mode Network (DMN), its function has been associated with Memory processing, and it has been referred to as the parietal Memory Network (PMN). Independent component analysis (ICA) is the most common data-driven method of extracting PMN and DMN simultaneously. However, the effects of data preprocessing and parameter determination in ICA on PMN-DMN segregation are completely unknown. Here, we employ three typical algorithms of group ICA to assess how spatial smoothing and model order influence the degree of PMN-DMN segregation. Our findings indicate that PMN and DMN can only be stably separated using a combination of low-level spatial smoothing and high-order model across the three ICA algorithms. We thus argue for more considerations on parametric settings for interpreting DMN data.
Stanislau Semeniuta - One of the best experts on this subject based on the ideXlab platform.
-
recurrent dropout without Memory loss
International Conference on Computational Linguistics, 2016Co-Authors: Stanislau Semeniuta, Aliaksei Severyn, Erhardt BarthAbstract:This paper presents a novel approach to recurrent neural Network (RNN) regularization. Differently from the widely adopted dropout method, which is applied to forward connections of feedforward architectures or RNNs, we propose to drop neurons directly in recurrent connections in a way that does not cause loss of long-term Memory. Our approach is as easy to implement and apply as the regular feed-forward dropout and we demonstrate its effectiveness for the most effective modern recurrent Network – Long Short-Term Memory Network. Our experiments on three NLP benchmarks show consistent improvements even when combined with conventional feed-forward dropout.
-
recurrent dropout without Memory loss
arXiv: Computation and Language, 2016Co-Authors: Stanislau Semeniuta, Aliaksei Severyn, Erhardt BarthAbstract:This paper presents a novel approach to recurrent neural Network (RNN) regularization. Differently from the widely adopted dropout method, which is applied to \textit{forward} connections of feed-forward architectures or RNNs, we propose to drop neurons directly in \textit{recurrent} connections in a way that does not cause loss of long-term Memory. Our approach is as easy to implement and apply as the regular feed-forward dropout and we demonstrate its effectiveness for Long Short-Term Memory Network, the most popular type of RNN cells. Our experiments on NLP benchmarks show consistent improvements even when combined with conventional feed-forward dropout.
Hui Liu - One of the best experts on this subject based on the ideXlab platform.
-
smart deep learning based wind speed prediction model using wavelet packet decomposition convolutional neural Network and convolutional long short term Memory Network
Energy Conversion and Management, 2018Co-Authors: Hui LiuAbstract:Abstract High precision and reliable wind speed forecasting is important for the management of the wind power. This paper develops a novel wind speed prediction model based on the WPD (Wavelet Packet Decomposition), CNN (Convolutional Neural Network) and CNNLSTM (Convolutional Long Short Term Memory Network). In the proposed WPD-CNNLSTM-CNN model, the WPD is employed to decompose the original wind speed time series into a number of sub-layers; the CNN with 1D convolution operator is used to forecast the obtained high-frequency sub-layers; and the CNNLSTM is adopted to complete the forecasting of the low-frequency sub-layer. To verify and compare the prediction performance of the proposed model, eight models are used. According to the results of four experimental tests, it can be observed that: (1) the proposed model is robust and effective in predicting the 1D wind speed time series, besides, among the involved eight models, the proposed model can perform best in wind speed 1-step to 3-step predictions; (2) when the wind speed experiences sudden change, the proposed model can have better prediction performance than the other involved models.