The Experts below are selected from a list of 303 Experts worldwide ranked by ideXlab platform
Neil Cohn - One of the best experts on this subject based on the ideXlab platform.
-
Drawing the line between Constituent Structure and coherence relations in visual narratives.
Journal of experimental psychology. Learning memory and cognition, 2016Co-Authors: Neil Cohn, Patrick BenderAbstract:Theories of visual narrative understanding have often focused on the changes in meaning across a sequence, like shifts in characters, spatial location, and causation, as cues for breaks in the Structure of a discourse. In contrast, the theory of visual narrative grammar posits that hierarchic "grammatical" Structures operate at the discourse level using categorical roles for images, which may or may not co-occur with shifts in coherence. We therefore examined the relationship between narrative Structure and coherence shifts in the segmentation of visual narrative sequences using a "segmentation task" where participants drew lines between images in order to divide them into subepisodes. We used regressions to analyze the influence of the expected Constituent Structure boundary, narrative categories, and semantic coherence relationships on the segmentation of visual narrative sequences. Narrative categories were a stronger predictor of segmentation than linear coherence relationships between panels, though both influenced participants' divisions. Altogether, these results support the theory that meaningful sequential images use a narrative grammar that extends above and beyond linear semantic shifts between discourse units. (PsycINFO Database Record
-
The pieces fit: Constituent Structure and global coherence of visual narrative in RSVP
Acta psychologica, 2016Co-Authors: Carl Erick Hagmann, Neil CohnAbstract:Recent research has shown that comprehension of visual narrative relies on the ordering and timing of sequential images. Here we tested if rapidly presented 6-image long visual sequences could be understood as coherent narratives. Half of the sequences were correctly ordered and half had two of the four internal panels switched. Participants reported whether the sequence was correctly ordered and rated its coherence. Accuracy in detecting a switch increased when panels were presented for 1 s rather than 0.5 s. Doubling the duration of the first panel did not affect results. When two switched panels were further apart, order was discriminated more accurately and coherence ratings were low, revealing that a strong local adjacency effect influenced order and coherence judgments. Switched panels at Constituent boundaries or within Constituents were most disruptive to order discrimination, indicating that the preservation of Constituent Structure is critical to visual narrative grammar.
-
The grammar of visual narrative: Neural evidence for Constituent Structure in sequential image comprehension.
Neuropsychologia, 2014Co-Authors: Neil Cohn, Ray Jackendoff, Phillip J. Holcomb, Gina R. KuperbergAbstract:Abstract Constituent Structure has long been established as a central feature of human language. Analogous to how syntax organizes words in sentences, a narrative grammar organizes sequential images into hierarchic Constituents. Here we show that the brain draws upon this Constituent Structure to comprehend wordless visual narratives. We recorded neural responses as participants viewed sequences of visual images (comics strips) in which blank images either disrupted individual narrative Constituents or fell at natural Constituent boundaries. A disruption of either the first or the second narrative Constituent produced a left-lateralized anterior negativity effect between 500 and 700 ms. Disruption of the second Constituent also elicited a posteriorly-distributed positivity (P600) effect. These neural responses are similar to those associated with structural violations in language and music. These findings provide evidence that comprehenders use a narrative Structure to comprehend visual sequences and that the brain engages similar neurocognitive mechanisms to build Structure across multiple domains.
-
Segmenting Visual Narratives: Evidence for Constituent Structure in Comics - eScholarship
2012Co-Authors: Neil Cohn, Ray Jackendoff, Philip Holcomb, Gina R. KuperbergAbstract:Segmenting Visual Narratives: Evidence for Constituent Structure in Comics Neil Cohn (neilcohn@emaki.net), Phillip Holcomb Department of Psychology, 490 Boston Ave Medford, MA 02155 USA Ray Jackendoff Center for Cognitive Studies, Miner Hall Medford, MA 02155 USA Gina Kuperberg Department of Psychology, 490 Boston Ave Medford, MA 02155 USA Department of Psychiatry and Athinoula A. Martinos Center for Biomedical Imaging Massachusetts General Hospital, Bldg 149, 13th Street Charlestown, MA 02129 USA Abstract Background Since the 1950s, research on language has stressed that sentences are organized into hierarchic Constituents, rather than linear word-to-word connections (e.g., Chomsky, An analogous distinction between local relationships and hierarchic segmentation has underlined research of text and discourse. Traditional approaches emphasize coherence relationships between individual sentences (Halliday & Hasan, 1976; Zwaan & Radvansky, 1998). Participants can intuitively segment texts into consistent groupings (Gee & Grosjean, 1984; Mandler & Johnson, 1977), suggesting that readers’ comprehension extends beyond adjacent relationships between individual sentences. Some theories have formally described hierarchical relationships in narrative, such as “story grammars” (e.g. Mandler & Johnson, 1977; Rumelhart, 1975; Thorndyke, 1977), which use phrase Structure rules to organize narratives around characters’ goal-driven events. Constituent Structure in these models was examined using clustering models (Gee & Grosjean, 1984; Mandler, 1987; Mandler & Johnson, 1977), similar to those employed by early psycholinguistic research on syntactic relations (e.g. Levelt, 1970). Participants divided a narrative into logical groupings, and these intuitions were then submitted to the algorithms in hierarchic clustering models. These analyses yielded tree Structures that closely correlated with the models predicted by the original theories. As in verbal discourse, theories of visual narrative have also emphasized linear relationships between individual images (McCloud, 1993). However, beyond such linear relationships, participants also intuitively divide visual narratives into segments (Gernsbacher, 1985). Again, this suggests that comprehension may extend beyond adjacent relationships. Recently, Cohn (2003, In Press) has proposed a theoretical model of narrative Structure that formalizes the We have proposed that visual narratives in comics are organized with a hierarchic narrative “grammar.” Inspired by classic “click” studies of syntax, we inserted blank “disruption” panels Before, At, or After the Constituent boundaries of comic strips. In self-paced viewing, Experiment 1 found that blanks After the boundary were viewed slower than Before or At the boundary. Panels immediately following blanks were slower than corresponding panels in sequences without blank panels, but only when placed Before or After the boundary. Three ordinal panel positions following the boundary, panels following blanks At the boundary or with No-Blanks were viewed faster than panels following blanks After or Before the boundary. This supports constituency because disruptions had greater impact within, as opposed to between, Constituents. Using ERPs, Experiment 2 found a larger anterior negativity to blanks within Constituents (Before/After) than between Constituents (At). This indicates disruptions of Constituents are recognized before reaching a subsequent panel. A larger P600 appeared to blanks After the boundary than in the first Constituent (Before/At). This positivity may reflect a reanalysis reflecting the inability to integrate all prior panels into a single Constituent, since they are divided by the Constituent boundary. Keywords: Grammar; Constituent Structure; Discourse; Narrative; Comics; ERPs; Left Anterior Negativity. Introduction Drawings have conveyed narratives through sequences of images for thousands of years, but in contemporary society they appear most prevalently in comics. In comparison with research on the Structure and comprehension of verbal narrative, however, little is known about mechanisms of processing sequential images. For example, are narrative units integrated linearly across each adjacent relationship or they organized into hierarchic Constituents?
-
CogSci - Segmenting Visual Narratives: Evidence for Constituent Structure in Comics.
Cognitive Science, 2012Co-Authors: Neil Cohn, Ray Jackendoff, Phillip J. Holcomb, Gina R. KuperbergAbstract:We have proposed that visual narratives in comics are organized with a hierarchic narrative “grammar.” Inspired by classic “click” studies of syntax, we inserted blank “disruption” panels Before, At, or After the Constituent boundaries of comic strips. In self-paced viewing, Experiment 1 found that blanks After the boundary were viewed slower than Before or At the boundary. Panels immediately following blanks were slower than corresponding panels in sequences without blank panels, but only when placed Before or After the boundary. Three ordinal panel positions following the boundary, panels following blanks At the boundary or with No-Blanks were viewed faster than panels following blanks After or Before the boundary. This supports constituency because disruptions had greater impact within, as opposed to between, Constituents. Using ERPs, Experiment 2 found a larger anterior negativity to blanks within Constituents (Before/After) than between Constituents (At). This indicates disruptions of Constituents are recognized before reaching a subsequent panel. A larger P600 appeared to blanks After the boundary than in the first Constituent (Before/At). This positivity may reflect a reanalysis reflecting the inability to integrate all prior panels into a single Constituent, since they are divided by the Constituent boundary.
Gina R. Kuperberg - One of the best experts on this subject based on the ideXlab platform.
-
The grammar of visual narrative: Neural evidence for Constituent Structure in sequential image comprehension.
Neuropsychologia, 2014Co-Authors: Neil Cohn, Ray Jackendoff, Phillip J. Holcomb, Gina R. KuperbergAbstract:Abstract Constituent Structure has long been established as a central feature of human language. Analogous to how syntax organizes words in sentences, a narrative grammar organizes sequential images into hierarchic Constituents. Here we show that the brain draws upon this Constituent Structure to comprehend wordless visual narratives. We recorded neural responses as participants viewed sequences of visual images (comics strips) in which blank images either disrupted individual narrative Constituents or fell at natural Constituent boundaries. A disruption of either the first or the second narrative Constituent produced a left-lateralized anterior negativity effect between 500 and 700 ms. Disruption of the second Constituent also elicited a posteriorly-distributed positivity (P600) effect. These neural responses are similar to those associated with structural violations in language and music. These findings provide evidence that comprehenders use a narrative Structure to comprehend visual sequences and that the brain engages similar neurocognitive mechanisms to build Structure across multiple domains.
-
Segmenting Visual Narratives: Evidence for Constituent Structure in Comics - eScholarship
2012Co-Authors: Neil Cohn, Ray Jackendoff, Philip Holcomb, Gina R. KuperbergAbstract:Segmenting Visual Narratives: Evidence for Constituent Structure in Comics Neil Cohn (neilcohn@emaki.net), Phillip Holcomb Department of Psychology, 490 Boston Ave Medford, MA 02155 USA Ray Jackendoff Center for Cognitive Studies, Miner Hall Medford, MA 02155 USA Gina Kuperberg Department of Psychology, 490 Boston Ave Medford, MA 02155 USA Department of Psychiatry and Athinoula A. Martinos Center for Biomedical Imaging Massachusetts General Hospital, Bldg 149, 13th Street Charlestown, MA 02129 USA Abstract Background Since the 1950s, research on language has stressed that sentences are organized into hierarchic Constituents, rather than linear word-to-word connections (e.g., Chomsky, An analogous distinction between local relationships and hierarchic segmentation has underlined research of text and discourse. Traditional approaches emphasize coherence relationships between individual sentences (Halliday & Hasan, 1976; Zwaan & Radvansky, 1998). Participants can intuitively segment texts into consistent groupings (Gee & Grosjean, 1984; Mandler & Johnson, 1977), suggesting that readers’ comprehension extends beyond adjacent relationships between individual sentences. Some theories have formally described hierarchical relationships in narrative, such as “story grammars” (e.g. Mandler & Johnson, 1977; Rumelhart, 1975; Thorndyke, 1977), which use phrase Structure rules to organize narratives around characters’ goal-driven events. Constituent Structure in these models was examined using clustering models (Gee & Grosjean, 1984; Mandler, 1987; Mandler & Johnson, 1977), similar to those employed by early psycholinguistic research on syntactic relations (e.g. Levelt, 1970). Participants divided a narrative into logical groupings, and these intuitions were then submitted to the algorithms in hierarchic clustering models. These analyses yielded tree Structures that closely correlated with the models predicted by the original theories. As in verbal discourse, theories of visual narrative have also emphasized linear relationships between individual images (McCloud, 1993). However, beyond such linear relationships, participants also intuitively divide visual narratives into segments (Gernsbacher, 1985). Again, this suggests that comprehension may extend beyond adjacent relationships. Recently, Cohn (2003, In Press) has proposed a theoretical model of narrative Structure that formalizes the We have proposed that visual narratives in comics are organized with a hierarchic narrative “grammar.” Inspired by classic “click” studies of syntax, we inserted blank “disruption” panels Before, At, or After the Constituent boundaries of comic strips. In self-paced viewing, Experiment 1 found that blanks After the boundary were viewed slower than Before or At the boundary. Panels immediately following blanks were slower than corresponding panels in sequences without blank panels, but only when placed Before or After the boundary. Three ordinal panel positions following the boundary, panels following blanks At the boundary or with No-Blanks were viewed faster than panels following blanks After or Before the boundary. This supports constituency because disruptions had greater impact within, as opposed to between, Constituents. Using ERPs, Experiment 2 found a larger anterior negativity to blanks within Constituents (Before/After) than between Constituents (At). This indicates disruptions of Constituents are recognized before reaching a subsequent panel. A larger P600 appeared to blanks After the boundary than in the first Constituent (Before/At). This positivity may reflect a reanalysis reflecting the inability to integrate all prior panels into a single Constituent, since they are divided by the Constituent boundary. Keywords: Grammar; Constituent Structure; Discourse; Narrative; Comics; ERPs; Left Anterior Negativity. Introduction Drawings have conveyed narratives through sequences of images for thousands of years, but in contemporary society they appear most prevalently in comics. In comparison with research on the Structure and comprehension of verbal narrative, however, little is known about mechanisms of processing sequential images. For example, are narrative units integrated linearly across each adjacent relationship or they organized into hierarchic Constituents?
-
CogSci - Segmenting Visual Narratives: Evidence for Constituent Structure in Comics.
Cognitive Science, 2012Co-Authors: Neil Cohn, Ray Jackendoff, Phillip J. Holcomb, Gina R. KuperbergAbstract:We have proposed that visual narratives in comics are organized with a hierarchic narrative “grammar.” Inspired by classic “click” studies of syntax, we inserted blank “disruption” panels Before, At, or After the Constituent boundaries of comic strips. In self-paced viewing, Experiment 1 found that blanks After the boundary were viewed slower than Before or At the boundary. Panels immediately following blanks were slower than corresponding panels in sequences without blank panels, but only when placed Before or After the boundary. Three ordinal panel positions following the boundary, panels following blanks At the boundary or with No-Blanks were viewed faster than panels following blanks After or Before the boundary. This supports constituency because disruptions had greater impact within, as opposed to between, Constituents. Using ERPs, Experiment 2 found a larger anterior negativity to blanks within Constituents (Before/After) than between Constituents (At). This indicates disruptions of Constituents are recognized before reaching a subsequent panel. A larger P600 appeared to blanks After the boundary than in the first Constituent (Before/At). This positivity may reflect a reanalysis reflecting the inability to integrate all prior panels into a single Constituent, since they are divided by the Constituent boundary.
Christophe Pallier - One of the best experts on this subject based on the ideXlab platform.
-
Brain correlates of Constituent Structure in sign language comprehension
NeuroImage, 2018Co-Authors: Antonio Moreno, Stanislas Dehaene, Fanny Limousin, Christophe PallierAbstract:During sentence processing, areas of the left superior temporal sulcus, inferior frontal gyrus and left basal ganglia exhibit a systematic increase in brain activity as a function of Constituent size, suggesting their involvement in the computation of syntactic and semantic Structures. Here, we asked whether these areas play a universal role in language and therefore contribute to the processing of non-spoken sign language. Congenitally deaf adults who acquired French sign language as a first language and written French as a second language were scanned while watching sequences of signs in which the size of syntactic Constituents was manipulated. An effect of Constituent size was found in the basal ganglia, including the head of the caudate and the putamen. A smaller effect was also detected in temporal and frontal regions previously shown to be sensitive to Constituent size in written language in hearing French subjects (Pallier et al., 2011). When the deaf participants read sentences versus word lists, the same network of language areas was observed. While reading and sign language processing yielded identical effects of linguistic Structure in the basal ganglia, the effect of Structure was stronger in all cortical language areas for written language relative to sign language. Furthermore, cortical activity was partially modulated by age of acquisition and reading proficiency. Our results stress the important role of the basal ganglia, within the language network, in the representation of the Constituent Structure of language, regardless of the input modality.
-
Cortical representation of the Constituent Structure of sentences
Proceedings of the National Academy of Sciences, 2011Co-Authors: Christophe Pallier, Anne-dominique Devauchelle, Stanislas DehaeneAbstract:Linguistic analyses suggest that sentences are not mere strings of words but possess a hierarchical Structure with Constituents nested inside each other. We used functional magnetic resonance imaging (fMRI) to search for the cerebral mechanisms of this theoretical construct. We hypothesized that the neural assembly that encodes a Constituent grows with its size, which can be approximately indexed by the number of words it encompasses. We therefore searched for brain regions where activation increased parametrically with the size of linguistic Constituents, in response to a visual stream always comprising 12 written words or pseudowords. The results isolated a network of left-hemispheric regions that could be dissociated into two major subsets. Inferior frontal and posterior temporal regions showed Constituent size effects regardless of whether actual content words were present or were replaced by pseudowords (jabberwocky stimuli). This observation suggests that these areas operate autonomously of other language areas and can extract abstract syntactic frames based on function words and morphological information alone. On the other hand, regions in the temporal pole, anterior superior temporal sulcus and temporo-parietal junction showed Constituent size effect only in the presence of lexico-semantic information, suggesting that they may encode semantic Constituents. In several inferior frontal and superior temporal regions, activation was delayed in response to the largest Constituent Structures, suggesting that nested linguistic Structures take increasingly longer time to be computed and that these delays can be measured with fMRI.
Janet F. Mclean - One of the best experts on this subject based on the ideXlab platform.
-
Is children's acquisition of the passive a staged process? Evidence from six- and nine-year-olds' production of passives.
Journal of child language, 2011Co-Authors: Katherine Messenger, Holly P. Branigan, Janet F. McleanAbstract:We report a syntactic priming experiment that examined whether children's acquisition of the passive is a staged process, with acquisition of Constituent Structure preceding acquisition of thematic role mappings. Six-year-olds and nine-year-olds described transitive actions after hearing active and passive prime descriptions involving the same or different thematic roles. Both groups showed a strong tendency to reuse in their own description the syntactic Structure they had just heard, including well-formed passives after passive primes, irrespective of whether thematic roles were repeated between prime and target. However, following passive primes, six-year-olds but not nine-year-olds also produced reversed passives, with well-formed Constituent Structure but incorrect thematic role mappings. These results suggest that by six, children have mastered the Constituent Structure of the passive; however, they have not yet mastered the non-canonical thematic role mapping. By nine, children have mastered both the syntactic and thematic dimensions of this Structure.
-
Constituent Structure is formulated in one stage.
Journal of Memory and Language, 2002Co-Authors: Martin J. Pickering, Holly P. Branigan, Janet F. McleanAbstract:Abstract In four syntactic priming experiments, participants completed target fragments as “prepositional object” sentences (e.g., “The patient showed his leg to the doctor”) or “double object” sentences (e.g., “The patient showed the doctor his leg”) or used another non-ditransitive form. The syntactic form of a prime sentence affected the form of participants' target completions. Experiments 1 to 3 used written sentence completion. Experiment 1 demonstrated that priming is a two-way process by comparing “prepositional object” and “double object” priming conditions with a baseline condition containing an intransitive verb. Experiments 2 and 3 found that “shifted” primes (e.g., “The racing driver showed to the helpful mechanic the problem with the car”) did not prime the production of “prepositional object” sentences but instead behaved like baseline primes. Experiment 4 found similar results to those of Experiment 3 in spoken sentence production, where participants repeated the prime and then completed it. We interpret the results in terms of accounts that assume that Constituent Structure is formulated in one stage.
Simon Corstonoliver - One of the best experts on this subject based on the ideXlab platform.
-
linguistically informed statistical models of Constituent Structure for ordering in sentence realization
International Conference on Computational Linguistics, 2004Co-Authors: Eric K. Ringger, Martine Smets, David Rojas, Robert C. Moore, Michael Gamon, Simon CorstonoliverAbstract:We present several statistical models of syntactic Constituent order for sentence realization. We compare several models, including simple joint models inspired by existing statistical parsing models, and several novel conditional models. The conditional models leverage a large set of linguistic features without manual feature selection. We apply and evaluate the models in sentence realization for French and German and find that a particular conditional model outperforms all others. We employ a version of that model in an evaluation on unordered trees from the Penn TreeBank. We offer this result on standard data as a reference-point for evaluations of ordering in sentence realization.