Tufts University  |  School of Arts and Sciences  |  Find People  | 

Events: Conferences

Building Meaning From Language
Tufts University Initiative on Emerging Trends in Psychology
June 14 - June 19, 2007
Tufts University, Medford, MA


What are words?
Daniel C. Dennett

Curiously enough, philosophers who devote their careers to 'linguistic philosophy' and 'analytic metaphysics' often take words for granted, as if they were just as unproblematic bits of the world's furniture as tables and chairs, raindrops and sunrises. Yet when asked whether words are "in their ontology" some philosophers find themselves swallowing hard and saying that, strictly speaking, there are no such things as words! If words exist (and I am sure they do!), what, exactly, are they? And how did there come to be so many of them?

Morton Ann Gernsbacher
Fine Tuning for Meaning: Behavioral and Neural Imaging Experiments

Over fifteen years ago, we began asking research participants to read a sentence and then judge quickly whether a test word was related to the overall meaning of the sentence that they had just read. Some test words were related to the final word of the sentence but unrelated to the sentence's overall meaning, for example, the test word "ace" and the sentence,  "He dug with the spade." When the test words were presented immediately after participants read the sentences, participants were slower to correctly reject those test words as not being related to the sentence than they were to correctly reject test words that were completely unrelated to the sentences. Thus, participants experienced interference. However, when the test words were presented after a brief delay, this interference was attenuated. We interpreted this pattern of immediate interference followed by attenuation as manifesting the action of a cognitive mechanism of suppression. We have explored the basis of this suppression mechanism behaviorally with numerous participant groups (e.g., less-skilled readers, elderly readers, readers with small working memory spans), and we have used event-related functional magnetic brain imaging to identify its neural basis. These data will be presented to support the argument that fine tuning for meaning requires attenuating inappropriate information (i.e., suppression) as well as activating appropriate information.

Premotor cortex, action control, and language.
Arthur Glenberg

To effectively control action, the brain has evolved to solve a number of thorny problems: Learning complex action sequences with hierarchical structure, exquisite timing of movements (e.g., in tennis, piano playing, walking) when sensory feedback may be too slow to help, and determining just what information in the sensory array might be useful. Interestingly, similar problems arise in learning and using language. Might the brain use mechanisms of action control to learn, produce, and comprehend language? Recent findings of mirror neurons tuned for action and speech recognition in premotor cortex (Broca's area in particular) suggest a positive answer. In this talk, I will illustrate how a formal theory of action control, Wolpert's HMOSAIC model, can be modified to account for basic facts in language. Then, I will discuss the results of several projects testing theoretically derived claims regarding language acquisition, how manipulating the motor system affects language comprehension, and how manipulating language comprehension affects the motor system.

Building Linguistic Meaning
Ray Jackendoff

What does a theory of sentence and discourse meaning have to account for?

  • Word and construction meanings are stored in long-term memory, in association with phonological and syntactic information.  Word and construction meanings can contain variables that stipulate combinatorial potential.
  • Sentence and discourse meanings are built up online in working memory, in part by instantiating variables in word meanings.
  • Because of the structured nature of semantic combinatoriality, semantic working memory cannot consist simply of the parts of longterm memory that are activated.
  • Word, sentence, and discourse meanings involve the interaction of at least two kinds of combinatorial structures:  a quasi-algebraic Conceptual Structure, which encodes categorial and function-argument information, and a quasi-geometric/ topological Spatial Structure, which encodes details of shape and spatial configuration.
  • Conceptual Structure itself is organized into a number of discrete but interacting tiers, including propositional (function-argument) structure, referential structure, and information (topic/focus) structure.
  • Both Conceptual Structure and Spatial Structure interact not only with language but also with perception and action.  This is what allows us to talk about what we see and act in response to instructions.
  • Rules of inference (including heuristics) are defined over Conceptual Structure and Spatial Structure.
  • Word meanings have internal combinatorial structure that enables them to trigger patterns of inference.
  • Word meanings are not just sets of necessary and sufficient conditions.  They also involve various sorts of violable constraints that show up in nonstereotypical situations.
  • Building sentence and discourse meaning from word meanings involves more than just combining word meanings.  Many aspects of meaning (often called pragmatic) are not represented by any words in the sentence but arise out of the necessity to combine word meanings in semantically well-formed and/or situationally appropriate fashion.

Two levels of verb meaning: Neuroimaging and neuropsychological evidence
David Kemmerer

For over 20 years, research in linguistics has supported the existence of two levels of verb meaning. The first level consists of an austere representation, sometimes called the "event structure template," that (a) is common to all the verbs in a given class (e.g., "manner of motion" verbs), (b) is composed primarily of simple predicates and variables for arguments, and (c) strongly constrains the range of morphological and syntactic constructions that are possible. The second level reflects the uniqueness of every verb and has been dubbed the "constant" because it captures idiosyncratic semantic features that (a) distinguish each verb in a given class from all the others (e.g., stroll vs. strut vs. stagger), (b) are often modality-specific in format, and (c) are grammatically irrelevant. I present evidence from two neuroscientific approaches -- specifically, functional neuroimaging studies with normal subjects, and neuropsychological studies with brain-damaged subjects -- that begin to reveal how these two levels of verb meaning are implemented in the brain. This research suggests that "event structure templates" depend on cortical structures in the classic left perisylvian language system, whereas "constants" depend on cortical structures in anatomically distributed sensorimotor systems, including regions involved in vision and action.

Statistical Semantics
Walter Kintsch

Statistical semantics attempts to infer semantic knowledge from the analysis of linguistic corpora.  For example, Latent Semantic Analysis (Landauer & Dumais, 1997; Landauer et al., 2007) constructs a high-dimensional map of meaning that allows the ready computation of similarities between word meanings as well as text meanings. I briefly describe LSA as well as several related methods and then focus on two limitations of such systems.

Typically, semantic representations are generated from data that consist only of word co-occurrences in documents, neglecting information about word order, syntax, as well as discourse structure. Ways to include word order as well as syntactic information in the construction of corpus-based semantic representations are described. Specifically, dependency grammar will be used to guide the construction of semantic representations and comparisons.

Secondly, statistical semantics is based solely upon verbal information, whereas human semantics integrates perception and action with the symbolic aspects of meaning. A map of meaning that considers only its verbal basis can nevertheless be useful, in that language mirrors real world phenomena. Furthermore, it is argued that meaning, while clearly based on perception and action, transcends this basis and includes a symbolic level, which we attempt to model by statistical semantics.

The Neural Basis of Comprehension: Temporo-Spatial evidence from Event-related Potentials and functional Magnetic Resonance Imaging
Gina Kuperberg, MD PhD

ERP and fMRI findings converge to suggest that, within simple, active English sentences, semantic violations between verbs (denoting actions) and their subjects (denoting Agents carrying out these actions) evoke a neural response that is more similar to that evoked by morphosyntactic violations between verbs and their arguments, than to that evoked by violations arising only at the level of our real world semantic knowledge. On the basis of these data, I will suggest that normal language comprehension proceeds along at least two dissociable but highly interactive neural processing streams: an associative semantic memory-based mechanism that is based mainly on accessing the frequency of co-occurrence of words or events, as stored within semantic memory, and a combinatorial mechanism in which structure is assigned to a sentence not only on the basis of morphosyntactic rules, but also on the basis of certain action-relevant (thematic) semantic constraints.

Based on ERP data, I will suggest that the semantic memory-based analysis operates as a first-pass mechanism, primarily between 300-500msec, and that a morphosyntactic and thematic-semantic combinatorial analysis around a verb begins within this time window, at least partially in parallel to semantic memory-based processing. Any conflicts between the different representations that are output by the semantic memory-based and combinatorial streams lead to continued or second-pass combinatorial analysis, operating between 500-900msec. This may serve as a double check to ensure that we effectively make sense of incoming information.

Based on fMRI data, I will suggest that the semantic memory-based analysis is reliant on activity within the left anterior inferior frontal cortex that, together with temporal cortices, acts to retrieve information about the likelihood of events in the real world. In contrast, both morphosyntactic and thematic-semantic combinatorial analyses around a verb appear to engage a common frontal/inferior parietal/basal ganglia network, known to mediate the execution and comprehension of goal-directed action.

Finally, based on both ERP and fMRI studies examining visual actions depicted within short, silent movie-clips, I will suggest that that these two processing streams may generalize beyond the language system and may also be engaged in relating people, objects and action during real-world event comprehension. I will conclude by briefly considering the implications of this model of language and real-world visual comprehension for understanding neurocognitive basis of neuropsychiatric disorders such as schizophrenia.

Meaning and structure: Influences of real-world events on language comprehension
Ken McRae

A significant proportion of everyday utterances concern real-world events. Thus, people's knowledge of everyday events, including their common participants, is an important component of sentence comprehension. Our original research on this topic focused on verb- specific thematic role conceptual knowledge as an important basis for expectancy generation in language with respect to both upcoming fillers of thematic roles, and upcoming structure. That is, as is common in the literature, we considered structural and semantic expectancy generation in sentence processing as being driven primarily by the verb in the aggregate. However, it has become apparent that the empirical phenomena demand a richer, less verb- centered approach in three ways. First, rather than being verb- specific, the evidence demands an event-specific explanation. This is particularly pertinent to situations in which verbs have multiple senses and thus can refer to multiple classes of real-world events, which is the case with many verbs, at least in English. Second, other sentential elements can influence event-based expectations. We have focused primarily on various types of thematic role fillers (i.e., agents, patients, instruments). Third, extra-sentential context can bias language comprehenders to a range of event spaces. I will present experimental results that provide evidence for this richer event-based view of language comprehension.

Polysemy and Coercion
James Pustejovsky

Recently, there has emerged a new appreciation of the complexity at play in the interpretation of polysemy. Two classes of parameters have been broadly identified as contributing to the interpretation of polysemous expressions: more complex lexical representations, and a means of incorporating local context compositionally. In this talk, I formalize this distinction as that of inherent versus selectional polysemy, and demonstrate that polysemy cannot be modeled adequately without enriching the compositional mechanisms available to the language. In particular, lexically driven operations of coercion and type selection provide for contextualized interpretations of expressions, which would otherwise not exhibit polysemy. I contrast this with the view that it is not possible to maintain a distinction between semantic and pragmatic ambiguity. I will argue that a strong distinction between pragmatic and semantic modes of interpretation can be maintained, and is in fact desirable, if we wish to model the complexity of contributing factors in compositionality in language.

Visual World Studies of Language Processing
Michael Tanenhaus

In the Visual World Paradigm (VWP), participants' eye movements are measured as they follow instructions to perform actions in a circumscribed visual world. This approach allows investigators to examine how language is interpreted in the context of perception and context-specific goal-directed action, and how language, vision and action interact. I'll review the logic of the VWP, including how it combines the 'language-as-product' and 'language-as-action' traditions, focusing on the effects of action-specific affordances, intentions and interlocutors' joint goals on real-time syntactic processing, reference-resolution and spoken word recognition. I'll then review in-progress work with Kate Pirog and Dick Aslin that uses artificial languages with the VWP with fMRI to examine activation of motion-sensitive areas in V5 during spoken word recognition.

Meaning, Argument Structure, and Parsing: Building Meaning from Language Using Lexically Stored Syntactic Representations
Matthew Traxler

A long-running debate in psycholinguistics pits autonomous syntax against lexically-driven structure-building processes. According to autonomous syntax accounts (e.g., Chomsky, 1965; Frazier, 1979, 1987; Pinker, 1997), syntactic structures are built on the basis of abstract, word-category representations by a mechanism that operates independently of other levels of representation. Lexicalist accounts (e.g., Boland & Boehm-Jernigan, 1998; MacDonald et al., 1994; Trueswell et al., 1993; Vosse & Kempen, 2000) suggest instead that elements of syntactic structure are tied to individual entries in the mental lexicon. Determining how words in sentences relate to one another (i.e., parsing the sentence) starts with accessing individual word representations, activating argument-structures and syntactic frames associated with those representations, and applying multiple sources of constraint to choose one structure from among competing alternatives. Syntactic priming experiments can be used to test these theories.

Syntactic priming occurs when a prime sentence affects processing of a subsequent target sentence because the two sentences share elements of syntactic structure. According to autonomous syntax accounts, priming should occur whether two sentences have overlapping words or not, as long as the two sentences have the same syntactic structure. Lexicalist accounts predict that priming effects should be larger when specific words are repeated across the prime and target sentences. A series of eye-tracking and ERP experiments have established the following:

  1. Priming occurs in comprehension.
  2. Priming occurs because of facilitated syntactic processes (rather than facilitated semantic processes).
  3. It depends on overlapping lexical material in sentences involving argument relations.
  4. It occurs independent of lexical overlap in sentences involving adjunct relations.
  5. It does not depend on readers predicting the upcoming structure.

The overall pattern of results in comprehension is most consistent with the argument structure hypothesis (Boland & Boehm-Jernigan, 1998; Boland & Blodgett, 2006) and lexically mediated parsing (Traxler & Tooley, 2007).

Comprehending with Language
Rolf Zwaan

Language comprehension has long been understood as the comprehension of language--first as the recovery of the syntactic and semantic structure of the linguistic input, and later as the construction of a situation model based on the linguistic input and background knowledge. I will argue that language comprehension is better understood as comprehension with language. That is, language comprehension is a special form of event and action comprehension. There are similarities between how we understand an action that we observe (e.g., someone pouring himself a cup of coffee) and an action we hear or read about (e.g., He poured himself a cup of coffee). Specifically, there is overlap in the brain systems that are involved (e.g., the area of premotor cortex that controls movement of the right hand). In both cases, comprehension appears to involve a mental simulation of the actions and events. However, comprehension with language is special because the mental simulation is not modulated directly by the observed actions and events, but indirectly via language. Thus, the key to developing a theory of language comprehension is to examine how language modulates mental simulations of actions and events. I will discuss recent empirical findings that speak to this issue.