Johns Hopkins University – Homewood Campus – (410-516-5250/office phone)
Thursday, March 16, 2006
3:30 p.m.
Saarland University
“Adaptive Human Language Processing”
Theories of sentence comprehension have traditionally been based on evidence from reading studies, and have emphasized cognitive limitations and invariant parsing strategies. There is increasing evidence, however, that human language comprehension is highly adapted both to long-term experience and to immediate linguistic and non-linguistic context. In this talk, I will present a collection of recent experiments focusing on adaptation to immediate context in situated language comprehension. Using the "visual world" paradigm, which monitors eye movements in scenes during spoken comprehension, we have demonstrated the ability of the human language processor to rapidly adapt to and exploit diverse linguistic cues, prosody, as well as information from the immediate scene. These findings have led to the proposal of the "coordinated interplay account" of scene and utterance processing (Knoeferle and Crocker, Cognitive Science, to appear), which argues for the rapid use, and even priority, of scene information during spoken comprehension. In order to model the rapid use of both experience and immediate context, we have developed a family of connectionist models based on Elmans Simple Recurrent Networks. These networks have been modified to include inputs not only for the utterance, but also for the current (visual) context, and have been trained to model the findings of five experiments. In addition to the general ability of the networks to successfully learn the task, they also model on-line behavior: (a) anticipation of role and filler, (b) frequency biases derived from training, (c) early influence of depicted events, (d) delayed disambiguation without the scene, and (e) relative priority of scene information (Mayberry, Crocker & Knoeferle, IJCNLP, 2005). In conclusion, I argue that the mechanisms that underlie language comprehension are fundamentally adaptive in nature: optimized to recover the most likely interpretation based on relevant linguistic and non-linguistic information sources. I further suggest that theories and models of human language comprehension must be expanded to explain on-line interaction with other cognitive and perceptual processes.
Visit our website: http://www.cogsci.jhu.edu (menu header: Colloquia)