Short term picture comprehension and memory

An RSVP sequence of pictures

When told what to look for, viewers can detect a picture of "a picnic" or "a woman on the phone" within a rapid sequence of pictures, although afterwards they cannot remember most of the pictures they have just seen. In recent work we have found, however, that memory for rapidly presented pictures does persist briefly, consistent with the idea that there is a conceptual short-term memory (CSTM) for meaningful material. A question we are currently studying is whether this brief picture memory is primarily visual or primarily conceptual -- do we just remember that we saw "a picnic," or do we remember the colors and spatial layout of the picture as well? In other studies we have found that people can detect a target picture within rapid sequences where up to four pictures are presented on a frame, but they consolidate memory for pictures presented in such sequences more slowly, one at a time. [Click here for related publications]

An RSVP sequence with multiple pictures per frame

Attention in search tasks: The Attentional Blink

When the viewer is looking for two targets (such as blue letters) in a sequence of non-targets (such as black numbers) presented at a rate of 10/s, detection of the second target is markedly impaired when it arrives 200-500 ms after the first target, an "attentional blink." Mark Nieuwenstein (a postdoc in the lab) has shown that cuing the onset of the second target (for example, by showing a blue number immediately before a blue letter target) reduces the impairment caused by the attentional blink, overcoming the stickiness of attention to the first target. Ongoing work investigates the nature of this cuing effect. For example, Mark has found that the cuing effect is contingent on attentional set: when the second target can be presented in either red or green, a cue in either color will help the second target, but when the target is always presented in green, only green cues produce a substantial improvement. [Click here for related publications]

Cuing and the Attentional Blink

Detecting words in two streams at high rates

We have also done work with word targets presented among number and symbol strings, extending the attentional blink procedure described above to rates of 19 items/s and using two simultaneously presented sequences of stimuli. Interestingly, at very short intervals between targets (such as 53 ms), the second target is actually more likely to be seen than the first target. At longer intervals, we obtain the familiar attentional blink pattern in which the first target is more likely to be seen. In further studies with this task we have found substantial effects of semantic priming. [Click here for related publications]

The Attentional Blink with "crossover" effect (F = first target, S = second target)

Comprehension of an object and its background

Jodi Davenport, a graduate student in the lab, studies the interaction between a foreground object and its background, in colored photographs presented very briefly and masked. When the background and object are mutually consistent (e.g., a football player on a football field), both the report of the object and the report of the background are more accurate, as compared to when they are inconsistent (e.g., a football player in a church). Ongoing work examines scenes with multiple foreground objects. [Click here for related publications]

Examples of consistent scenes (a), inconsistent scenes (b), and isolated objects and backgrounds (c); from Davenport & Potter, 2004

Conceptual short term memory

The lab's work also encompasses other questions about perception, attention, memory, and language processing, including repetition blindness, the creative misperception of a nonword as influenced by semantic context, cross-modal (visual-auditory) processing of sentences, and the conceptual basis of "verbatim" recall of sentences. For example, the lab has shown that a 14-word sentence can be understood and remembered when presented as rapidly as 12 words per second. In contrast, a sequence of unrelated words (even if no more than 4 or 5 words long) is much more difficult to process and few words are remembered.

These studies contribute to our understanding of how a stimulus such as a word, sentence, or picture quickly generates an interpretation and a fleeting or stable memory. They flesh out the evidence that, early in processing, multiple associations are briefly activated and used selectively to generate a stable interpretation of ongoing experience. There is rapid forgetting of irrelevant associations and of stimuli that do not fit the best interpretation (the viewer may not even become conscious of a stimulus unless it fits with the interpretation). We have termed this process conceptual short term memory (CSTM). [Click here for related publications]