[ CogSci Summaries home | UP | email ]

Barsalou, L.W. (1999). Perceptual symbol systems. Behavioral and Brain Sciences, 22, 577-609.

  author = 	 {Lawrence W. Barsalou},
  title = 	 {Perceptual symbol Systems},
  journal = 	 {Behavioral and Brain Sciences},
  year = 	 {1999},
  volume = 	 {22},
  pages = 	 {577--609},

Author of the summary: Jim Davies, 2000, jim@jimdavies.org

Cite this paper for:

The actual paper can be found at http://www.psychology.emory.edu/cognition/barsalou/papers/Barsalou_BBS_1999_perceptual_symbol_systems.pdf

amodal: nonperceptual [p3]

What gets stored in long term memory? A subset of the neural representation of the physical input. The symbols are modal, meaning that they are represented in the same perceptual systems that gave rise to them. They are analog (as opposed to discrete) because they are modal.

Perceptual symbols are distinguished from feature lists. [p4]
Amodal symbols have no "correspondence" with the perceptual states that produced them.
"The amodal symbols that represent the colors of objects in their absence reside in a different neural system from the representations of these colors during perception itself." They are arbitrarily related to the perceptual states, like the word "chair" is arbitrarily related to an actual chair.

Even some connectionist models have amodal symbols: The hidden units are interpreted as a conceptual system that is arbitrarily connected to the input units, which are the actual perceptual state.[p5]

Shortcomings of the amodal view Glaser 1992 and Seifert 1997: conprehensive review that shows that concepts have a perceptual character.
Damasio 1989: Categorical knowledge is not amodal. Damage to the sensori-motot system inhibits conceptual processing of visual categories (e.g. birds)

Amodal symbol systems that try to represent spatio-temporal knowledge are bittle, cumbersome, and intractable (Clark 1997, Glasgow 1993, McDermott 1987, Winograd & Flores 1987).

There is no good theory of transduction from percepts to symbols.
No neural evidence that this transduction process exists in the brain, or how to get the perceptual states from the symbol. [p6]

What about the connectionist example? What about a pointer for what something looks like pointing to the detectors that make it up? Barsalou says that the perceptions are doing all the work, so the amodal symbol is redundant.

Amodal symbols are too powerful. They can explain anything post hoc.
Kosslyn argues that there are amodal symbols that can create mental imagery in temporary working memory. This distinguishes Kosslyn and Barsalou. [p7] This article should show that perceptual symbol systems can: [p8]

  1. represent types and tokens
  2. combine symbols productively to produce new conceptual structures (ball + snow = snowball)
  3. bind types to tokens for propositions
  4. represent abstract concepts
Perceptual symbols are interpreted-- not just recorded from life.

Perceptual symbols are not mental images. [p9] Perceptual symbols are records of the neural states that underlie perception.
"The brain uses active configurations of neurons to represent the properties of perceived entities and events."
"Although mechanisms outside sensory-motor systems enter into conceptual knowledge, perceptual symbols always remain grounded in these systems." [p10] The core content of perceptual symbols are unconscious neural representations, not mental imagery.

Perceptual reasoning need not be conscious. What the mind focuses on deterimines which percepts get encoded as the perceptual symbol.

"...the symbol formation process selects and stores a subset of the active neurons in a perceptual state." [p11] A perceptual symbol is an attractor in a neural network. (see summary author notes)

A possible objection is that you can't have certain kinds of imagistic abstractions, like a triangle without orientation. But this applies only to conscious images.

[p14] "the frame for car contains extensive multimodal information of what it is like to experience this type of thing."

Simulators use perceptual symbols to simulate the environment. There are rules that govern the behavior.[p15] Simulators include 2 levels of structure.

[p16] Categorization: "If the sumulaor for a category can produce a satisfactory simulation of a perceived entity, the entity belongs in the category.

Simulators contain a lot of multimodal information.

Your ability to simulate doing something with an object is its affordance. [p17]

There are optical illusions for which top-down processing will not effect it.[p18]

Sometimes psychotics will have bottom-level processing effected by top-down, though. The `filling in' of stimuli of many modalities is converging evidence for the closeness of concept/percept relationship. [p19]

Frame: "an integrated system of perceptual symbols that is used to construct specific simulations of a category. [p20]

Figure 3 shows a diagram of a perceptual symbol of a car. Barsalou 1992: Frames are constituted with 4 basic properties: [p21]

  1. Predicates.
    Unspecialized frames. Figure 3 A is roughly equivalent to CAR(door=x, window=y, ...)
  2. Attribute-value bindings.
    This happens when specializations occur. The door percept is abstract, the attribute, and the individual door instances are the values.
  3. constraints.
    Allow you to recall instances. The second car you saw has links to the percepts involved. If this is not strengthened, errors in recall can occur.
  4. recursion.
    sort of like object-oriented inheritence. Not recursion in the functional sense.
Productivity: "the ability to construct an unlimited number of complex representations from a finite number of symbols using combinatorial and recursive mechanisms." [p23]

Perceptual symbol systems can be productive because they are made of constituent percept parts that can be re-combined. How do perceptual symbols implement propositions? [p26]

"It is fair to say that this ability is not usually recognized as possible
in perceptual theories of knowledge, again because they are widely
construed as recording systems. Indeed, this belief is so widespread that
the term "propositional" is reserved solely for nonperceptual theories of
knowledge. As we shall see, however, if one adopts the core properties of
perceptual symbol systems, the important properties of propositions follow
naturally.  Because perceptual symbol systems have the same potential to
implement propositions, they too are propositional systems."

Looking at his examples, you can see why. As an example of an amodal 
proposition, on page 26 he offers:

 ABOVE(celing, floor)

and eariler (figure 4) he gives the following type diagram

above- A
below- B

Where there are three perceptual symbols on the left (top to bottom, they
are A, the above/below relation, and B).  

As you can see, this is no different, really, from symbols with labeled
relations, which can just be written as:

Figure 5a [p26] shows an airplane in a thin border. This represents a multimodal simulator for a jet. The one with the dotted line represents the simulation that matches the jet in the scene.

New propositions are new frames that link to the necessary symbols and relations.

To represent emotions, the behaviors associated with those emotions may be activated directly. [p32]

How can perceptual symbols represent abstract concepts directly? Three mechanisms are central to the representation:

  1. It's background is a simulated event sequence
  2. Selective attention picks out the important parts of the events
  3. perceptual symbols of introspective states. (like emotions, etc.)
[p37] "...working memory is the system that runs perceptual simulations."

[p41] "Modern digital computers are amodal symbol systems. They capture external input using one set of sensing mechanisms (e.g., keyboards, mice) and map it to a different set of representational mechanisms (e.g., binary strings in memory devices). As a result, arbitrary strings come to stand for the states of input devices."
To get around this the computer should save the state of the peripheral. Memories should be able to activate states of the peripheral.

Functionalism is wrong because computer peripherals are different enough from biological sense organs. Perceptual symbols in computers would look much different.

Summary author's notes:

Back to the Cognitive Science Summaries homepage
Cognitive Science Summaries Webmaster:
JimDavies (jim@jimdavies.org)