This dissertation is based on the general idea that visual representations provide a level of abstraction at which two otherwise dissimilar domains may be more alike. There are many theories that also resolve symbolic mismatches by finding similarities at a higher level of abstraction. For example, in conceptual dependency theory (Schank 1972), verbs are categorized into ACTs, which are abstractions of actions. Bhatta and Goel's Generic Teleological Mechanisms (1997) cover different instantiations of mechanisms that perform the same function. Falkenhainer's Minimal Ascension (1988) rule uses a generalization hierarchy to determine the distance between concepts. This dissertation is also research of this kind. I will show that visual abstraction too is a useful mechanism for analogical problem solving.
The first problem is this: under what conditions are visual analogies useful? This dissertation will argue that one of the ways visual representations are useful is in resolving symbolic mismatches in analogical problem solving. Symbolic mismatches encountered in non-visual representations are resolved by providing a level of visual abstraction at which two different symbols are similar.
With respect to the second problem, that of representing visual information for computational purposes, this research introduces Covlan, a language of visual primitives, and will show that it is effective for representing problem-solving episodes involving physical systems, to facilitate analogy. It will consist not only of a language of visual primitives, but also rules for turning non-visual representations into visual ones.
The scope of this work is problems involving physical systems, which I define as those whose solutions involve physical changes to the state of the system in question. For example, solving a problem by changing ownership of something, or by changing a power relationship among people is not a physical change, but physically moving someone's chair so that they sit somewhere else is. System in this work refers to all the physical things associated with a problem situation, the relationships between them, or the spatial representation of those things.
This scope was chosen because it was large enough to be interesting but small enough such that my theory could generalize within it. This is not to say that visual representations are not useful for problem solving in non-physical systems, just that this work will not show evidence for it.
To summarize, at the highest level my hypotheses are:
To make comparisons between visual and non-visual representations, it is important to be precise about what I mean by ``visual.'' I will use the terms ``visual representation'' and ``visual knowledge'' interchangeably, and by them I mean any representation that encodes visual information. Visual information are the visual properties of something, and visual knowledge is visual information encoded for use by an intelligent agent with some representation language. Specifically, this dissertation deals with the following kinds of visual information: shapes, their sizes, locations, motions, and spatial relationships between shapes (e.g. connections, overlaps).
This dissertation will use symbolic descriptive representations, which are structured descriptions of visual information. This is differentiated from depictive representations, or bitmaps. A depictive representation ``specifies the locations and values of points in space'' (Kosslyn, 1994, p.5). There is widespread agreement that visual reasoning, particularly in problem solving and analogy, is a symbolic process. Not surprisingly, all previous computational visual analogy programs also use symbols to represent visual information (Thagard et al., 1992; McGraw et al., 1993; Ferguson, 1994; Croft & Thagard, forthcoming). A more detailed discussion of visual representations will follow in the discussion section.