[ CogSci Summaries home | UP | email ]
http://www.jimdavies.org/summaries/

Hinricks, T.R., & Forbus, K.D.(2011). Transfer learning through analogy in games. AI Magazine, 32(1), 70-83.
@Article{HinricksForbus2011,
  author = 	 {Hinricks, Thomas R., and Forbus, Kenneth D.},
  title = 	 {Transfer learning through analogy in games},
  journal =  	 {AI Magazine},
  year = 	 {2011},
  volume = 	 {32},
  number = 	 {1},
  pages = 	 {70-83}
}
Author of the summary: Michelle Sappong, 2011, naana.sappong@gmail.com

Cite this paper for: Transfer: applying previous knowledge in a new situation/scenario "domain"

Near transfer: when the domains are similar to each other, knowledge can be transferred exactly
Far transfer: when domains are different, transfer requires abstraction

Analogical transfer is the transfer of learned knowledge from the source domain to a target domain through the use of analogy [p.71]. Analogies are used to transfer knowledge by translating symbolic representations from the source to the target.

Problem: finding a good mapping between a source and target with different surface representations

Structure mapping: process of analogy that specifies which statements map onto each other. Statements can only be mapped if their arguments align [p. 77]
“Minimal ascension” and “metamapping” enable structure mapping between domains with different vocabularies (i.e. non-identical predicates) [p. 70]

Near Transfer Experiment [p. 71-72]

Freeciv game: goal is to grow a civilization that could become the superpower of the world.
Transfer task: To grow one city through trial and error, and then use that knowledge (via a direct transfer algorithm) to grow another city faster.
Performance was measured as total food production after 50 turns
Results: Near transfer improved performance on the initial game, but the effect was not robust [p.72]
Implications : analogical mapping can be used to do transfer learning

Far Transfer Experiments

Three families of games were used: Escape, Wargame, Micro-Rogue. They are similar, but differ in the aspects of gameplay that they emphasize. [p.73]

Hinricks & Forbus (2011) use the “Structure Mapping Engine” [p.71] to implement far transfer

The output of learning is a set of strategies for winning the game organized in a “hierarchical task network (HTN)” [p.73]

Learning the Source Game:

Static Analysis In retrospect, goal analysis could be improved by "considering alternative heuristics for progress" [p.75]
The purpose of static analysis is take background knowledge of games in general and apply it to extract abstract structural representations that will be effective for analogical mapping

Experimentation

The learner experiments (bottom-up) to satisfy learning goals.
Primary learning goals: Example of learning by experiment: In Micro-Rogue, go to a snake, conseuqently find out what it does (it kills you), read scroll to see what effect that had on the game (health levels, etc.). [p. 76]

Dynamic Anaylsis Learned HTN's must be flexible - not too sensitive to specific initial conditions of source game, and not too detailed (they need to adapt to changing conditions as game is played)

How is flexibility achieved?
1. some primitives (leaves of HTN) which are run first invoke “doPlan operator”, so that after each step replanning occurs to accommodate "unforeseen effects of actions and adversarial responses" [p.76-77]
2. specific action sequences are turned into hierarchical network of subtasks. This way, each task is a hierarchy, and if some action method doesn't transfer properly, it alone can be rejected without tossing the entire task
3. "lifting" [p.77]: replacing constants with variables (e.g. spec coordinates replaced with descriptions "exit") so that they can be analogically mapped to target entities

Learning the Target Game: Non-identical predicate mapping

How are non-identical predicates optimally matched?
Rather than explore the entire space of non-identical predicate matches, search is constrained to find overlaps in higher-order predicates [p.77]

What is the rationale behind mapping non-identical higher-order predicates?
Structure mapping theory states that statements only map if arguments align. So if args (primitives) align, higher-order structures must align. Also, higher-order predicates (connectives, argument structure) tend not to vary across domains - example: relationship between precondition, task, and consequence is the same across all games

2 techniques for non-identical predicate matching: minimal ascension and metamapping

Minimal ascension: Metamapping: Which analogy technique is better?
Minimal ascension (MA) accuracy is better than Metamapping (MM) accuracy because it uses structure info from game as well as derived structural info, whereas MM only uses derived info.
MA is more accurate but MM has better coverage
When mappings conflict, system prefers MA [p.79]

Method:
Each target game was learned twice (once with transfer, once without). Improvement was measured by “regret score” [p.80]. The higher the score, the higher the performance achieved in fewer trials. There were 5 different conditions based on the relationships between the source and target games used. [p.81]

Results:
Average regret across all scenarios was 27.38 [p.81]

Implications:
When HTN’s are successfully transferred from one game to another, that game is learned faster and with less trial and error. Therefore, metamapping can be successfully used to perform analogical transfer between unrelated domains

Problems with experiment:
1. difficulty of the source game is possible confound (some games were very difficult to learn - only 1 of 8 source games was mastered in the D condition) [p. 81]
2. cross-mapping of predicates between source and target [p. 81]

In future, try:
  • Broader range of domain type
  • Different domain learning techniques
  • Repairing mappings that don’t work, rather than discarding them
  • Analogical mapping where the AI must first choose a relevant source from a range of possible sources [p. 82]

    Summary author's notes:
    Back to the Cognitive Science Summaries homepage
    Cognitive Science Summaries Webmaster:
    JimDavies (jim@jimdavies.org)