[ CogSci Summaries home | UP | email ]

Pynadath D. V. & Marsella, S. C. (2005). PsychSim: Modeling Theory of Mind with Decision-Theoretic Agents. Proceedings of the International Joint Conference on Artificial Intelligence. 1181--1186.

  author =				        {Pynadath, David
  V. and  Marsella, Stacy C.},
  title =	     {PsychSim: Modeling Theory of Mind with
  Decision-Theoretic Agents},
  booktitle =    {Proceedings of the International Joint
Conference on Artificial Intelligence},       
  crossref =     {},          
  key =          {},       
  pages =        {1181--1186},       
  year =         {2005},       
  editor =       {},       
  volume =       {},       
  number =       {},       
  series =       {},       
  address =      {},       
  month =        {},       
  organization = {},       
  publisher =    {},          
  note =         {},       
  annote =       {} 


Author of the summary: Ehssan Taghavi, 2006, 2est@qlink.queensu.ca

Cite this paper for:


Computational modeling of human social behaviour is an important area of research.
Our beliefs about others (theory of mind) play an important role within the context of human social interactions and therefore it should be taken into account when modeling human social behaviour.
Most of the previous computer-simulated models of the theory of mind have used first-order logic to represent beliefs and goals. However, such models have some shortcomings.
On the other hand, PsychSim is one model that generates more plausibly human behaviour.
PsychSim enables a quick setup of social scenarios within which agents (representing individuals) can easily interact with each other.

These PsychSim agents have their own beliefs about other agents and their environments.
They also have goals and policies for achieving those goals.
The PsychSim agents are embedded within a decision-theoretic framework.
Decision-theoretic frameworks are based on assumptions of rationality that people constantly violate. [1]
This framework is an extension to the Com-MTDP model of agent teamwork.[2]

Each agent starts with a representation of its current state and the Markovial process by which that state evolves over time in response to the actions that are performed to change the world:
-State: Includes hidden or obvious objective facts about the the agent's world.
Within PsychSim, a vector is used to represent a state
-Action: Includes an action type, an agent performing the action , and another agent that's the object of the action. An agent has the option of choosing one action from the actions available.
-World: Each action that is performed by the agent changes the state of the world.
Probability functions are used to represent the dynamics of the world state.

An agent also has motivation for behaviour as a reward function within the decision-theoretic framework. This reward function has 2 components:
1. Minimize/maximize "feature(agent)" goal corresponding to a negative/positive reward proportional to the value of the give state feature.
2. Minimize/maximize "action(actor, object) goal corresponding to a negative/positive reward proportional to the number of actions performed.
* All these preferences and the relative priority among them are represented as a vector of weights.

These computer-simulated agents only have a subjective view about the world around them. For instance, the agent A's view of agent B has the same basic structure as the real agent B. [2]
However, agent A's subjective view of agent B is considered separate from the real agent B within the framework and this allows the representation of errors in beliefs. [3]
Hence, these belief models have a recursive structure, but in reality people seldom use very deep optimal behaviour.[2]
An agent's beliefs are affected by action.


Each agent's policy is a function representing the process by which it selects and action based on its beliefs.
This policy is modeled in a way that the agent seeks to maximize expected reward of its behaviour.

To simplify the agent's reasoning, the agents' mental models are realized as simplifies stereotypes of the richer lookahead behaviour models of the agents themselves.
These simplifies mental models include potentially erroneous beliefs about the policies of other agents.
Each agent believes that other agents follow much more reactive policies as part of their mental model of each other.
The use of these more reactive policies in the mental models has two benefits. First, from a human modeling perspective, the agents perform a shallower reason that provides a more accurate model of the real-world entities they represent. Second, from a computational perspective, the direct action rules are cheap to execute and hence the agents gain major efficiency in their reasoning.

Agents use messages to influence the beliefs of one another.
Each message has 4 components: source, recipients, subject and content.
Messages can refer to beliefs, preferences, behavioural policies and other aspects of other agents.


Influence factors:
- consistency: people expect, prefer and are driven to maintain consistency between beliefs and behaviour.
- Self-Interest: The inferences we draw and how deeply we analyze information are all biased by self-interst.
- Speaker's Self-Interest: The tendency of the source of the message to be more critical if it benefits greatly if the recipient believes the message.
- Trust, Likeability, Affinity: Trusting, liking or having an affinity for the message sender greatly affects believing the message.
These factors can all be modeled by rendering each as a quantitative function of beliefs that allows an agent to compare alternate candidate belief states.


A good social scene to be modeled by PsychSim is bullying among young kids.
First generic agent models are selected for various groups or individuals to be simulated and specialaized.
These models compute outcome expectancies as the expected value of actions, meaning that the agent considers the immediate effect of an act of aggression and the possible consequences including the change in the beliefs of other agents.

A bully is given three subgoals for an act of aggression:
1- To change the power dynamic in the class by making himself stronger
2- To change the power dynamic by weakening his victim
3- To earn the approval of his peers

Two possible mental models are implemented into the bully's classmates:
1- Encouraging: they laugh
2- Scared: they laugh only if the teacher doesn't punish them

Three possible mental models of the teacher are implemented into the bully:
1- Normal: teacher punishes the bully
2- Severe: teacher more harshly punishes the bully
3- Weak: teach doesn't punish the bully
*The relative priorities of these subgoal within the bully's overall reward function provide a large space of possible behaviour.
*When creating a model of a specific bully, PsychSim uses a fitting algorithm to automatically determine the appropriate weights for theses goals to match the observed behaviour

Three specific bully models from the overall space are selected:
1- dominance-seeking
2- sadistic
3- attention-seeking

RESULTS: PsychSim allows one to explore miltiple tactics for dealing with a social issue and see the potential consequences.



PsychSim is an environment for multi-agent simulation of human social interaction that employs a formal decision-theoretic approach using recursive models.
PsychSim has a range of technology within itself that eases the task of setting up models.
PsychSim has a range of innovative applications, including computational social science and the model of social training environments.