College of Computing

Georgia Institute of Technology
Atlanta, GA 30332-0280, jim@jimdavies.org

Mitsubishi Electric Research Labs, 201 Broadway,

Cambridge, MA 02139, {lesh, rich, sidner}@merl.com

The MITRE Corporation
202 Burlington Rd, ms. K302

Bedford, MA 01730, gertner@mitre.org

USC Information Sciences Institute

4676 Admiralty Way, Suite 1001

Marina del Rey, CA 90292-6695, rickel@isi.edu

Computer tutors and intelligent software assistants have traditionally been thought of as different kinds of systems. However tutors and assistants share many properties. We have incorporated tutorial strategies into an intelligent assistant based on the COLLAGEN architecture. We are working on an agent, named Triton, which teaches and helps users with the graphical user interface of an air travel planning system. We found that the collaborative model underlying COLLAGEN is an excellent foundation for both an assistant and a tutor, and that both modes of interaction can be implemented in the same system with different parameter settings.

Intelligent tutoring systems, software agents, intelligent assistants, collaboration, discourse

As a student gains expertise, a good tutor's behavior often begins to resemble an assistant's. Similarly, a good assistant can teach tasks as well as do them. We believe that, especially for procedural tasks, tutoring and assisting are best thought of as points on a spectrum of collaboration.

In this paper, we report on our work in progress to incorporate tutorial strategies into an intelligent assistant for an air travel planning application. Our initial findings are (1) that by viewing both tutoring and assisting as collaborative activities, the same set of mechanisms can be used to support both, and (2) that COLLAGEN can be extended and generalized to support both activities. This paper is a case study of how we extended COLLAGEN and used it to build a tutoring/assisting agent called Triton. We will describe some of the differences between tutoring and assisting, and discuss how these differences can be encoded as parameters, in a single system

COLLAGEN (COLLaborative AGENt) [6] is middleware for building collaborative interface agents based on discourse theory. It provides application developers with a platform on which to implement agents for applications. Its theoretical underpinnings provide the developer with a general model of communication between user and agent (including interpreting intentions), and with a model of actions for manipulation of the application itself. The action model provides for action decomposition (via recipes) which the developer specifies for a particular application. The agent uses these recipes to interpret user actions and to plan its own actions.

As shown in Figure 1, COLLAGEN provides individual user and agent "home" windows for communication purposes. Agents built in COLLAGEN interact with both the user and the application. Our agent, Triton, was built with COLLAGEN to assist or tutor the user in the use of an air travel application. In contrast with COLLAGEN, in typical ITS (Intelligent Tutoring System) authoring programs, developers are required to intermix encoding of domain knowledge, pedagogical strategies, and the application environment. These systems do not support modeling the communication and actions of the agent and user as a collaboration.

The air travel application can be used without an agent. Routes are selected by clicking on cities on a map. The times the user needs to be in a city are specified with sliding interval bars. Flights fitting the constraints are displayed.

We have previously reported [6] on an intelligent assistant we built for the air travel application using COLLAGEN. The assistant monitors what the user does and communicates with her so that they have a shared understanding of the task. With this knowledge, the assistant can propose next actions or even perform them itself. Figure 2 is an example scenario of the assistant’s behavior (the city constraints of the trip were communicated earlier in the dialogue).

…

36. User: Let’s plan the second leg of the trip.
37. Assistant: Okay.
38. User: [selects Washington] What next?
39. Assistant: [points to San Francisco] I will select San Francisco.
40. User: Okay.
41. Assistant: [selects San Francisco]
42. User: What next?
43. Assistant: Let’s remove an airline constraint.

…

When a task is proposed by the user or the agent (as in lines 36 and 43 respectively), either collaborator can do the action (the user does the action in line 38; the agent in line 41). The agent can suggest what is next (line 43) because it knows what is likely to follow based on its recipe library.

The most fundamental difference between tutoring and assisting is that for assistants the main shared objective is to get the task done (a task goal), whereas for tutors the main objective is for the user to learn how to get the job done (a learning goal). When the student wants to learn as well, this goal is shared.

There may be other goals as well that are in service of the learning goal, but may sometimes conflict with it. For example, the goal of keeping the student’s motivation level high might conflict with the overall learning goal, because it suggests helping the student when she gets stuck rather than making her perform every action herself. An assistant may sometimes form a subsidiary goal of teaching the user during task performance. The assistant may want to make sure that the user is able to do the task herself next time. In that case, the learning goal is in service of the task goal.

The differences between the objectives of tutoring and assisting required us to add functionality to the assistant in order for it to also be a tutor. Furthermore we discovered that many of the necessary differences could be implemented in COLLAGEN as adjustable parameters. The values of these parameters determine how assistive or tutorial the agent's behavior is, and also the tutoring style used. In this section we describe how these differences were encoded in Triton’s parameters. Like the original assistant, Triton is an agent built with COLLAGEN.

Figure 3 is a scenario demonstrating some tutoring behaviors that we will use as an example. In this scenario, "What next?" is a generic help button the user can press when unsure of what to do. The constraints of the flight were communicated to the user earlier in the dialogue. In this scenario the user asks for help a lot; this is only to show Triton’s functionality. The user could do many correct actions in a row, and Triton would follow along without interrupting.

…

17. User: What next?
18. Triton: Please specify the route of the trip.
19. User: What next?
20. Triton: Select Atlanta and then select Boston.
21. User: What next?
22. Triton: Please select Atlanta.
23. User: What next?
24. Triton: [points to Atlanta with cursor] Please click on the dot near Atlanta on the map.
25. User: [clicks on Atlanta] What next?
26. Triton: Please select Boston.
27. User: [clicks on Dallas]
28. Triton: Oops! What you did was select Dallas; what you should have done was select Boston. I will bring you back to the previous state by clicking undo. [undo] Please select Boston.
29. User: [clicks on Boston]
30. Triton: Great job!
31. User: What next?
32. Triton: Select the dates and times of travel.

Figure 3: A tutorial scenario in which Triton is leading the user through a worked example of planning a flight from Atlanta to Boston.

Tutors and assistants often differ in what they know about tasks. While an assistant usually has generic recipes, a tutor often relies heavily on worked examples. A generic recipe has many unspecified parameters such as what city to select, whereas a worked example the recipes for a worked example has specific bindings, such as selecting Atlanta. Worked examples are particularly important for artificial tutors, for which it might be difficult to automatically generate explanations and error recovery strategies from purely generic knowledge.

Our worked example recipes in Triton were developed as extensions of the generic recipes in the original air travel assistant.

When there is a conflict of opinion between agent and user on what should be done next, an assistant usually trusts the user’s judgment. This is not always the case with tutoring, where the student is less likely to be right, and where correcting errors is important.

If the user says she is done with a task, our original assistant assumed the goal is satisfied. In a tutoring situation it is the tutor who ultimately decides when the task is correctly completed.

COLLAGEN interprets user actions with reference to its recipe library [4]. Using this plan recognition capability, it can understand when a user begins working on a new task. If the user is doing something that is not a part of a known recipe, COLLAGEN treats it as an unrecognizable action.

When the user starts doing something the agent doesn't understand, an assistant will usually let the user proceed without interruption. But the more a user doesn’t understand what she is doing, the more likely that interruption is actually an error. Tutors intervene to get students back on track.

In Triton, the point at which the agent intervenes is controlled by a parameter. Triton can never intervene or stop the user after n unrecognized actions. We chose as Triton's current default behavior, based on Anderson et. al. [1], to intervene as soon as a mistake is detected (i.e. n=1).

New response mechanisms were also added to COLLAGEN to support error remediation. Triton first tells the user what she did, and reminds her of what she should have done. Then it uses the "undo" button, to put the application back to the previous state, and reminds her of what to do next (See Figure 3, line 28 of for an example).

An assistant will usually try to be maximally helpful with the task at hand. In contrast, a tutor may choose not to be helpful because the goal of having the student learn outweighs the goal of getting the task done. This broad difference has the following consequences.

If an assistant sees that a particular action needs to be done next, there is no reason for it not to propose that action immediately. A tutor, on the other hand, may want to wait and give the student a chance to come up with the next action on her own, or ask for help if she is really stuck. The level of initiative was a already a parameter of COLLAGEN. Its behavior needed to be refined somewhat, however, to better support tutorial dialogue.

When there is an action to be done, an assistant should be willing to do it. A tutor, however, may decide that the student must do the action herself in order to learn effectively. In Triton, a parameter determines whether anyone or just the user does application actions.

Assistants and tutors both should be able to suggest what to do next. In Triton, the user gets more help as she asks for it. At first, Triton gives a task description of what to do next (e.g. Fig. 3, line 18). This is called proposing an action. If the user asks for more help, Triton returns a more detailed explanation of what to do (e.g. Fig.3, lines 20 and 24).

In Triton, an explanation consists of any number of utterances, pointing acts or demonstrations. Explaining behavior is implemented using recipes, which are hierarchical structures of actions: (1) Composite actions, which can be decomposed into other actions, and (2) Primitive actions, which typically coorespond to single clicks on the application GUI. Figure 4 shows the part of the recipe used in Figure 3.

It is a common strategy in ITS's to initially respond to students' help requests by giving a high-level hint, and then to provide more specific information if the student asks for more help [1, 2]. Triton uses this strategy. The first request for help results in a task description. The second request results in an explanation. Each action has an explanation associated with it stored in an explanation recipe. The explanation recipe for a composite action is an automatically generated utterance listing the task descriptions of the actions composing it. The explanation recipe for a primitive action is an application-level description of what to do on screen (e.g. "Click on the dot near Atlanta on the map").

As the user asks for more help, Triton goes on to the next action. If the previous action was composite, then the next action will be its first component. The idea is that if the student does not understand how to do the composite action even after hearing what it is composed of, what she needs to hear next is how to do the first component. In Figure 3, line 21, the user asks for more help after the breakdown explanation has been given. Triton responds by moving down the action composition hierarchy to the next action, which in this case is a primitive. The task description is given first in Figure 3, line 22. The user asks for help once again in line 23, and then the explanation for the primitive action, an application-level instruction, is given in line 24.

Sometimes a tutor might want to demonstrate how something is done. For Triton this means performing a sequence of actions, followed by a series of undo actions to put the application in its previous state. We put demonstrations in the explanation recipes: Triton does any actions that are in explanation recipes (including, usually, the appropriate number of undo’s). This simple change generates the desired demonstration behavior. Both task descriptions and explanation recipes were a part of the existing COLLAGEN architecture.

The original assistant would always use its cursor to point to the part of the application to be worked on when proposing actions. The assistant was parameterized so that when tutoring, it would point at the correct place in the application at a more appropriate time: when providing an explanation for a primitive action.

Intelligent tutors and assistants have much in common. There has been a great deal of research in both tutoring [7, 8] and assisting. The goal of this work is to bridge these two areas. Triton provides both modes of interaction, showing that a single architecture based on collaborative discourse theory can support them both. In natural settings, both tutoring and assisting are mediated by dialogue [3, 5], shared goals, and plans. Tutors can smoothly transform into assistants, and assistants can tutor when needed.

This work makes two main contributions to the ITS field. The first is a middleware architecture that is independent of both the domain being taught and the pedagogical style used. The second is the use of recipes as a single representational structure that can accommodate abstract actions, explanations, and worked examples, including GUI actions, pointing actions, utterances, explanations, and demonstrations. A new error response system had to be added to COLLAGEN. For tutoring strategies to take place, some recipes must be written as worked examples and explanation recipes. All of the other behavioral differences between Triton and the original assistant were made into parameters. The new parameters are (1) When to intervene after error detection (after how many unrecognized steps), (2) who should default to do the task actions (the user, or anyone), and (3) when to point (upon proposal or explanation).

In the future, among many other open questions, we would like to investigate how to use a student model [8] to inform parameter value changes so it can shift smoothly from a tutor to an assistant role.