Ron Sun: Prolegomena to Integrating Cognitive Modeling and Social Simulation

The goals described here have a great deal of overlap between my research as far as models and AI go. Sun provides an extensive discussion of literature in both cognitive modelling and social science. The important difference between the work described here is that it is focused towards scientific applications, whereas my work is in expressive applications, where simulation is used to explain or express ideas rather than emulate reality. Cognition defined here is pretty wide, encompassing: “thinking, reasoning, planning, problem solving, learning, skills, perception, motor control, as well as motivation and emotion.

Some bullet points expressing the chief questions asked by this intersection of studies:

• How do we extend computational cognitive modeling to multi-agent interaction (ie, to social simulation)?
• What should a proper cognitive model for addressing multi-agent interaction be like?
• What are essential cognitive features that should be taken into consideration in computational simulation models of multi-agent interaction?
• What additional representations (for example, “motive,” “obligation,” or “norm”) are needed in cognitive modeling of multi-agent interaction?
• What are appropriate characteristics of cognitive architectures for modeling both individual cognitive agents and multi-agent interaction?

On methodology and methods of simulation: Simulation develops and tests theories. An open question: is there a strong method of simulation to test or understand models? Sources to explore are Axelrod, 1997 and Moss 1999. Regarding the connection of simulation to cognitive science: Sun 2001.

Another reason for looking at social simulation, specifically, is that cognition is a sociocultural process. Note Lave 1988 and Hutchins 1995. This connects the idea of cognition with larger cultural and social meanings. “Only recently, cognitive science, as a whole, has come to grips with the fact that cognition is, it least in part, a sociocultural process. To ignore sociocultural processes is to ignore a major underlying determinant of individual cognition.” (p. 13) Sun mentions later that cognition emerged to satisfy needs and deal with environments, so cognition is necessarily situated and embodied (as opposed to abstract and symbolic). The environment is a part of thinking, and by extension, so must be other agents.

Discussing motivation, thinking, and existent structures, (a sort of intertwined cognitive triad), Sun explains: “The ways in which these three major factors interact can evidently be highly complex. It may therefore be argued on the basis of complexity, that the dynamics of their interaction is best understood by ways of modeling and simulation. One may even claim that the dynamics of their interaction can be understood only by modeling and simulation, as some researchers would. In this endeavor, computational modeling and simulation are the most important means currently available for understanding the processes and their underlying structure…” (p. 14)

There is a small note connecting models and theories: Specifically referenced is van Fraasen, 2002, referencing a position called “constructive empiricism”. The position is that every model is a theory. This idea pulls back to the interesting relationship between models, theory, and practice.

The introduction is primarily concerned with connecting cognition of individuals to the larger scope of social science. Individual thinking at one level is necessary to witness coherence at a higher one. This means that if we wish to understand social science from a coherent perspective, we must look at individual agents and understand how they behave locally, rather than looking at society-wide graphs. This connects to the idea of policy in simulation of expressive systems: comparing a system controlled by a drama manager versus one that is character based.

Taatgen, Lebiere, and Anderson: Modeling Paradigms in ACT-R

This chapter discusses the ACT-R cognitive architecture. ACT-R seems to be designed towards low-level modeling. The architecture seems to be heavily informed by the biological structure of the brain, using layers to handle different cognitive tasks. The system also employs an activation model for memory, which echoes the connectionist model of neural networks. However, oddly, the first example given in describing how the architecture performs is by a control problem, a “sugar factory”. This is an extremely disembodied and disconnected abstract problem. It strongly resembles the sort of feedback cycles described by Weiner’s cybernetics. In later examples, the focus of the architecture is learning when to apply different clearly defined rules.

Wray and Jones: Considering Soar as an Agent Architecture

This section documents Soar as an architecture and as a general theory of intelligence. Right away, the authors begin making the claim that Soar can be used as a holistic and complete model of how everyone thinks, falling well within Alison Adam’s feminist criticism of AI paradigms. Supposedly most or many applications of Soar are intended to be models of specific domains, rather than cognition in toto.

The main features of Soar are thinking cycles, problem solving, and operators. One criticism of the architecture as described here could be the manner in which the problem space is highly disconnected from the actual context surrounding the problem itself.

Clancey, Sierhuis, Damer, and Brodsky: Cognitive Modeling of Social Behaviors

This essay discusses several aspects of social behavior. The paper starts by using a number of terms that should be familiar in the context of social science or sociology: roles, procedures, norms, etc. The paper is also concerned with the idea of collective cognition, which shifts the focus of investigation from goals towards behavioral patterns. This idea is strongly connected to activity theory. This references Lave and social extensions to cognition.

This study specifically looks at a (real world) NASA simulation of a Mars landing team, using extensive footage of the participants enacting the simulation in the FMARS station on Devon Island in the Canadian Arctic.

A key part of this study is the use of the Brahms model, which formalizes field observations for use in developing simulations. The approach used by this study includes a set of clear steps:

• Understanding activities as patterns of what people do, when, and where, using what tools or representations;
• Representing activities in a cognitive model using a subsumption architecture (i.e., conceptualization of activities occurs simultaneously on multiple levels);
• Understanding that conceptualization of activities is tantamount to conceptualization of identity, “what I’m doing now,” which is the missing link between psychological and social theory (Clancey, 1997, 1999, Wenger 1998).
• Simulating collective behavior in a multi-agent simulation with an explicit “geographic model” of places and facilities, using the Brahms tools.

The Brahms model is intended towards real-life analysis of human behavior, but it is formal enough to extend into the simulated domain. The model is intended as a system for understanding group dynamics in the workspace. Most analytic models are descriptive, that is that the cannot be used for generation or simulation, so it is notable that the Brahms model falls under this category. The focus is on practice and observing what people do, though this encompasses emotion, attitude, and personality. Most shockingly, the Brahms model uses an approach that activity is the same as identity. Being is the same as doing in this case, which resonates deeply with Goffman’s theory of social performance.

The process of simulating the already recorded events is especially tricky, as the simulation must account for many uncontrollable human variations. There is the idea of simulation fidelity, which is the capacity of the computer simulation of the model to accurately recapture the behavior of the participants without doing anything wierd, such as having them all stand up simultaneously at the end of a meeting. What arises again, though is that the FMARS habitat is a simulation as well, and its participants are all performers. So what we have here is an electronic simulation attempting to simulate a performed human simulation. If we bring a theorist like Baudrillard into play, he would probably say that there is no way to actually capture real social behavior or activity, since it is all simulated anyway. However, there is still a gap between the human simulation and the virtual, and this is a gap that can be narrowed.

The way to more closely simulate the humans is to understand that social behavior is a necessary component of individual behavior. Additionally: knowledge is also hard to model. Finally: roles are improvised and are blurry. There are some interesting formal descriptions of the behavior rules. The interaction with the environment works via a number of perception functions and stored variables. The behavior is stateful, but according to cases.

Another interesting thing in the study is the emphasis on biological needs. This makes sense for NASA, but it does not really apply to narrative in the cases that I am working on. It does however lend a certain natural credibility to the simulation, emphasizes the embodied nature of the subjects, and it echoes the decision to have biological needs expressed in The Sims. This has some interesting consequences, though: “The inclusion of biological motives in explaining human behavior provides an interesting problem for cognitive modeling. For example, consider KQ warming her drink in the microwave and then standing by the side of the table. There might be many explanations for this behavior: Her drink may be cold; she might be cold; her back may hurt; she may be bored with the meeting; someone at the table who hasn’t had a shower in a week may smell, etc. One doesn’t know her goals, aside from, perhaps, warming her drink. Even this may be a kind of convenient cover for accomplishing her ‘real intention.'”

There is a strong critique of rational frameworks present here. Simulation is generally concerned with advancing state, and not necessarily determining intention, although certain behaviors may strongly hint towards intentionality. The cognitive model described by the Brahms framework (as well as Soar and every other AI framework, also note Cavazza) involves a top-down model of behavior. These presuppose goal and structure driven models, which may not be appropriate. Top-down models cannot accomodate for human flexibility and ambiguity. This suggests that a situational and context-driven model is key to representing human behavior.

A final word on the modeling philosophy: Modeling “a day in the life” is a starting point, but on its own it is a pastiche (!), much like The Sims.

A connection to Newell’s perspective on cognitive modeling: Newell says that interaction is oganized into isolated and discrete bands, which pulls back to rational goal-driven behaviors. This does not account for social norms and personal habits, which are essential to understanding social behvior.

Gratch, Mao, Marsella: Modeling Social Emotions and Social Attributions

The focus on this paper is on emotions with social elements, stemming from human interactions. These involve not only causality, but also intentionality and free will. The essence in this idea is developing a theory of social intelligence. The social interaction described here resonates with Geertz and Goffman. The goal in this paper is to develop a framework for modeling emotions.

There are several specific points to the cognitive model developed here, building from cited sources (Minsky, Oatley, and Johnson-Laird):

1. How emotion motivates action
2. How emotion distorts perception and inference
3. How emotion communicates information about mental state

A tool used in this paper is Appraisal theory, which explains that emotion arises from two sources: Appraisal and coping. Appraisal itself is the process by which knowledge is understood and reacted to, and coping is the response to events, sometimes leading to change. A key author in developing the model here is Lazarus, 1991. Both the coping and the appraisal processes are complicated and feed back into each other significantly, so the study uses Soar to develop a model of the complex cycle between the two forces.

Appraisal is arranged into variables, and these are described:

• Perspective: from whose persepctive the event is judged
• Desirability: what is the utility of the event if it comes to pass, from the perspective taken (i.e., does it causally advance or inhibit a state of some utility)
• Likelihood: how probable is the outcome of the event
• Causal attribution: who deserves credit or blame
• Temporal status: is this past, present, or future
• Controlability: can the outcome be altered by actions under control of the agent whose perspective is taken
• Changeability: can the outcome be altered by some other causal agent

Beyond that, there are several tyeps of coping strategies:

• Action: select an action for execution
• Planning: form in intention to perform some act
• Seek instrumental support: ask someone in control of an outcome for help
• Procrastination: wait for an external event to change the current circumstances
• Positive reinterpretation: increase utility of positive side-effect of an act with a negative outcome
• Resignation: drop a threatened intention
• Denial: lower the probability of a pending undesirable outcome
• Mental disengagement: lower utility of desired state
• Shift blame: shift responsibility for an action toward some other agent
• Seek/suppress information: forma  apositive or negative intention to monitor some pending or unknown state

This collection of coping strategies is really great, especially as it pushes the classical AI scope of planning and action only. It exposes a great deal of underlying potential and variety in modeling emotional behaviors. This also raises the question of how this sort of coarse structure might be defined against the fine granularity of simulation. Note that there is a great deal of importance on interpretation.

The decision/emotional cycle represented in the EMA application is as follows:

1. Construct and maintain a causal interpretation of ongoing beliefs, desires, plans and intentions.
2. Generate multiple appraisal frames that characters the state in terms of appraisal variables.
3. Map individual appraisal frames into individual instances.
4. Aggregate instances and identify current emotional state.
5. Propose and adopt a coping strategy in response to the current emotional state.

Note that the current implementation emphasizes task oriented goals. This relates to the general criticism of Soar and planning-based AI paradigms. The authors mention that the selection of tasks does not account for social norms and standards, and propose a model of dis-utility to associate with breaking these, but it still involves a utility based model, which does not seem to be a satisfying solution.

Attribution theory connects to the idea of intention and responsibility, which might better handle credit and blame. The theories on this fall under Shaver and Weiner. However, embedded into the implementation model described is the ever present figure of authority. Attribution is of great importance in a chain of command, but exact attribution is never really used in real circumstances. Every action in this model contains not only the performer, but also the individual who coerced or ordered the action. This model is appropriate in a military simulation, but carries an undesired value in other circumstances.

The authors give a very significant and powerful logical model of attribution theory, basing on a set of primitive logical functions, axioms, and rules for discerning attribution. These all hold from a rational perspective, which again makes sense in a serious application where the logic is meant to solve problems and discern information, but not in an expressive application. The entire layout forms a powerful attribution framework, but with the idea of increasing complexity and partial or faulty information, the idea breaks down for other social simulations.

Overview:

Mackay’s ultimate goal in this work is to develop a legitimate framework for interpreting role playing games as a performative art form. His analysis covers the cultural, formal, social and finally aesthetic structures of RPGs. Mackay is interested in the artificial world and the networks of meaning established by the performance of the games. Performance is the key element that makes role-playing an aesthetic subject.

Mackay’s arguments in the later stage of the book situate the idea of role-playing in history as a natural consequence of the desire to recreate a sense of lived otherworldliness which has been supressed since the age of Enlightenment. Role-playing leads to the development of deep and personal imaginary worlds, which becomes an artistic object or artifact when recalled in memory.

This book ties together the diverging threads of performance, simulated worlds, and the expressive power of participation and interpretation.

Notes:

Introduction frames the goal and direction of study: Analyze role-playing games as an art made up of both role-playing, and game. The ultimate value of the game comes as performance, which has its own aesthetic background. The “Performance of the role-playing game brings the game into existence, and it is therefore of the foremost importance.” This study is not a poetics, which would describe how to create games, but an aesthetics, which defines a system of analysis of games. The analytic structure for games is taken from Eric Zimmerman and Frank Lantz. A game is split into three dimensions: formal, social, and cultural. Mackay adds a fourth dimension, which is the aesthetic dimension.

The word “narrative” has been used to describe the over-arching story of the game. This makes sense, as the narrative is the history of the game, and the use of the term is supported by the fact that it is verbal telling that is used to drive the game. This construction operates against the sense of archetypical dramatic narratives: narrative here is something that is told (enacted) and then re-told (described).

Cultural Structure

Mackay opens discussing the cultural structure of role-playing games by examining their history and origin arising from war games, which arose in the early 1800s with “Kriegspiel”. Most traditional games, ie card or board games are zero-sum games. War games diverged from this model with a nonzero-sum approach, which led to interesting dynamics using cooperation and subterfuge.

There is a very explicit connection to RPGs and literature. RPGs that emerged out of the wargaming tradition were heavily anchored in settings defined by pulp literature. Mackay writes an equation that sums this up: “Fantasy Literature + Wargames = Role-Playing Games”. This connects to the idea of models, but we do not yet see a connection of the world to game mechanics. We also primarily see fantasy, sci-fi, and pulp literature used here, as opposed to settings that are more “highbrow”. Literature forms the world and the cultural frame of meaning around the game itself. This point connects very strongly to Jenkins (and possibly Michel DeCerteau), and encourages the idea that fans are coopting these cultural artifacts.

The emergence of D&D in the 60s and 70s is also rooted in American culture, exposing a sort of backward-looking nostalgia for a pre-technological era, and a setting without the moral ambiguity present in the political climate. The desire for clarity in distiction between good vs. evil was not satisfied by the “mirages of communism versus free world, cowboy versus Indian, and good guy versus bad guy that permeated the political rhetoric and cultural climate of the 1950s.”

The influence of role-playing games and culture is the most evident in computer games and especially online environments from MUDs to MMOGs. The importance specifically relates to world setting and the interpretation and cultural meaning thereof. Mackay pulls Baudrillard into this connection using the idea of the “semiosphere”, an atmosphere of signs. RPGs regurgitate cultural myths, narratives, and world settings.

The aesthetic of fantasy is the depth of detail and setting. The fiction can be so detailed that it can be imagined. This idea not about realism, but the impression of reality. This idea connects back to the immersion. Player engrossment is through the character, the player co-constructs the fantasy through his or her own imagination. Electronic games cannot do this because they establish a role that is opposing the player. Human imagination is stiffled when presented with observable detail. There must be something about writing and fiction especially that enables this.

Formal Structure:

An interesting connection is made here to amusement parks and “themed entertainment”. The player as spectator model is consistent with how I personally run games, but it is not universal. A connection is made to the aspect of simulation, specifically through Barthes and Baudrillard: The idea is that the logic internal to the game comes to have a life of its own, and detaches from both the real world and its origins. That is, a game world may have originated there, but it no longer lives in sourcebooks, it comes to have a mythology and life detached from physical anchors. Uri Rapp wrote explicitly on this in “Simulation and Imagination, Mimesis as Play” in 1984.

Mackay brings up the interesting example of Everway, which is about “Visionary Role-playing”. This has an abstract conceptual ambiguity that is diametrically opposed to D&D’s rigorous attention to mathematics. It represents a contrast with Cartesian and non-Cartesian thinking. In Everway, aesthetics are incorporated into the formal structure.

Connecting Schechner on performance: Rules guide a performance through constraint, creating safety and security. Note that this is entirely consistent with conversation with Miashara earlier. The RPG narrative is created by performance. This is interesting to compare with other game studies, the relationship between performance and play. The difference between Schechner and the RPG model has to do with the code that defines the performance: “The role-playing game exhibits a narrative, but this narrative does not exist until the actual performance. It exists during every role-playing game episode, either as a memory or as an actual written transcription by the players or game master. It includes all the events that take place in character, nonplayed character backstories, and the preplayed world history. It never exists as a code independent of any and all transmitters, like Schechner’s definition for drama suggests.” (p. 50)

There is some discussion connecting Goffman and framing to the levels at work in games. This describes the various principles and rules and forces that are at work in guiding the game experience. Drama is described as a force that operates on the game at a meta-level. This is explicitly stated in Everway. While players may be aware of the dramatic force at work, the player characters are not. This enforces the notion that drama is simulated like any other rule. This makes an interesting connection to drama managers, which operate on a meta-level in a very similar way.

Social Structure:

Performance and experience exist in all frames simultaneously. The character/player exists in all these levels as well, and identity blurs as the levels meet. Mackay is using Schechner to critique the borders of frames as defined by Goffman, Fine, and Gregory Bateson. Schechner argues that ritual takes place on a level that transcends the frames of interaction.

This section is also called “The Structural Foundation of the Role-Player’s Subjectivity”, which echoes Bogost’s description of simulation, as the gap between the rule-based representation of a world and the player’s subjectivity. The player’s subjectivity in this case also represents the agency of the player to co-construct the game world. Drawing on Barthes, Mackay argues that role-playing games function by exposing the construction of meaning. He muses that the religious right has reacted strongly against role-playing games because they represent a world where people give meaning to things and “try to render intelligible the process behind creation.” (p. 68) The creation of meaning is driven by “blanks” as described by Wolfgang Iser.

Game worlds and game culture take on the idea of speculative or fantastic recreation. Fantastic recreation is what drives the “global villiage” of the Epcott Center. This connects with Bakhtin’s idea of the desire for an alternate or unofficial culture, which also sounds connected to utopian desire.

The relationship between culture and gaming: Constructed characters are reflections/echoes of existing culture, like Deleuzian assemblages. An interesting concept mentioned here is the notion of the “decontextualized tropes” or “fictive blocks” which are tiny bits of culture that can exist without context. Fictive blocks are essentially instances of meaning in a sound-byte culture. To explain how these are used, Mackay references Arnold Van Gennep (1908), who describes stages of separation, liminality, and reincorporation, which are used in rites of passage. The three stage process applies here to fictive blocks in cultural artifacts. A sound byte or image or idea might be taken from a fictional work, then isolated and disconnected from its context, and later reincorporated into some other creative material. This idea connects again very strongly to DeCerteau and Jenkins.

Mackay brings up Foucault to describe power relations in role-playing. The space of the role-playing game is an interesting target for studying power play, especially given the absolute power of the game master. However, this idea goes back and can be applied in an interesting way to power in electronic games. In electronic games, the player has no power, but is not surveiled, but given a certain autonomy, players have massive freedom. The level of personal relation in RPGs allows for odd features that relate to discipline. In a role-playing game, the player is certainly compelled to behave with a certain level of discipline, especially in terms of keeping in character and observing social standards. In electronic games, both the online and offline varieities, players have no compulsion, and will behave very rudely, inconsistently, and incoherently. The strongest example of this is when players attempt to push the limits of a game and break it. There is a strong cultural tradition of this, but it is something that will not be tolerated in role-playing sessions, even when the game mechanics allow for abuse of in-game power. Why this is the case is a deep and complex question.

Aesthetic Structure:

The aesthetic structure is a necessary component to role-playing games. Discussing the engrossing and enchanting power of other types of games, Mackay writes: “The role-playing game performance shares these structures with other activities. However, it also participates in a fourth structure, an aesthetic commonly attributed to art: a cathartic structure that encourages identification with its content and that persists after the performance has disappeared. This structure is at once a social process, a cultural process, and a formal process, but it is also something more. It is the creation of an aesthetic object that results from the collective interpretive process of the role-plaing game performance.” (p. 122) This also exposes some of the lack in electronic games: players have control over the social, cultural, and formal levels of experience, but are not able to contribute to the aesthetic structure.

Connecting art and theatricality: According to Michael Fried, art is opposed to theatricality. Modernist (literalist) art takes the extreme position of reducing art to pure objects. For instance: a painting is just paint on canvas. Fried describes Tony Smith’s car ride on the New Jersey Turnpike before its completion. The idea here is that kinetic, immersive, explosively imaginative experiences work towards the aesthetic of the role-playing game narrative. The aesthetic is the residue left behind in memory after the experience has passed.

In describing historical reenactment, Mackay connects once more the world of literary fiction and wargaming. “I see this moment, when the increasing aestheticization of the war gaming narrative finally culminated in the development of role-playing game performance form, as a reaction to the poverty of the imagination that emptied the architecture of everyday life of any meaning and the scarcity of vision that burdens contemporary philosophy and literature. The imaginative faculty is a built-in function of the human organism–the equivalent to pulses of the heart or respiration of the lungs. If a people do not find that faculty fulfilled in the world they have been handed, they will build their own.” (p. 153)