Jointly shaping meaning:
The interactional generation and transformation of semiotic media

Hanne De Jaegher and Ezequiel Di Paolo

The First Conference of the Swedish Association for

Language and Cognition, Lund University, 29 Nov-1 Dec

In the different areas of research on social cognition, the idea that we read each other’s behaviours and infer conclusions based on these perceptions in order to predict and explain each other’s mental states is still prevalent. Think for instance about the Theory of Mind theories or their rivals, Simulation theories, that have been proposed.

However, these views come under increasing criticism. Some argue that we should replace them with an embodied alternative, and relocate the explanatory emphasis to the expressiveness of our bodily comportments – our gestures, postures, vocalisations. In order to explain how we perceive an other’s intentionality in these, “embodimentalists” sometimes refer to innate neurological mechanisms (e.g. mirror neurons).

An argument can be made, however, that such mechanisms on their own do not suffice to explain interpersonal understanding. Indeed, following the embodied alternatives to cognitivist explanations of social understanding, it has been suggested that the latter need to be complemented with an approach that centres on the interaction as a meaning-generating process. That is, we need to look at the bodies-in-interaction of social encounters.

Coordination plays a crucial role in social interactions between embodied persons, specifically the coordination between the bodily movements (including utterances) across the agents involved. How bodies coordinate in interaction can be examined, coordination patterns can be mapped, and so on. But how does this relate to how persons convey meaning to each other? In other words, how does the physical coordination of bodies in interaction relate to the interpersonal understanding (or non-understanding) that goes on between people?

Enactive approaches to cognition define it as sense-making: the cogniser’s active, meaning- and value-generating engagement with her world. This idea has recently been extended into the social domain, where we can speak of participatory sense-making: the interactional coordination of individual sense-making activities, with its own characteristics, which can open up different domains of sense-making than those that are available to an individual alone.

A subdomain of participatory sense-making is the generation of physical objects of meaning. A medium that, by its material properties, allows for different and changing degrees of mediation throughout its transformation during the interaction process, has the possibility of modulating ensuing interactions. The medium becomes an object of meaning in this dialectical process of mutual transformation. Meaning is thereby historical and often changing in its mediating distance (or semioticity) by acts of interaction that re-inscribe (re-affirm or modify) the modulatory effects (or significance) that the object has on the interaction itself.

We discuss three examples of this: a co-authored paper written in a collaborative word-processing tool, a list drawn up by two people together on a piece of paper, and a record of scores of a game on a sandy beach. The question guiding the exposition of these examples is: How does the (generated, transforming) materiality of these significant objects influence the interaction between participants of the social encounter, and vice versa, how does the interaction influence the generation and transformation of these objects? The perspective proposed allows for a discussion of semioticity at at least two timescales: the generation and transformation of such objects within one single interaction (e.g. a conversation), and throughout a history of interactions (for instance a series of several meetings about a paper). We discuss how the specific materialities of these objects and interactions constrain and modulate each other in the process of participatory sense-making.