The Shannon-Weaver model for communication posits face-to-face as the most immersive, rich, and information-dense mode and medium. I do not disagree with this. The human mind is so nuanced at social perception, and all of our senses can perceive the embodied expressions of another person, and how they receive our own expressions (whether we are conscious of them or not). That is truly incredible — a magnificent feat for anthropologists, sociologists, and psychologists to understand, let alone for any advanced technological system to replicate, simulate, or emulate. Alas, I think this is a futile quest, and I do not think this is ultimately the point. Emulating face-to-face communication should not be the gold standard and benchmark for any new media technology.

Why re-create what is already there?

It seems that with every new major technological invention, the technicians try to show off how potent the system is in emulating a natural process or phenomena. Writing preserved spoken stories from our ancestors. The daguerreotype allowed artists to accurately “trace” reality, to preserve it.  Portraiture was a way to reproduce the face of a person. Once another technology replaced a function of a former, sufficiently fulfilling it, then the latter could now be free to go outside the bounds of being technically precise.

What impresses me most about the art that I enjoy is that it refers to something that does not have an explicit reference point in “objective” reality. And yet, it is ostensively and viscerally familiar. For instance, Escher’s drawing of first-person perspective accurately captures an embodied viewpoint (even given its faults). Cinematography can artfully capture the mood, emotions, and attention of a person or group. Some may argue that these artworks create their own symbols that then become emblematic (a posteriori), but I disagree; Rendering these experiences comes from a place of familiarity (a priori). We all dream, and remember, and we know what a dream or a memory looks like, and how emotionally evocative a dream can be. It is, in my opinion, unfair to classify these works of art as “subjective” because they are emblematic of a shared experience that most people can readily identify. “Phenomenological” might be more accurate, but the term does not usually imply shared mental representations. For instance, most people know what it is like to faintly remember a place, or an interaction with a person, without the details. In Eternal Sunshine of the Spotless Mind, Michel Gondry depicts this familiar mental representation by showing a bookstore with dim, blurred section titles, book titles, and background. Memory is not a bit-by-bit translation of a lived experience; Perception does not even grasp all of the information of reality “accurately”, nor “completely.” So why should we hold technology to a standard that is incompatible with our lived experience and mode of forming mental representations? Instead, can art and technology help us to reveal more of our “undisclosed” mental lives, as well as our embodied modes of accessing, grasping, and perceiving the world?

I believe that the more honest we can be with ourselves and one another about these experiences, and the more we can develop tools to agree upon their common or idiosyncratic features, then the more we can express and communicate. This can allow us to feel a greater sense of communion and transcendence with a shared, but nuanced, human experience. Is there any more happiness-inducing feeling than that? My argument is that we can fuse artistic insights with the technical possibilities of new devices to create tools that can represent our shared embodied and mental modes of accessing the world. This will allow us to then go more deeply into understanding embodied subjectivity as it is experienced by and for others. It can also invite a meditation on our own habitual modes of conducting ourselves in the world by considering a variation that was expressed by another person via this technology. I want people to reconsider how they pay attention to sensations in their bodies and in the world around them, and realize that every moment is a choice and an opportunity to direct our attention in new ways. When we change how we direct our attention, our perception, and by extension, our reality, literally changes. More importantly, I would like for other people to be our gateway into this insightful experience so that it enriches our social connections. I want us to think of other people as other uniquely refined modes of access to this shared world, and to a shared embodied subjectivity that is a condition onto the world given by being human.

So, here is my argument: Yes, any skilled meditation practitioner or skilled expert can have a refined mode of attenuating to their experience. However, what if we use technology to help reveal these modes of attenuating as experienced by or for another real person? And what if, then, after having our “senses awakened” (so to speak), we could then have a conversation with this person?

Here is an example of what I can imagine a user saying after using this technology. “Wow, today I was sitting in the woods and I was able to be guided with a gaze cursor and auditory cuing to everything that you noticed. When I heard the bluebird, my haptic device sent a vibration to my hand. I saw your hand tracked as you pointed to a tree. “Look for the blue!” I heard you say. The gaze cursor focused on a moving blue spot, and I could then make out the bluebird. And then I heard you say that you were so excited because this was your favorite bird. It was fun to actually be cued to feel your excitement in my body the way you felt it in yours.”

Yes, we could have this experience by sitting next to someone and having them describe their experience to us — but how would we align our perceptions so that we ultimately see, hear, feel, and know to direct our attention to the same sensations? This is a difficult feat! The technology doesn’t aim to replace face-to-face — nothing ever will or can. But it provides something altogether new and different. We just have to solve the puzzle for how to adequately capture and transmit these embodied perceptions from one person to another, and I think we have technology available now that can do that really well. It’s just that most people would never even conceive of that as the end-goal.

I would like to propose that technology can offer new ways of sharing experiences with one another. This is the seed of my passion. It seems so obvious to me that we all crave connection, and currently technology is delivering such a superficial level of social contact and interaction that a new sense of isolation is emerging. The modern human is lonely in a way that is completely novel because social technologies provide a deception of social connection and engagement. I do not know if this can be solved, but I think that we can really use technology as a communication tool to explore our shared humanity, as well as the unique alterity and intricacies of each of our minds and imaginations. I think that technology can genuinely evoke a sense of awe and wonderment for the varieties and complexities of human experience. It’s a tool among tools; A great writer, for instance, can also get a reader into the mind of a character.

Augmented Social Cognition. This is the term that I use to describe the vision I have for technology. I think that we can literally share worlds with one another. I want to be taking a walk somewhere, and then one of my friends can send me a capture of a birdsong from a walk they took that morning in some other part of the world. I want to be able to hear the bird from the same embodied perspective as my friend, up in a tree. I want to take a depth-field photogrammetry image of a landscape, or even just an object like a leaf, and send this to someone somewhere else so that they can place it in their environment. I know that there is some work in AR that relates to this, but in my view it always misses this social connection component.

Now let’s go a layer deeper. Rather than just seeing the objects and scenery in the environment of another person, I would like to use innovative sensor and capture technologies to actually transmit modes of embodied experience from one person to another/others. I want to share the sensation of an ocean breeze on my skin (ideally), or something akin (a haptic model of this sensation, for instance).

To be continued…