Consistent and coherent: The recipe for digital presence

Here are two scenarios:

1.Once upon a time, on a planet far, far away, I rocket-jumped over the ruins of alien civilizations, raining death down upon the enemies' heads, amidst a frenzied assault on their stronghold.

I bounced from pad to pad over the vertiginous infinity of star-bedecked space, my ears full of the WHIZZZ! of the rockets, the BLOPBLOPBLOP! of the plasma rifle, the satisfying SPLAT! of gore, and the piteous grunts and death wails of those I had fragged.

2.I sit in front of a computer monitor trying to pay attention to what the speaker is saying. A patchwork of video feeds to the right of the manager's slides fight for my attention.

A few participants clearly wish they were elsewhere, being surrounded by lush tropical vegetation or sitting on a white beach, and they all have different dimensions, foci, lighting, and shadows.

When others speak, the diverse acoustic qualities of their voices betray the fact that they are all in wildly different spaces to me—when I speak, my voice, to me at least, is not tinny and hollow and it interacts with the space in which I sit.

I imagine that a large number of you will identify with at least one of these experiences.

Some might even be able to follow the breadcrumbs I provide to identify the first scenario (a first-person perspective computer game) or even the specific game (Quake 3 Arena—yes, I am that old) and the second scenario as a video conference.

Common to these two scenarios is the concept of presence, or immersion as it is known in the field of computer games.

But what makes for ‘good’ digital presence? In this article I will explore how we become present and the challenges of becoming present in digital worlds.

What is presence?

Briefly defined, it is the feeling of being in a place and being able to act within and upon that place.

It first came to the fore as 'telepresence' in the 1980s in connection with the remote control of robots over vast distances and time delays (think of rovers on the Moon and Mars).

But the concept was co-opted by the field of Virtual Reality where the 'place' of presence is the digital world rather than some distant, but still very real, lunar or planetary landscape.

We can even use the concept of presence to explain our engagement with non-digital worlds such as those brought to life in novels, which I describe in a new anthology that will be published this year.

We need to feel present somewhere because, like any creature, we must be able to act in a world and survive in that world. If this were not the case, our ancestors would have been gobbled up by all those saber-toothed tigers.

Are you not also present somewhere, right now, looking at your computer monitor or smartphone, sipping your coffee or cola in your office or bedroom as you ponder my words?

As we inexorably are dragged deeper into the digital domain, presence will become an increasingly important topic.

We create the place in which to be present

I should state at this point that I am a constructivist to some degree.

That is, reality is the experience of a perceptual environment that our brains dynamically construct from several factors including:

· Sensations—what arrives at our bodies from the world.

· Experience and knowledge—how we have learned to interpret sensations.

My environment and your environment have a common basis—presumably we both understand what red is when we see it as opposed to green—and we know from experience that diving into a swimming pool will not be like falling onto concrete.

But we are also individuals with different sensory abilities.

Some of which we are born with and some of which change as we age or our bodies are abused (beware those listening for several hours a day at high volume to music on earphones!).

And we also have different experiences and so associations that arise in relation to certain sensations.

For example, I feel disgust when I see a banana let alone smell one—a long story to do with the pet chameleons I once had — but I imagine, strangely enough, that most of you find bananas delicious.

Equally, you cannot occupy the same physical space at the same time as me.

So you will never get precisely the same sensations and resulting perceptions as I do even though you might be standing right next to me.

Reality is a construction

Here's something to think about to back up my constructivist stance.

It seems perfectly logical to suppose that our brains process light waves before they process sound waves from an audio-visual event that is within 10–15 meters away.

After all, the speed of light is 299.292.458m/s while the speed of sound in air is a paltry 344m/s.

Clearly the image we perceive will be present in our consciousness long before the sound wave has even arrived at our ears!

Well . . . no. Scans of the brain's processes demonstrate that we perceive the sound of a near audio-visual event before we perceive the image—the difference can be as much as 40msecs.

Yet the end result is the perception of one event.

There can be several reasons for this.

Perhaps more processing is required to produce the image or perhaps the limited visual horizons of our prehistoric ancestors meant it paid to attend to sound first in a predator-filled jungle environment.

Whatever the reason, it shows that not only does the brain construct the single perception of the event from radically different sources (light waves and sound waves), but the brain is also playing around with time as it constructs the perceptual environment.

Consistency and coherency are key

Our feeling of the here and now, reality, is merely an illusion—our brain is in control, assessing and organising and presenting something quite artificial for us to experience.

But the brain is also matching sensation to our knowledge of the world and so is interpreting new sensations in such a way that a perceptual environment can be modelled. It is in this environment that we have our feeling of presence.

If an elephant at the zoo squeaks like a mouse, then that is inconsistent with our experience, not only in the specific sound we expect of an elephant but also with our experience that large objects tend to produce deeper and louder sounds.

What we see and hear must form a coherent event.

No surprises please—I need an environment that can be easily and quickly modelled according to my experience.

We can be flexible

We can be flexible regarding consistency and coherency, and our brain is quite adept at sticking disparate sensations together to form one perceptual event.

Even when our technology interferes with the natural order.

For example, if I were in front of you now, speaking, the sound waves quite naturally emanate from my mouth.

However, if you were in a cinema looking at me speaking on the screen, the sound waves emanate from loudspeakers positioned some way from the screen and usually to the sides, but we still perceive the one event centred on the screen.

In both cases, we are perfectly capable of creating a coherent model of the world, the environment, by re-ordering and re-positioning various sensations.

Even when sensations are conflicting

We can also be flexible when we receive conflicting sensations.

For instance, where are you present when you don a VR headset but the sources of the sensations you receive, light and sound waves, temperature, touch, smell, and so on, are spread across both the digital world and the real world?

This is a matter of two things: Directed attention (I want to play the game so all sensations from elsewhere are kept on the periphery) and of coherence.

The last means that if the sensation of my feet on the real bedroom floor does not match what I am seeing and hearing, then that sensation does not form part of my perceptual environment.

What is a good digital presence?

Let us return to the two scenarios I presented at the beginning.

The first-person perspective computer game, despite utilizing only two senses, does so in a consistent and expected manner, because it is designed to be similar to our perceptions of sensation in the real world.

My own actions on the mouse and keyboard have a very perceptible effect on the game world and its characters.

I am present because I am in a place and able to act within and upon that place.

The video conference, on the other hand, while also using just two senses, has not been designed with presence in mind.

There are too many conflicting spaces (visual and acoustic) and it's not even possible to look a colleague in the eye and have them look back at you (try it).

It's as if each of you is utterly unaware of the other.

The artifice of the situation gets in the way of any possibility of modelling a coherent environment for presence.

It's like watching a film directed by 20 different people each with their own style and equipment: bewildering, exhausting, and certainly no place in which to be present.

What makes the best digital presence?

If we are to feel present in a digital world, then the sensations produced by that digital world must be sufficient to model a perceptual environment from.

To do this, the digital world must provide consistent and coherent sensations—the elephant always trumpets and the mouse always squeaks and they must sound as if they are in the same space and look as if they are under the same sun. In this, the computer game in scenario 1 succeeds spectacularly while the video conferencing of scenario 2 fails miserably.