From silent dialogues to vivid memories – here’s how the science of inner experience could transform gaming

But the gaming industry has also been facing challenges. The market is shrinking from its peak at the height of the pandemic, and there has been a rash of layoffs and studio closures.

In this uncertain environment, game developers are, more than ever, looking to create experiences that stay with their players. A great deal of effort goes into visual elements, including the appearance and overt behaviour of characters, for example. But at the level of subjective experience, the inner worlds of gaming characters are not so often explored.

This is where the science comes in. My research over the last 30 years has been about inner experience: the things in our minds that we are conscious of, such as thoughts, memories, inner dialogue, visual imagery, feelings and emotions. Traditionally considered impossible to study because of its private nature, inner experience is becoming established as an important field of cognitive science. And it has the potential to transform gaming.

In so many ways, video games just keep getting better and better. With the shift toward mobile gaming, the question for studios has become less about how to persuade people to buy this product (they may be downloading it for free) but to keep coming back to it.

Graphics are becoming ever more vivid and lifelike. What studios are doing with audio design is stunning. The player feels they are actually there in that medieval village or that rainforest or on that spaceship travelling between galaxies. And yet a common criticism of many games is – still – that they are just like watching a film.

That’s a puzzle, because of how gaming, more than other media, creates so much potential for a truly interactive experience. You are not just watching that ship sailing across the galaxy; you are captaining it. You can choose your body shape and physical skills, and see them depicted there on the screen in astonishingly lifelike detail. But these qualities are mostly on the surface, at the level of appearance and overt behaviour.

How could gaming go deeper into inner experience? Here’s one example. Many people report having a silent, internal conversation with themselves for much of the time. Our research has shown that inner speech comes in several different forms and has varied functions in thinking, planning and emotion regulation. But when inner speech is depicted in video games, it tends to lack the qualities and variety that make the experience so different between people.

Another example is the kind of memory we have for the events of our own lives. Autobiographical memory can take different perspectives, vary in vividness and show a range of multisensory qualities. Memory does not work like a video camera, but instead brings together sights, sounds, smells and other kinds of information in a dynamic, endlessly shifting way. We are even beginning to understand how these different qualities of memory are realised in the brain.

Opening the black box

Making a game is fundamentally about creating an experience – seeding an experience in the mind of its player.

When we do get to share a game character’s inner experience, it tends to lack the variety and nuance that the science tells us is there: the different qualities of inner speech, the various features of memory and visual imagination. The inside of a game character’s mind is often a black box.

There are of course exceptions to this rule. In Disco Elysium, for example, you can play with the main character’s thoughts and mental attributes in an unusual way. This text-based game was, however, limited in what it could do to depict the subjective qualities of inner experience – the colourful pageant of the everyday mind.

To give an example of a game I consulted on, Hellblade is the multi-award-winning story of an eighth-century Pictish warrior, Senua, whose inner experience is distinctive in that she hears voices and has other unusual perceptions and beliefs. In the first game, Senua’s Sacrifice, and in its just-about-to-be-released sequel, Senua’s Saga, Senua experiences psychosis, and you as player share these experiences with her.

I think the gaming industry can only benefit from understanding inner experience better. What I hope we’ll see is a more fluid, realistic, immersive gaming experience where the inner worlds of gaming characters live and breathe as much as their actions in the game world.

As well as having the potential to create new, interesting and memorable takes on the gaming experience, this work has real-world implications for accessibility, mental health, neurodiversity and sensory inclusion.

In fact, our starting point is that there is no such thing as a “normal” mind – we are all different, and our own minds differ from moment to moment. Rather than just taking your own mind into the gaming experience, gaming provides an exhilarating opportunity to experience a different kind of mind while you are there.

Gaming has incredible potential to work for good. Most of all, it’s not about some worthy educational ideal. Games are – and should be – about having fun. Beyond opening up creative possibilities for game developers and players, I believe that knowing more about our own inner experience can be helpful, restorative, even therapeutic. Gaming is a powerful way to push that ideal forward.

Charles Fernyhough, Professor of Psychology, Durham University

This article is republished from The Conversation under a Creative Commons license. Read the original article.