---

“You are a multisensory being. So why is it that so much of our media only play to one or two of those senses? We are engaging more of our body, more of our brain and we are creating a more holistic, more human experience”, Charles Melcher (2016).

---

In the past, the majority of artworks were displayed, focusing solely on visual presentation, as we can see from paintings and sculptures; the emphasis on sight making the audience pose merely as passive players. Leaping forward to the present, progress and technological advancement as well as interactive artwork allows people to view art not only through the form of visual experience, but through a combination of many, known as “multisensory experience”.

Figure 1.0; the artwork in Belvedere, Vienna, and Tate Modern, London. (Harrnpinijsak, 2019)

The important impact of this is that it has shifted the role of the audience from being a passive to an active player in the midst of art. This role change means that an interaction between both artwork and audience has been developed for the senses. Furthermore, they are now sometimes invited to be a part of the experience employing all the senses. This realisation has led to artworks being utilised by many artists throughout the world to evoke a new form of experience.  For example, the work of Olafur Eliasson, the Danish-Icelandic artist revolves around the form of sensations. In many of his works they were designed to make the audience more aware of their senses as never before. The main factor in making such artworks possible is the use of apparatus resulting from technological development, thus, leading to new ways of designing multisensory experiences, such as interactive technology, augmented reality and virtual reality. Distinctive examples from this can be seen in the work of Marshmallow Laser Feast, which draw people into an immersive experience in the virtual world.

In order to design an immersive experience artwork sensory design has, without a doubt, the most impact, “sensory design activates touch, sound, smell, taste, and the wisdom of the body. The sensory design supports everyone’s opportunity to receive information, explore the world, and experience joy, wonder, and social connections, regardless of our sensory abilities.” (Lupton, E. and Lipps, A., 2018, p. 9).  In response to this, Charles Spence an experimental psychologist who has contributed most to the understanding of sensation and perception has said that “what we see ultimately determines what we perceive, even when the other senses may be sending our brains a different message.” Nothing could be more real than understanding vision as the dominant sense, which can influence other senses and affect overall perception (Spence, 2012). Yet, on the other hand, Juhani Pallasmaa a Finnish architect and former professor of architecture and dean at the Helsinki University of Technology argued that “vision separates us from the world, whereas the other senses unite us with it. (Pallasmaa, 2014) So what if this distinctive sense, the sense of sight, is diminished, are the other senses stimulated?

Correspondingly, in this paper, I will discuss two different viewpoints, and consider how sensory interaction affects perceptual experience in an immersive artwork. This is because the main objective of my research is not only to gain insights into sensory interaction and multisensory perception but also to understand immersive and sensory environment in order to design such an immersive experience. I shall attempt to analyse and answer the questions of how multisensory perception works, how one sensory input impacts another, how sensory interaction works, how sensory information affects people’s feelings and what happen to the other senses if one is shut down.

In Chapter 1, I will explain some background based upon literature on multisensory perception and give some examples of sensory interaction, as well as reviewing a case study of multisensory art experience, Tate Sensorium. For Chapter 2, my experiment through the design project, Translator will be illustrated. Chapter 3 will be a demonstration of how visual deprivation affects sensory augmentation at the exhibition and a commercial context will be demonstrated using the case studies of Dialogue in The Dark and Dining in the Dark. Moreover, two more exciting case studies in immersive artwork will be presented: Your Blind Passenger by Olafur Eliasson. This leads to the development of a design project named Sounds, and Sweet Airs in Chapter 4. In the final chapter, conclusions reached from this research will be debated.

Chapter 1: Multisensory Inputs and Perception

---

“The senses move us through the space. Our sense organs are connected to a head that turns, arms that reach, and bodies that wander and seek. Sensory experience hits us from all directions.” (Lupton, E. and Lipps, A., 2018, p.10)

---

Figure 1.1;  Sensory experience (Joy Monice Malnar and Frank Vodvarke, Ranges of the senses, from Sensory Design, University of Minnesota Press; 2004)

In everyday life, people understand and perceive the world with their senses in order to receive sensory information. Sensation is activated by the receptors (forms of neurons activated by stimuli) located in the eyes, ears, skin, nasal cavities, and tongue. The five basic senses are sight (visual sense), hearing (auditory sense), taste (gustatory sense), smell (olfactory sense) and touch (somatosensory sense). In addition to these senses, there are other sensory modalities, including the sense of temperature (thermoception), the senses of movement and position (kinesthesia and proprioception), pain, balance, vibration, and various other internal stimuli. (‘Sense’, 2019; Ciccarelli, S. and Noland White, 2017). The insights from understanding senses can have a crucial impact on our lives. Ciccarelli and Noland White (2017, p.89) claimed that “without sensations to tell us what is outside our own mental world, we would live entirely in our own minds”.

1.1. Sensation and Perception.

Sensation and perception are interrelated and cannot be distinguished from one another if the physical world and beyond is to be understood. Thus, to understand ourselves and the world both sensation and perception are required.

I will first give an overview of what defines the two and how they lead to sensation interaction.  First and foremost it must be realised in this regard that senses are encoded within an organism to receive and interpret sensory information. Sensory signals are united and unified to form a perception of the world,  perception being the conscious form of sensory experience (Goldstein, 2009; Harris et al., 2015).

First of all, sensation is defined as the process of organising and interpreting the sensory data we have received which later transfers to our brain. In detail, it begins when a person processes visual information in the brain, and then visual information signals are sent to other neural sites (Damasio, 2000). Perception is the process of organising and interpreting sensory information, thus enabling us to recognise meaningful objects and events.

Consequently, there are two main approaches to understanding theories of perception or how human beings perceive the world. The first process is bottom-up processing, which primarily refers to the quality and content that sensory input plays in determining further sensory data processing in our cognition.

For instance, when we perceive a tree, our sensors detect and collect data such as points, horizontal and vertical lines. Later, we assemble them into the complex perception of an object identifying it as a tree. (Démuth, 2013; Ciccarelli and Noland White, 2018). In short, it is a process involving the analysis of small features and information to build up a complete perceived picture (Ciccarelli and Noland White, 2018). Bottom-up processing is also known as data-driven processing since perception begins with the stimulus itself. McLeod (2008)

Contrarily, the other theory of a top-down process indicates that perception is possible only using mental representations, creating pictures of given reality. In summary, “the top-down process utilizes preexisting knowledge to organize individual features into a unified whole” (Ciccarelli and Noland White, 2018). For example, handwriting that is hard to read is more accessible when reading complete sentences than when reading single and isolated words. This is because the meaning of the surrounding words provides a context to aid understanding (McLeod, 2008, p.1). From these explanations, it could be understood that sensation is the primary process of receiving the stimuli from the physical world, whereas perception is associated with memory and experiences in a person effecting a higher context of mental exploration.

Traditionally, the human sensory systems (e.g., vision, audition, and olfaction) have been viewed as separated entities, but, in fact, information extracted from a single object is experienced across different senses, which becomes perceptual experience (Persson, 2011).

1.2. Multisensory perception

In the past, a common belief regarding personal sense was that vision is used for perceiving material objects; hearing or audition for the perception primarily of the sources of sounds. However, to perceive objects, it involves not only seeing but also hearing things in the external world. Today, scientists shape the understanding of sensory perception as multisensory. Matthew Nudds is a British philosopher and Chair of the Faculty of Social Sciences at the University of Warwick who specialises in work on the philosophies of mind and perception. He sheds light on the science of multisensory perception, showing that perceptual experience operates across all our senses simultaneously resulting from not only one sense but rather through many (Nudds, 2014) More than one sense in perceiving one particular object is being utilised. Processing information via different senses guides how data is processed from the same object. Therefore, the traditional understanding of how senses operate should be re-evaluated. Regarding his direct claim, Nudds highlighted that “multisensory perception cannot simply be the combined operation of each of the individual senses, but that some kind of integration of information across the senses is required.” (Nudds, 2014, p.20).

The Finnish architect, Professor Juhani Pallasmaa, who is interested not only in the realm of architecture but also in the exploration of philosophy, also firmly believes that our body is not only a mere physical entity, but is filled with memories and dreams, as well as our past and future. The area of multisensory experience piqued his interest in attempting to fuse the theory of sensory experience with how art and architecture are perceived. He pointed out that our eyes are the foundation of how senses and body collaborate; more importantly, our sense of reality is strengthened and articulated by this interaction (Pallasmaa, 2014).

At the heart of his research, Pallasmaa explored how multisensory experience is portrayed in architecture and how people sense it and interact with it. Human experience of the world is not limited to the use of one sense at a time, but uses many at the same time, interacting, integrating and influencing each other. He therefore, drew correlations among senses where vision had revealed what the sense of touch already knew. Moreover, the sense of touch projects vision to the subconscious and scent elicits memories of sights, imagination bringing them together.

In return, we humans are in constant dialogue and interact with the environment and are the entities experiencing art which integrate themselves into this art as well. He wrote “architecture strengthens the existential experience, one’s sense of being in the world, and this is essentially a strengthened experience of self” (Pallasmaa, 2014, p. 45).

The question of why the subject of multisensory experience is essential to this paper is far beyond only understanding how senses associate. Driver and Spence (2000) stated that our brains continually integrate the information from multisensory environments that surround us. It is a process of information integration from different sensory receptors that generates a unified multisensory perceptual experience that fills our everyday lives.

1.3. Sensory interaction (Crossmodal interaction) 

Crossmodal perception has been studied by Charles Spence at the Oxford Crossmodal Research Laboratory. He is a professor of experimental psychology and has stated in recent years that studying the five classic senses of vision, hearing, touch, smell and taste in isolation is less effective than applying crossmodal principles, as interaction takes part in sensory processing suggesting it is modulated by the interaction of information from and to the other senses. As a consequence, Spence et al. (2009) pose questions asking why the voice of the ventriloquist appears to come from the lips of his/her dummy or why food loses its taste when your nose is blocked, or whether the blind hear better or the deaf have enhanced visual sensitivity. I will present two examples focusing on visual-auditory interaction. By studying this, it helped me design a better experience in my first project, Translator, which plays with two main senses, sight and hearing.  The way an audience perceives the sound is affected by whether they have their eyes open or shut. I will explain more in detail how this works in my projects , Translator as well as Sounds and Sweet Airs, in Chapter 2 and Chapter 4 respectively.

1.3.1. The McGurk Effect

The McGurk effect is one of the most eye-opening effects of how visual and auditory stimuli affect our perception of psychology. Psychologists Harry McGurk and John MacDonald (1976) conducted this experiment, referring to it as “hearing lips and seeing voices”. They first recorded the sound “ba”   which was then dubbed on to lip movements for “ga” on the video. In the experiment, participants were asked to watch the video and answer what sound they had heard when watching the video. The twist here is that they heard “da”, and neither “ba” nor “ga” (Tiippana, 2014). Tiippana (2014) added that the McGurk effect happens due to multisensory integration, which in return results in an altered auditory perception, and if this integration does not happen, participants would likely respond to either what they hear or what they see.

The reason this experiment is at the centre of attention in the area of perception is because it portrays a crucial aspect of how speech is perceived. Before this, it was regarded that speech perception was a purely auditory process, however, this experiment shaped the subject of multi-sensory integration.

1.3.2. The Ventriloquism Effect

The ventriloquism effect is another case of multi-sensory integration. The word ventriloquism refers to crossmodal interaction between auditory and visual inputs which are observed in sensory conflict situations (Bertelson, 1999). In other words, this effect can be seen when we perceive that sound is coming from a different location to where the sound is produced due to the influence of visual  information. In most cases, a person (or a puppeteer) is equipped with his puppet in his or her hand.  The speaker talks with an immobile mouth while at the same time moving the puppet’s mouth and facial expressions according to what he or she is speaking about, as if the sound was being produced by the puppet itself. This ventriloquism effect is a great example of visual capture. Auditory information received in this case is challenging since the cues are distorted by reverberation in the environment, and in humans at least, the auditory system is inferior to the visual system (Callan et al., 2015). I will explain this effect in both my projects, Translator as well as Sounds and Sweet Airs, in Chapters 3 and 5 respectively.

Figure 1.2: The McGurk effect (BBC, 2010) and The Ventriloquism Effect, Ray Alan and Lord Charles (1954)

All in all, the two effects of multisensory integration are not only integration and influence, but also show that sometimes, one sense can dominate another, resulting in the distortion of our perception.

1.4. Bottom-up and Top-Down Processing

A mention of the two most renowned psychologists James J. Gibson and Richard Langton Gregory cannot be neglected as they are the contributors to how we, today, see perception theory and how it has shaped the way we view the world.

Fundamentally, Gibson´s bottom-up theory is concluded from a visual system by ever-increasing the complexity of analysis of raw sensory data, claiming the environment contains all of the information needed to specify its properties. Hence, perceiving these properties is a matter of detecting the information available in the environment (Gibson, 2015, p.140). Gibson focused solely on the biological aspect of what perception is, relying on ecological aspects. However, what makes his theory most controversial is that he avoids touching upon the difficulty of explaining how the mind organises sensation stimuli.

Gregory’s top-down theory argued that stimuli received from the environment lose their properties over the time it takes to reach the brain. Therefore, we must rely on our cognition, memory, and experience to construct our reality. There are however, two views of the science behind perception.

Figure 1.3;  The comparison between Bottom-Up and Top-Down Processing (Harnpinijsak, 2019)

There is a clear distinction between the two with Gibson believing that perception is a direct process between the environment and ourselves without any learning or interpretation necessary along the way in order to specify reality. Gregory on the other hand, firmly believed that the external information received is not sufficient to portray reality, and experiences and memory play crucial roles in determining how the world actually is. I shall explain this in Chapter 3 in which I describe my design project experiment, Rain Translator, and I will give some concrete examples of how it works.

1.5. Sight as a dominant sense.

---

The perception of sight as our most important sense is well-grounded in physiological, perceptual and psychological facts. (Juhani Pallasmaa, 2005, p.39)

---

Sight is the most obvious necessity for viewing artwork; it is the first step in drawing people into experiencing a work of art. This claim is also supported in neuroscience by Jennifer K. Bizley and Andrew J. King (2012). The scientists explained that visual stimuli influences other senses in numerous ways, one of them being when neurons send signals from and to the brain which later distribute to the auditory cortex for the identification of possible physical functions. In theory, the influences could emerge from direct projections or be inherited from multisensory nuclei; another possibility is that multi-senses associate with other areas in the temporal, parietal, and frontal cortex.

Bizley and King (2012) have also underlined the case of how visual perception has a crucial impact on the evaluation of flavour with Wheatley (1973) suggesting that there is a strong correlation between colour and pleasure which is derived from food. The experiment utilises colour-masking lighting conditions, starting with a group of people eating a meal of steak, french fries and peas, however, halfway through the meal the actual colours of the food were revealed showing a blue steak, green French fries and peas. As a result, Wheatley concluded that vision was sufficient to make people feel nauseous and that it also has a profound influence on the other senses. Spence (2010) also commented that altering the colour of food and drinks can have an impact on taste and flavour. Another study we should consider is by Zampini et al., (2008). Zampini and his colleagues conducted one experiment in which they changed the hue of a drink. Cherry-flavoured drinks were coloured green and orange and the intriguing result suggested that participants experienced tasting lime and orange due to the altered colours.

Obviously we rely on our perception in viewing and comprehending the world with its objects, events, memories, and experiences which result from the integration of sensory input that simultaneously evokes reactions in our sensory systems (vision, audition, touch, taste, and smell) (Zampini and Spence, 2012). The reason why I mention this example is to illustrate that not only vision can affect sound and vice versa, which I mentioned before, but vision may also influence other stimuli: auditory, gustatory, olfactory and somatosensory. So evidently vision dominates the other senses and the way one sense interacts with another could affect our perceptual experience in many ways.

1.6. Multisensory experience in art museum reviews

---

“Stimulate your sense of taste, touch, smell and hearing in this immersive art experience” Can taste, touch, smell and sound change the way we ‘see’ art? (Tate, 2019) 

---

In the general setting of an art museum these days, the phrase ‘do not touch’ is the boundary between artworks and the audience to prevent people stepping close to the exhibits. I ask why it is only one sense that is provided for in a gallery setting, while others are dissuaded?

Figure 1.4; David Adjaye, Making memory at Design Museum. (2019)

Penelope Curtis (2014, cited in Beanland, 2014), former director of Tate Britain, stated that “our challenge is to enable people to see and feel art in ways that maintain the primacy of that experience.” Likewise, she believed that “looking at art slows us down and takes us in unexpected directions: this is increasingly unusual — and something people cherish.” It is undoubtedly true that spending time in an art museum allow us to slow down and appreciate the artworks.  Only seeing an exhibit can, however, not attract people to stay and slow down as they are so often distracted by electronic devices such as mobile phones, and lose concentration on what is before them. Speaking personally, I wonder whether the direction that art in art museums is taking, could be developed in new ways to engage people and evoke all of the human senses. This has led me to study some of the experimental works on multisensory design. A single, but striking, example of this is the Tate Sensorium project. An aspect of multi-sensory, immersive design in an art museum has been investigated by Tom Pursey and David Lomas of Flying Object, London. This project was exhibited at Tate Britain in 2015. In order to evaluate visitors’ experiences, the team from the University of Sussex, Computer Human Interaction Lab took part in this project as well. I will consider their research paper called “The How and Why behind a Multisensory Art Display” (2017), as a case study.

The designers, Tom Pursey and David Lomas, wanted to explore and enhance the experience of visual art by adding  sounds, taste, touch and smell. The aim of this project was to design an art experience for museum goers that was immersive rather than isolated, and that was more explicit and more memorable by engaging all the senses, not just vision. (Pursey and Lomas, 2018) Thus, there were four different artworks presented in this exhibition. Basically, each painting was presented using two main sensory stimuli, vision and sound. Later other senses were added according to the designers’ plan see Figure 2.7 and Figure 2.8. According to Ablart and his team (2017), they collected feedback through questionnaires and an interview. More than 80 percent of visitors assessed the experience as being very appealing and showed interest in returning back for more multisensory experiences. (Ablart et al., 2017)

Figure 1.5: The four paintings and their multisensory components included (from left to right): Interior II by Richard Hamilton, Full Stop by John Latham, In the Hold by David Bomberg, and Figure in a Landscape by Francis Bacon. (Photograph: Joe Humphrys/Tate Photography, 2015)

 

Figure 1.6: Selected paintings and their associated sensory stimuli (Ablart et al., 2017)

All in all, the benefit of multi-sensory experience in the exhibition, Tate Sensorium, tended to increase audiences’ engagement with this new experience of viewing artwork, compared to traditional art galleries and attracted expansive publicity not only in the UK but also around the world (Ablart et al., 2017)

Interactive technology, expands the boundaries of design, enabling the development of artwork in the areas of interactive media and public engagement. Personally, I find this approach very interesting and could lead to  potential development of immersive artwork in an art gallery context, using interactive technology in multisensory design. The painting Figure in a Landscape by Francis Bacon which was exhibited with four sensory stimuli, drew my attention to how a multisensory experience should be presented. I wonder whether it was really necessary to present such an experience using so many sensory stimuli. In my case study in Chapter 3, I will explore this topic from a critical, yet curious viewpoint, asking what would happen if only one sense was used in order to prompt a reaction from other senses.

Chapter 2: Translator

“One evening last November, while walking back from Shoreditch, it started raining. I stopped by the road listening to the sounds, and all my memories of Bangkok unfolded through the drips and drops. I recalled the dark lonely street where I had been caught when it poured… remembered some warm moments with my friend, hearing the rain pelting on the metal roof… thought of being sheltered in front of shops and stalls… Fragments of different scenes and emotions all came together, I was overwhelmed by the downpour within me. This is the rain in my memory, and this is the feeling that I want to tell. But since no one can experience my experience, I would like to share my perception, allowing people to hear and feel it in their own ways. After all, everyone has a different journey. Here, I recall my past perception of rain sounds and translate my personal experience through this instrument. Other people will have their own ways of decoding stories.” (Harnpinijsak, 2019)

 

Translator performs as a medium, translating the sound of the rain from Bangkok to London to elicit sensory experience by using a mechanism and different kinds of materials to generate the sound. The installation does not reproduce the sound of rain, but instead focuses on representing a personal memory of the rain in my home, Bangkok, by using a range of mechanical instruments to compose different layers of rain. Due to this common basis, every listener’s interpretation is unique, the downpour unfolds within the imagination of the audience.

In Chapter 1, psychology, philosophy and neuroscience aspects were explored. This project will examine multisensory design where just one sense — sound — is provided. In order to understand how sensory interaction affects perceptual experience in immersive artwork, my teammates and I designed an experiment using our installation, Rain Translator to evaluate some aspects of the theoretical basis. In this chapter, the feelings aroused in people reacting to sensory input will be discussed.

Apart from providing a stimulus for the sense of hearing, we also shut down one primary sense, which is vision. In Chapter 1, I mentioned how vision as the dominant sense. (Pallasmaa, 2005) In this project, we challenge the audience’s senses by initially depriving them of their prevailing sense, and let them participate using their other senses before finally revealing an artwork. The project works as follows; we designed the sequence of an experience by shutting down one sense in order to evoke others. Instead of exhibiting the artwork itself, we used another design approach to draw the audience into this installation.

---

“To design for all the senses: start with a blindfold.” (Mau, 2018)

---

Figure 2.1; Putting on a raincoat and blindfold glasses (Harnpinijsak, 2019)

For the whole experience, see Figure 2.2, first of all, we asked the participant to put on a raincoat and blindfold glasses without saying what was about to happen or where they were about to be taken. Secondly, they were then guided to the 3D sound space led by a staff member. Thirdly, the sound performance produced by the mechanism in 3D sound space was then enjoyed with the participants still blindfolded, but otherwise left to themselves. Later, the glasses were removed and the machine that had produced the sounds they had just listened to in the 3D sound space was revealed, at which point they were also visually engaged. Visual/auditory interaction comes into effect and the sounds they hear link with visual experience. A short discussion followed in order to interview participants and this will be explained at the end of this chapter.

Figure 2.2; An Experience in Translator (Harnpinijsak, Tang, Lim, 2019)

To design a sensory experience, the theory of sensory interaction has to be considered. I mentioned multisensory interaction in Chapter 1. Obviously vision is a dominant sense and the senses of vision and hearing interact very closely. Spence et al. (2009) The McGurk effect shows us how the visual system can affect the work process of the auditory system and in our project we demonstrate this by initially withholding sight and not allowing both vision and hearing at the same time. Harry McGurk and John MacDonald (1976) Applied to this project, we gave some visual cues before the experience started. This led to the effect visual memory has on auditory perception when we shut down vision.

“I have translated my own experience through the artwork. In this case, I brought my own perception of rain from my memory of Bangkok to allow people to experience this sound. From an artist’s perspective, I wonder how other people´s perception is different to mine.” (Harnpinijsak, 2019)

Figure 2.3; The Diagram of Sound Layers from my Perception (Harnpinijsak, 2019)

Figure 2.4; Definig Four types of Machanism (Harnpinijsak, Tang, Lim, 2019)

Apart from the sensory experience in this exhibition, we also interviewed participants. The aim of this project was also to understand sensory perception and how sensory interaction works. How do sensory inputs linked with feeling lead to a memory? We therefore conducted short discussions with the audience at the end of the experience to study their perception/perceptual experience.

Figure 2.5; The interview’s result (Harnpinijsak, Tang, Lim, 2019)

---

“Prior experience tells our brains what to expect.” (Ellen Lupton and Andrea Lipps, 2018, p.10)

---

Results from the first question showed that according to audience’s different memories and experiences, they tended to perceive the sound of the rain differently. The audience knew nothing about this experience, so that is why the experiment showed that different people have different interpretations of what they perceive. In our case, Western people were more likely to think of a waterfall rather than rain. On the other hand, audiences from South East Asia tended to recognise this sound as rain falling heavily on different materials. Theoretically, according to top-down processing (Gregory, 1970), with insufficient information, your brain will apply previous knowledge and past experiences to construct their own perception and then interpret in their own ways.

---

“Sound acts as a key, opening doors to reunite us with the past; it stimulates images that all of us have hidden various depths of our psyche.”(Augoyard and Torgue, 2005, p.22)

---

In answering the second question, as shown above, the participants showed individual feelings based on their subjective experiences. How does this feeling happen? Once they receive sensory stimuli from the surroundings, they become aware of their own feelings. Damasio (2000) talks of how people feel. It is undeniable that human emotions are special in the way they associate complex ideas, values, principles, and judgments (Damasio, 2000). Therefore, he indicated different stages “from emotion to feeling”, beginning with the processing of sensory information in the brain, then sensory information signals are sent to other neural sites associated with areas inducing emotion. These sites then send signals to other brain sites for the body and the brain to create emotion which results as changes in the body. It is at this stage that feelings emerge (Damasio, 2000). The researchers also suggested that memory is enhanced by both positive and negative emotion, as well as having distinctive effects on cognitive processes (2008). In other words, individuals are likely to remember more emotional events better than non-emotional ones. (Linda J. Levine1, David A. Pizarro, 2004; Kensinger, 2008)

---

“Not only is this sound remembered, but all the other sensorial and affective components also cross the threshold of consciousness. All the senses may act as stimuli of anamnesis; sight, taste and smell as much as hearing.” (Augoyard et al., 2005, p.22)

---

As we had only provided the test persons with one sense, the sound of the rain, it was exciting to hear their answers to the final question, that this one sense could lead to the experience of other senses, in particular, smell. My view of this is that once vision is shut down, it opens up the imagination.

In conclusion, from this project I have learned that, as initially hypothesised, when the dominant sense is shut down, focus will be more on other senses with one sense sufficing in linking to personal sensory memory. Also, when the installation was revealed, audiences were able to match the sound they had heard in the 3D sound space with the installation in front of them. Thus it was shown that visual-auditory interaction works effectively when both sound and vision are synchronised. As a consequence, it can be seen that sensory interaction affects perceptual experience in order to transport people into their own experiences and link to personal memory.

I have learned three concepts from this experiment: firstly, based on different memories, experiences, and background, participants have different interpretations of what they perceive. Secondly, specific sensory memory, when related to positive or negative emotions rather than neutral emotions, reinforced subjective feeling. Thirdly, memory is linked with senses. In this experiment, sound could link to vision, touch and especially scent. As Lupton, E. and Lipps, A. (2018, p.10) stated “The senses mix with memory.”

From this case study, of all the results I got from the discussion, the most interesting was that  when the sense of sight is shut down, the other senses work better. This has led me to explore visual deprivation, which is related to sensory augmentation. How can senses be augmented in different ways and if one sense is shut down, what happens with the others?

Chapter 3: Visual Deprivation and Sensory Augmentation

---

 “Darkness helps us hear more clearly, shutting out visual signals can help bring other senses into focus.” (Lupton, E. and Lipps, A., 2018)

---

I mentioned in Chapter 1 how vision is our dominant sense in multisensory perception. (Zampini and Spence, 2012) Likewise in my report in Chapter 2 on Translator, I challenged people’s senses is this design project by blindfolding them and found that visual deprivation leads to an extraordinary experience and memorable sensory experience by evoking other senses. In this chapter, I will explore what happens to people’s perception if they are deprived of an essential sense, by giving examples.

John P.Zubek is a pioneer who wrote about sensory deprivation in his book named “Sensory Deprivation. Fifteen Years of Research”.  He discovered some of the most dramatic effects of sensory deprivation such as hallucinations and used them as a useful technique in the study of various topics such as brain function, stress, personality and motivational theory, as well as applied psychology.

However, in this paper, I will focus only on visual deprivation that augments other senses, as seen in our blindfolded experience. I will examine the results of the absence of visual stimuli in an exhibition context, Dialogue in The Dark, and the unique dining experience of, Dining in the Dark or Dans le noir. Later, the case study on visual deprivation by the artist Olafur Elisson, your blind passenger, will be touched upon.

How does the loss of one sense impact the others?

Case Study 01: Dialogue in the Dark

Figure 3.1: Dialogue in the Dark  (Photograph: Dialogue Social Enterprise,

https://www.timeout.com/melbourne/things-to-do/dialogue-in-the-dark)

Dialogue in the Dark is an immersive experience in absolute darkness. It is about educating human beings to understand the way blind people live their daily lives, employing many settings such as walking in the park, across the road during the traffic congestion and more. In this exhibition, visitors have to rely entirely on their other senses because of losing their sense of vision. They are accompanied by blind guides. Situations are reversed; the sighted becoming blind and the blind becoming sighted.

Since all senses are linked to each other as mentioned in Chapter 2, blind people interpret sensory information, linking it through cross-modal interaction. Even though they cannot see, they can envision their surroundings and events in their daily lives by using either top-down, or bottom-up processing. Donovan Tokuyama (2017) wrote in his article “Supersensors: How the loss of one sense impacts the others” that “The lack of sensory input causes a sequence of events within the brain that allows the other senses to take over the roles left unoccupied.” (Tokuyama, 2017)

“60 minutes in there made me so nervous, especially the first few minutes. I needed a lot of help but felt more acutely aware every single minute of that whole experience. Especially in the street situation; I smelled the cars’ exhaust, In this situation, my other senses were required to take in the surroundings. My favourite part was the café, full of the scent of coffee. It was good to drink without seeing. I focused on my sense of taste and smelled the aroma far more intensely than usual.” (Harnpinijsak, 2019) 

Experiencing Dialogue in the Dark and temporarily losing my sense of vision was, for me, an uncontrolled and unpredictable situation. It made me think again about my other senses and realise that I rarely use them to perceive the environment deliberately. Am I missing something in my everyday life? Am I neglecting to sense the environment in a more intense way?

Case Study 02: Dining in the Dark

Figure 3.2; Dining in the Dark (www.aspasios.com /blog/restaurant-dans-le-noir-unique-experience-dark)

My second case study, Dining in the Dark, is a unique dining experience in pitch darkness. A pioneer of the blind experience is Axel Rudolph, a psychologist. It is undoubtedly true that if you cannot see the food while you are eating, the other senses are heightened.  “You smell better, you are more receptive to differences in texture, consistency and temperature… it is a holistic experience” (Rudolph, cited in Read et al., 2011, p.16)

The reason for this being that closing one’s eyes has been shown to lead to a more interoceptive state of awareness (Marx et al., 2003). Thus, it could be argued that, if anything, any advantageous results on flavour perception during dining in the dark are more likely to be felt by those diners who choose to keep their eyes closed while eating and drinking. (Marx et al., 2003, cited in Charles Spence and Betina Piqueras-Fiszman 2012).

“Three months ago in London, I experienced dining in the dark. I felt as though I was in another world. I rarely use sound to navigate in my daily life, but did try to estimate how big the room was when I walked in. When my main course was served, I touched all the food before starting and tried to guess what was what and taste everything separately. I felt surprised every time I tasted the food and even more surprised when I recognised  what I had eaten during the last part of the experience.” (Harnpinijsak, 2019)

The only possible conclusion to be drawn from all this is that by being deprived of one crucial sense, in particular vision, this can heighten other senses during a dining experience and that people who close their eyes are likely to be more aware and perceive flavour better than people who leave their eyes open.

Case Study 03: Your Blind Passenger

Figure 3.3; Your Blind Passenger (Harnpinijsak, 2019)

Moving on to the third case study: what if vision is not totally shut down but only partially. Your Blind Passenger designed by Olafur Eliasson was exhibited at Tate Modern in 2019. This project is a 39 metre long tunnel installation, filled with fog, which provides visibility at just 1.5 metres. It was the artist’s intention to temporarily blind the visitors with brightly illuminated fog that requires them to rely on other senses to orient themselves in relation to their surroundings. As they progress through the corridor, they move through zones of different hues. “Our sense of orientation is challenged and the coordinates of our spaces, collective and personal, have to be renegotiated.” (Olafur Eliasson, 2010)

“when I was walking through the space, the first I saw was red light. I tried to see what was happening in front of me and felt uncomfortable as I could not see everything clearly at first. I could barely see the person in front of me, just as an outline and then he disappeared into the haze. Then I started listening to the sound surrounding me. I tried to position myself in the space by touching the wall. As time went on, I felt as though I was walking somewhere in summertime. In this long and narrow corridor, I felt rather hot. I then continued walking through different hues of lighting until, at the end of this tunnel, the light was white. I felt comfortable when I saw the exit door” (Harnpinijsak, 2019)

It can be seen that, visually, the colour of the lighting and the fog became an essential part of the surroundings in order to reduce vision and blur boundaries leading to a sense of awareness. All other senses awake when the most important sense cannot receive information clearly. Orange light in the first stage reminded me of being somewhere during the summer and thus also affected the temperature I perceived in the space.

Chapter 4: Sounds, and Sweet Airs (S&SA)

After having laid a theoretical foundation and presented some case studies, I will now put that into practice with my design project in which I will use technology to help in expanding an experience. This is a collaborative project with three specialised aspects. Within my specific area of interest, I will consider sensory experience in immersive artwork which is related to sensory interaction and will focus on visual deprivation and sensory augmentation, to create a new type of multisensory experience in an exhibition context. In this chapter I will explain two examples of this project which were exhibited in two different settings.  Moreover, the interaction between the audience and art installation in the space will be discussed.

  “Less visual given, more senses evoked.”(Harnpinijsak, 2019)

Figure 4.1; Walking into the space (Harnpinijsak, 2019)

---

“Be not afeard, the isle is full of noises. Sounds, and sweet airs, that give delight, and hurt not.”                           (Shakespeare and Kermode, 1998)

---

Sounds and Sweet Airs is a sensory design project, which reflects on natural sound ecologies to create a synthetic environment around human-technology interaction. We designed creatures to create a soundscape that reflects the state of the place in the physical world and extends a sensory experience into the virtual world. This should encourage people to rethink their relationship with the environment.

What I want to achieve in this artwork is that I want to activate people’s senses to be aware of themselves and their surroundings, allowing people to create an imagination and engage themselves with the world through the experience. In terms of developing sensibility, evoking feeling and also engaging. By applying the theory of sensory interaction, focusing on visual deprivation to augment the other senses. This would lead audiences to immerse themselves in the artwork.

In order to achieve this in our last project, Translator, we deprived our participants of the visual experience of our installation. In this project, we will not cut off all vision, but will provide some information. Once you cannot see things clearly, a sense of orientation becomes more important plus an understanding of the coordinates of the space. (Olafur Eliasson, 2010) You become aware of every single step you take, aware of the things surrounding you, aware of every sensory cue received, similar to the case in which I referenced Your Blind Passenger. As Damasio (2000) has mentioned, once you process sensory information in the brain an emotion occurs and then feelings emerge. Memory is then enhanced by emotion, whether positive or negative. In other words, once you are aware of your senses, this animates sensory memory due to the sensory cue and will probably guide you to a memory. As a consequence, I hope that the purpose of this project, which is to encourage people to rethink their relationship with the environment, will be executed in a few ways. Everything audiences hear, might not be the same as they see according to the ventriloquism effect in Chapter 1 where sensory interaction between auditory and visual stimuli distorts the localisation of the auditory source. Callan (2015) mentioned that the cues are distorted by reverberation in the environment, and in humans, the auditory system is inferior to the visual.  (Callan et al., 2015) So that is why even if sound comes from elsewhere, people tend to synchronise the source of the sound with visual events in the environment.

4.1. Interactive Artwork

We developed the technological artworks, which we, in this project, refer to as “creatures”. These creatures, physical objects in space, respond to audiences’ movement and interact with people differently. This leads to audience engagement, guiding participants to be active rather than passive players. The participants’ every movement will affect the surrounding sound; in this case, by using interactive technology such as sensors, a tracking system in the physical space, as well as the augmentation of human senses to mix reality with virtual sound. This virtual sound is composed using physical objects, which also reflect natural sounds in the virtual world.

4.2. Single Player and Multiple Players

There are two options to experience the artwork. One is that multiple players in the physical space. Another is that only one player in the virtual space overlapping with physical space; this is by wearing a provided headphone. According to the feedback, people who wore a headphone tend to focus more on listening. Nevertheless, people who did not wear a headphone tend to interact with the creatures and focus on objects in the space.

Figure 4.3; Single player and Multiple player (Harnpinijsak, 2019)

4.3. Designing the System: A Synthetic Environment with Creature Inhabitants

Instead of producing all sound within the space, we extended the experience into the virtual world, duplicating the surrounding soundscape and placing it in the virtual world. As the sound came from different directions in the virtual world, the less that participants could see of this in the physical world meant that they became more aware of their bodies in relationship to the installation. To create an emotional and imaginative experience, the associated sensations allowed audiences to engage with and immerse themselves in the environment. This is not just about seeing with eyes or hearing with ears but experiencing via the whole body.

 “What you see, it might not be the same as what you hear.” (Harnpinijsak, 2019)

Figure 4.3; Physical object and Virtual Sound (Harnpinijsak, 2019)

4.3.1. The interaction between creatures and human, also the environment

“When you move around, you will affect the sound surrounding you.” (Harnpinijsak, 2019)

The interactive component of the project takes place as the creatures interact with the participants. As they move around, they affect the creatures’ behaviour in the space as well. In this case, it is the sound of the creature which reflects their behaviour or could also be linked with the state of the space.  the table of creatures’s characteristic and their behaviour can be seen in Figure 4.4 and how we classified the creatures based on the sound in the environment can be seen in Figure 4.5

 Figure 4.4: Defining Four Types of Creatures Characteristic (Harnpinijsak, Tang, Lim, 2019)

Figure 4.5; Defining four types of the sound (Lim, 2019)

4.3.2. Application in different contexts

We created an experience as a system in which the creatures were exhibited in different contexts according to their characteristics and behaviour. However, the same mood and tone of lighting was set to be more imagination with pink and blue colours. The lighting become an important part to distort the physical space and create the sense of synthetic world in this project.

Context 1: FOLD Nightclub (Semi-public space) 

The S&SA exhibition is part of Prototypes in Public, which is the first outside exhibition of the Interactive Architecture Lab. S&SA exhibited in the locker room, which is the first room people walked pass. The size of space allowed more than one people, So it has a potential for both Single player and Multiple players

Figure 4.6; Layout Plan of the Overlap Experience between Physical and Virtual Space in FOLD (Harnpinijsak, 2019)

Context 2: Ars Electronica (Public exhibition space)

Ars Electronica is an international festival in the field of new media art.

Figure 4.7; Layout Plan of the Overlap Experience between Physical and Virtual Space in Ars Electronica (Harnpinijsak, 2019)

 

Figure 4.8; The setting and Lighting Design (Harnpinijsak, 2019)

Figure 4.9; The Interaction in Physical Space (Harnpinijsak, 2019)

Figure 4.10; Creatures’s setting (Harnpinijsak, 2019)

 

Table 4.1: The Audience’s feedback from Ars Electronica (Harnpinijsak, Tang, Lim, 2019)

As shown above, some people recalled their memory of “walk-in nature”, “childhood in the evening”. This is episodic memory that People felt that they could transport to somewhere via their senses. Some of the audiences reported their feeling as “relax” and “a bit scared”, according to Kensinger (2008) said that individuals are likely to remember more emotional events better than non-emotional ones. Moreover, the effect of sensory interaction has been seen from the feedback, “when opening the cabinet, not sure if the sound is from headphones or physical world.” According to the ventriloquism effect, this effect of mislocation could happen when sound and vision did not come from the same direction. One more exciting feedback, some people mentioned about robot since they saw physical creatures in the space and lighting and setting also affect their perception and mood. However, some people told that they did close their eyes during the experience to focus more on listening, similarly, Marx et al.(2003) mentioned in the case study of Dining in the dark. This has shown the effect of visual in different ways.

 

Figure 4.11; The Comparison between Rain Translator and Sound and Sweet Airs project (Harnpinijsak, 2019)

All in all, according to the design projects, I have learnt that different sensory design approach has a disparate impact on the audience experience. In Translator project, we gave only one specific sense, the sound of the rain. It could be seen that only one sense could lead to another, and the audience more focuses on only particular sense given with blindfold. From Sounds and Sweet Airs project, the audience can see some visual information in the context. Regarding the feedback, it has shown that visual engagement plays a vital role in audience perception, for example, someone said that “digital robot forest, a little bit scared, thinking that the spider will run out”. Visual perception evokes an imagination also linked to their previous memory with the spider. Moreover, the sense of scent in Sounds and Sweet Airs at Ars Electronica. Some people notice the smell, but it had no evidence. Secondly, the context affects the way people experience the artwork. Because of the size of space, the location as well as the environment. It could be seen in Sounds and Sweet Airs at Ars Electronica that fewer people allow in the space and with the limited space, people tend to spend less time and less interact with artworks in the exhibition, compared to FOLD. Lastly, interactive technology, according to the feedback, people reported that enjoy communicating with interactive artworks in Sounds and Sweet Airs. Regarding the technical problem, both virtual sound and interactive artwork in physical space worked very well at Ars Electronica. However, the virtual sound did not work at FOLD because of the unexpected problem with tracking systems.

Chapter 5: Conclusion

In this paper, I have discussed a sensory experience referencing an immersive artwork and have also studied the theories of multisensory perception, sensory interaction, as well as sensory deprivation. I applied this to my work through a series of explorations and experiments. As the results from my design project show,  sensory experience is something people can sense and feel, but cannot accurately describe. What I can, however, provide in this paper is an example of how sensory interaction affects perceptual experience through my immersive artworks.

The answers can be seen in my projects, Translator, and, Sounds and Sweet Airs. I interviewed people about their feelings after experiencing either, and all of their answers were perceptual; they elaborate on their sensory experience. People are perceptually transported to a rich environment through their senses. They explained how they sensed the environment by using metaphors such as feeling as if in a rainforest. As we did not discuss what a rainforest actually is, in a sense, none of this is semantic. The perceptual experience of a forest influenced people to experience this feeling. In the first project, Translator, there was little sensory interaction. Some aspects of rain mainly address a specific sensory channel. In my second project, Sounds and Sweet Airs, I provided more senses of the rainforest. I added the scent of forest and designed an environment with some visual information. However, I still did not create the whole rainforest. People felt that they could transport to a forest from their actual environments via their senses. By doing so, people involved in the experience who had had their senses evoked, completed the picture by using top-down processing. With inadequate data, their brains could apply prior knowledge or experiences to construct perception. It was all subjective experience as some people are not familiar with the rain of this kind. They might disagree, but their own personal experiences and their brains find a suitable memory and that is the one that is evoked. Moreover, individual memory has been identified, but it is sensory memory, which is linked to episodic memory. Even when people said “rainforest”, it differed from person to person. Everyone has their memories which take them back to remember unique events at a particular time and place.

As I am a designer, I design experience, but people receive from it what they feel themselves. They are complete in their personal lives which makes the experience something perceptual, which is more about feeling and senses, rather than conceptual, which is all about facts and knowledge.

In order to design sensory environments and immersive artwork in an exhibition context, there are some aspects I would like to suggest. Firstly, as in my research, I found that people immerse themselves more in artwork if one vital sense is cut off – visual, in this context – rather than providing direct visual information. This can induce humans to experience through their senses and travel to their own memories. Visual engagement, however, has to be taken into consideration, since what audiences see will also affect their perception. Secondly, apart from visual deprivation and visual engagement, context is a priority factor which should be considered well, especially in an exhibition context. This is because it affects the whole experience as we are multisensory beings. (Charles Melcher, 2016) This was discussed in Chapter 4 on two examples of my work in an exhibition context. The former was semi-public viewing in a commercial place, and the latter, public viewing at an international festival.  In these two separate situations on different scales, slightly different rules were required. If in public, I cannot control the environment and guide people properly,  then I have to design for that. I have learnt from my work. Moreover, I would suggest for an exhibition design that private viewing in a dark, silent space in which all lighting and sound can be controlled, is the preferable location for a sensory experience. A suitable atmosphere engages people with an immersive artwork better than a random atmosphere. Lastly, interactive technology can draw people into being part of an artwork as an active player, rather than just being passive. That is obvious in my second project, Sounds and Sweet Airs. I provided tracking technology and made use of many kinds of sensors such as a distance sensor, a temperature sensor as well as an IR sensor to design the sensory experience. Feedback showed that interactive technology is essential to exhibition design and this could expand its boundaries to create a more delightful experience and a sense of wonder when a participant spontaneously finds themselves part of an environment within our immersive interactive artwork.

My next step will be to develop an immersive artwork using slight visual deprivation and observe how vision engages with the other senses. This means the development of an installation design as well as the supervision of light and atmosphere in the space. Moreover, I will carefully choose the context using a dark, silent space so I am able to control light and sound. It could be either private viewing by invitation, or well designed in a public space. Lastly, I will develop the design for an interaction between an audience and an installation by using the theory of sensory interaction and consider the suitable kinds of interactive technology, which match with the design concept. Considering all aspects, I expect to develop my design for an immersive environment and engage audiences into this sensory experience. This could impact their senses in a way which they cannot experience in real daily life.

Even my research question started from an artist’s point of view. It also has the potential for further study from a curator’s point of view, in order to improve the ways of curating an exhibition to cover this whole field.

 

Bibliography

Ablart, D. et al., (2017). The how and why behind a multisensory art display. Interactions, pp. 38-43.

Augoyard, J., F. (2005). Sonic experience. s.l.: Quebec: McGill-Queen’s University Press.

Beanland, C. (2014). Is there a future for the traditional museum?. [online] The Independent. Available at: https://www.independent.co.uk/arts-entertainment/is-there-a-future-for-the-traditional-museum-9855822.html [Accessed 16 Sep. 2019].

Bertelson, P. (1999). Chapter 14 Ventriloquism: A case of crossmodal perceptual grouping. [Online] Available at: https://www.sciencedirect.com/science/article/pii/S016641159980034X [Accessed: 2, September, 2019].

Jennifer K. Bizley and Andrew J. King., 2012. What Can Multisensory Processing Tell Us about the Functional Organization of Auditory Cortex?. In: The Neural Bases of Multisensory Processes.. s.l.:CRC Press.

Braund, M. J. (2007). The Structures of Perception: An Ecological Perspective. [Online]  Available at: https://pdfs.semanticscholar.org/84fe/c80b149f90ba344e5f99bb743402d334dc09.pdf

[Accessed: 30, August, 2019].

Callan et al., (2015). An fMRI Study of the Ventriloquism Effect. [Online]  Available at: https://www.ncbi.nlm.nih.gov/pubmed/25577576 [Accessed: 22, August, 2019].

Chin, A. (2010). olafur eliasson: your blind passenger. [online] designboom | architecture & design magazine. Available at: https://www.designboom.com/art/olafur-eliasson-your-blind-passenger/ [Accessed 13 Sep. 2019].

Ciccarelli, S. and White, J. (2018). Psychology. Harlow, Essex: Pearson.

Damasio, A. (2000). The Feeling of What Happens. New York: Harcourt Brace & Company, P.35-57.

Démuth, A. (2013). [online] Available at: https://www.academia.edu/30094149/Perception_Theories [Accessed 27 Aug. 2019].

Dorment, R. (2003). A terrifying beauty. [online] Telegraph.co.uk. Available at: https://www.telegraph.co.uk/culture/art/3606332/A-terrifying-beauty.html [Accessed 1 Sep. 2019].

Driver J, S. C., (2000). Multisensory perception: beyond modularity and convergence. Current Biology, 10(20), pp. R731-R735.

Ebbinghaus, H. (1885). Memory: A Contribution to Experimental Psychology. 

Classics in the History of Psychology. [Online] Available at: http://nwkpsych.rutgers.edu/~jose/courses/578_mem_learn/2012/readings/Ebbinghaus_1885.pdf  [Accessed: 20, August, 2019].

Gibson, J. (2015). The Ecological Approach to Visual Perception. New York, N.Y.: Psychology Press.

Goleman, D. (1995). Emotional Intelligence. London: Bloomsbury, P3-19.

Henning, H. (1916). The quality series of taste (Die qualitätenreihe des geschmacks): Zeitschrift für psychologie Vol. 74: 203—219, 1916. [Online] Available at: https://www.sciencedirect.com/science/article/pii/0149763484900265 [Accessed: 26, August, 2019].

Kensinger, E. and Schacter, D. (2008). Memory and Emotion. [Online] Available at: https://www2.bc.edu/elizabeth kensinger/Kensinger_Schacter_HandbookEmo.pdf [Accessed 20, August, 2019].

Kensinger, et al. (2007).  Effects of Emotion on Memory Specificity in Young and Older Adults. [Online] Available at: http://scholar.harvard.edu/files/schacterlab/files/kensingergaroff-eatonschacter2007b.pdf [Accessed: 20, August, 2019].

Levine, L. and Pizarro, D. (2004). EMOTION AND MEMORY RESEARCH: A GRUMPY OVERVIEW. Social Cognition, Volume 22, pp. 530-554.

Lindemann, B. (2001). Receptors and transduction in taste. [Online] Available at: https://www.researchgate.net/publication/11789886_Receptors_and_transduction_in_taste [Accessed: 26, August, 2019].

Lupton, E. and Lipps, A., (2018). The senses. s.l.: New York: Cooper Hewitt, Smithsonian Design Museum.

McLeod, S. (2018). Visual Perception | Simply Psychology. [online] Simplypsychology.org. Available at: https://www.simplypsychology.org/perception-theories.html [Accessed 2 Sep. 2019].

Mau, B., 2018. Designing LIVE. In: The Senses. s.l.:Princetin Architectural Press, p. 20.

Merleau-Ponty, M., (1964). The Film and the New Psychology. In: Sense and non-sense. s.l.: Northwestern University Press, pp. 48-59.

Noë, A., (2006). Action in Perception. s.l.:A Bradford Book.

Nudds, M., (2014). Is Audio-Visual Perception ‘Amodal’ or ‘Crossmodal’?. In: Perception and Its Modalities. s.l.: Oxford Scholarship.

Pallasmaa, J., (2014). The eyes of the skin. Chichester: Wiley.

Persson, V. (2011). Cross modal correspondences between visual, olfactory and auditory information. [Online] Available at: http://www.diva-portal.org/smash/get/diva2:422823/FULLTEXT01.pdf [Accessed: 1, September, 2019].

Phi, C., 2016. Closer to Human – Interview with Charles Melcher. [Online] 

Available at: https://phi-centre.com/en/post/en-interview-charles-melcher-sensory-stories-2016/ [Accessed 27, August, 2019].

Pursey, T., (2016). Does a Francis Bacon smell like bacon? Lessons from the Tate Sensorium. [Online] Available at: https://www.theguardian.com/culture-professionals-network/2016/jan/15/francis-bacon-lessons-tate-sensorium-museums [Accessed 25 08 2019].

Pursey, T. a. L. D., (2018). Tate Sensorium: an experiment in multisensory immersive design. The Senses and Society, pp. 354-366.

Schellenberg, S. (2014). The Epistemic Force of Perceptual Experience. Available at: https://www.researchgate.net/publication/257637227_The_Epistemic_Force_of_Perceptual_Experience (Accessed: 8 September, 2019).

Slocombe et al., (2015). Cross-modal tactile-taste interactions in food evaluations. [Online]  Available at: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5072487/#targetText=Introduction,Chaudhari%20%26%20Roper%2C%202010). [Accessed: 1, September, 2019].

Smith, B. (2013). The Nature of Sensory Experience: the Case of Taste and Tasting. [online] Available at: https://www.researchgate.net/publication/281900045_The_Nature_of_Sensory_Experience_the_Case_of_Taste_and_Tasting [Accessed 8 Sep. 2019].

Spence at al., (2009). Crossmodal Processing. [Online] Available at: http://multisensorymind.com/images/bibtex/Spence%20et%20al_2009_Experimental%20Brain%20Research_Crossmodal%20processing.pdf [Accessed: 1, September, 2019].

Spence, C. and Piqueras-Fiszman, B., 2012. Dining in the dark. [Online]
Available at: https://thepsychologist.bps.org.uk/volume-25/edition-12/dining-dark
[Accessed 21 08 2019].

Stokes, D., Matthen, M. and Biggs, S., 2017. Perception and its modalities.. s.l.:New York: Oxford University Press..

Tate, 2015. Tate Sensorium. [Online] Available at: https://www.tate.org.uk/whats-on/tate-britain/display/ik-prize-2015-tate-sensorium [Accessed 25 08 2018].

Tiippana, K. (2014). What is the McGurk effect. [Online] Available from https://core.ac.uk/download/pdf/82847523.pdf [Accessed: 2, September, 2019].

Tokuyama, D., 2017. Supersensors: How the loss of one sense impacts the others. [Online] 

Available at: http://www.neuwritewest.org/blog/supersensors-how-the-loss-of-one-sense-impacts-the-others [Accessed 21 08 2019].

Wheatley, J.,  (1973) Putting colour into marketing. Marketing, October, 24-29, 67 

Whiting, D. (2012), ARE EMOTIONS PERCEPTUAL EXPERIENCES OF VALUE? P. 93—107. Available at: https://www.researchgate.net/publication/263612737_Are_emotions_perceptual_experiences_of_value (Accessed: 8 September, 2019).

Wikipedia., 2019. Sense. [Online] Available at: https://en.wikipedia.org/wiki/Sense [Accessed 20 08 2019].

Zampini et al., (2008). Multisensory flavor perception: Assessing the influence of fruit acids and color cues on the perception of fruit-flavored beverages Food Qual. Prefer., 19 (2008), pp. 335-343.

Zampini, Spence, C. (2012). The Neural Bases of Multisensory Processes, Chapter 37: Assessing the Role of Visual and Auditory Cues in Multisensory Perception of Flavor. [Online] Available at: https://www.ncbi.nlm.nih.gov/books/NBK92852/ [Accessed: 2, September, 2019].