Flavour and taste are distinct, yet related entities; taste refers to a five-sense modality originating from taste buds inside the mouth, and it includes sweet, sour, salty, bitter and umami. On the other hand, flavour is fundamentally hedonistic. Flavour is associated with emotions such as satisfaction and pleasantness and is wholly intertwined with an individual’s expectation. (Spence C., 2015). Subsequently, as different people have different quantities of taste buds depending on their genetic makeup they simultaneously have their own personal preferences, and hence different people tend to prefer different tastes (Stevenson, Presscott & Boakes, 1999).

  Charles Spence, an experimental psychologist who established a crossmodal lab in London to study cross-modal correspondences and multi-sensory modalities has since indicated that there are several factors influencing how human perceive food taste. For example, the colour, smell, texture, sound and even surrounding environment are influential (Spence C., 2015). According to his ‘Flavour Conductor’ (see figure 5), research in collaboration with Bompas and Parr, knowledge and techniques of cross-modal correspondences are used in multi-sensory installation to create different flavours upon being influenced by visual and auditory cues.

  In consideration of this, it seems apparent that the notion of a multi-sensory environment to create a unique experience in perceiving a flavour of food has become increasingly popular in modern society, especially among modern day creative ‘foodies’.  A plethora of new, unique restaurants have been popping up all over the world in several countries; Out of The Blue in Berkeley (see figure 1), Tsukihana in Tokyo and Alinea in Chicago. This specific genre of a restaurant is created with an initial set-up compromising a pre-considered itinerary of particular flavours for the participants to experience in a specific order, aiming to provide a desirable experience. Hence, a dining table has undoubtedly become more and more like a stage, a place where people can be entertained and Chefs, together with waiters, can perform a show (Spence C., 2017)

  The implementation of personalisation is on the rise in most industries, not least because businesses have learnt how to apply this concept wisely in their own business. Some businesses utilise data analysis in order to create a personal, relevant and unique experience for their customers. The core idea of Spotify, Netflix and Youtube exemplifies this idea perfectly since their main function is to suggest any upcoming music, tv shows or videos that the users may or may not like according on their personalisation. Data is recorded every time the user types or searches for something repeatedly. Hence, despite this idea becoming ever-present in a variety of sectors, food personalisation is still relatively unknown in the current food industry.

  As aforementioned, what makes flavour and taste different from one another is that flavour refers to something distinct and individual, whereas taste is merely an objective combination of 5 characteristics (Spence C., 2017). Based on there already being an implicit personalisation within food, it is important that the notion of personalisation in food be studied in greater detail. If flavours can be created, theoretically, based on personal information, this means that in the future people could potentially be conceptualised by their desirable flavour experience. Hence, this thesis is primarily focused on the integration of personalisation within a multi-sensory performative flavour experience through synesthetic interaction. Data was collected from multiple domains to consider this proposed framework, including scientific, systematic, and interactive, and could be used as a guide for the designing of a future project. With high expectations, this research aims to be one of the main sources contributing to our academic understanding of flavour personalisation, as well as the actual implementation of this theory in the food industry where dining and performing co-exist.

Flavour is not just a taste

 Undoubtedly, it is difficult to differentiate between the terms ‘flavour’ and ‘taste’, since both of them refer to the sensation people experience after eating something. However, what makes these two words unique from one another is that ‘flavour’ refers to a combination of several elements; for example, a scent that you smell, a sound that you hear, a visual cue that you see, an environment that you are surrounded by, or a combination of each of these (Spence C., 2017). Therefore, flavour has a variety of properties it could be e.g. Fruity, citrusy, smoky, and even burnt. On the other hand, the word ‘taste’ is specifically associated with the use of our taste buds; this includes the four original tastes: sweetness, saltiness, sourness and bitterness. However, on the back of Kikunae Ikeda’s seminal research in 1908, there are now five main standard tastes. Ikeda was awarded with being the founder of Umami, a word which simply means delicious in Japanese. Umami is when there is a formation of glutamic acid, an amino acid, sometimes occurring together with monosodium glutamate. This combination has the potential to make a person feel like there is an ocean of saliva surging inside their month. So, it is imperative for me to understand all of these sensory modalities in order to form a coherent conclusion (Spence C.,2015).

  Without a doubt, when considering the most important sensory modalities for humanity, the smell of food is crucial. For both food and drinks, smell plays a dominant role in determining how an individual experiences taste after eating (Spence & Youssef, 2015). By considering this, according to one argument from the scientific literature, data shows that roughly 75 to 95% of what people normally think of as ‘taste’ is actually coming from olfactory receptors in the nose, not from the taste buds inside the mouth. Nevertheless, Professor Charles Spence claimed that such a statement has no scientific support. Instead, researchers widely support that the smell of the food plays a dominant role in how people perceive taste, but cannot define a specific percentage (Spence,2015).

  Studying how people smell food is of the utmost importance when considering its role in multi sensory experiences of food. There are two main categories of smell, Orthonasal and Retronasal. The former is a smell that people have from sniffing external aromas, whereas the latter is a scent that runs from the back of the mouth into the end of the nose after swallowing drinks or foods. These two factors are particularly similar as both of them play a vital role in how humans experience the taste of their food (Diaz, 2004). However, there are fundamental differences that can be found between these two categories, and they will be discussed in more detail in the next chapter.

  
In order to study these two categories of smell in greater detail, a JellyBear experiment was subsequently conducted to help develop empirical understanding. The JellyBear experiment is an experiment that is conducted in order to prove that both Orthonasal and Retronasal play a dominant role in flavour perception. In the experiment, the participants were asked to close their eyes in order to eliminate visual cues from their taste perception. After that, different Jelly bears with different flavours were randomly given to the participants, such that the participants did not know which flavour jelly bear they were eating. In the experiment, over 80 percent of participants could correctly identify the flavour. However, after these people were asked to pinch their nose tightly closed, in order to prevent their olfactory receptors from receiving the smell molecules, most participants found it harder to tell the flavour of the food. There was a lack of stimulation and information, so much so that all they could identify was the sweetness of the sugar. According to this experiment, the findings show that scent is the most important factor influencing how people perceive the taste of their food. So, it is of the utmost importance to delve deeper into the research on how this function actually works and how it leads to a change in particular flavours.

Synesthetic connections between odour and taste is a common effect. The majority of people appear to experience odour-taste synesthesia. It is said that when smelling an odour, most people can more easily recognize a taste-like quality, such as sweetness (Howes, 2006).

  By considering this, there is a wealth of research that has been conducted in order to explore how different odours influence a human’s perception of sweetness and sourness. The findings show a sweet-smelling scent, such as caramel and strawberry, can help degrade the sourness of citrus and enhance the sweetness of sucrose. Comparatively, a lower sweetness scent will enhance the sourness of citrus and degrade the sweetness of sucrose (Stevenson, Prescott & Boakes, 1999).

 

 

 In order to further understand this theory, it is important to consider the ‘Marshmallow experiment’. This experiment was conducted in order to empirically explore how different smells influence particular flavours. There were 6 different scents used in this experiment, including sweet-smelling odours such as Chocolate, Vanilla and pandanus, and sour-smelling odours such as Lime, Orange and Peach.  

All of these previously stated scents are applied by the use of essential oil filled inside a tiny glass bottle, connected with a funnel glass on one side and a pump of air on the other. By doing this, the air pump releases air into the glass bottle so that the scents will be sent to the funnel glass where participants are located. In the experiment, the participants were first asked to eat a single marshmallow without adding any additional flavour and were then asked to indicate how they perceived the taste. After that, the participants were then asked to eat a marshmallow simultaneously with smelling different scents mentioned earlier.

     The findings showed that people are likely to perceive different tastes according to different scents received. However, it is important to consider that it is quite difficult to obtain precise data in experiments like this since the flavour is hedonic and subjective. Thus, results may depend on several different factors, from an individual sensation to a public expectation at that time. To illustrate this idea, the flavour Pandan is used; a flavour which has long been used in several foods with the Thai society. Hence, every time there is a scent of Pandanus, Thai people immediately think of a Thai dessert. By having this feeling, people are likely to perceive the taste as sweeter than it usually tastes. At the same time, people who have no relation to this feeling will not feel that these desserts are sweeter. Therefore, even though there is no exact information declaring that this principle is accurate, it still provides some evidence that scent influences how people perceive the taste of food. The findings may not reveal how these different scents occur, but it shows that different people have a different level of tasting complexity; thus, this allows people to experience several new flavours depending on the integration of multiple sensory perceptions.

As an experimental psychologist, Professor Charles Spence, a professor at Oxford University, has conducted most of his research on multi-sensory modalities and cross-modal correspondences. His work has provided evidence suggesting that it is not only smell that greatly influences how people experience taste, but that other sensory cues such as colour, texture and sound are also included in making a change.

  By considering this, the word ‘multi-sensory’ refers to an experience when there is more than one sense experienced simultaneously, whereas the term ‘cross-modal’ is when one sense influences the way one experiences other senses. To illustrate this idea, the use of red lighting is supposed to make the taste of wine in a dark glass sweeter and even fruitier. The cross-modal correspondences can also be considered multi-sensory. However, not all multi-sensory correspondences can be a cross-modal correspondence (Spence C., 2017).

 Research has also demonstrated that individuals tend to relate tastes, smells, and flavours with other unrelated sensory cues in a very consistent way. To exemplify this idea, a high-pitched sound of the piano is often related to the taste of sweetness, whereas a sound with a lower pitch generates a perceived bitter taste (Spence C., 2017).

  In order to test this theory, a cross-modal correspondence was also conducted. According to the study, more than two thousand participants were given a ‘dark testing’ glass of red wine and asked what the wine tasted like. The same wine was used, but people who drank with red ambient lighting and sweet background music rated the wine to be fruitier than those who drank with green lighting and ‘sour’ music. This experiment seems to show that one’s surrounding environment also plays an important role in manipulating how people perceive taste. The reasoning behind this is that different places create different senses for the consumer; for example, having sushi at a five-star restaurant would contribute to a totally different feeling to a consumer than having it at home (Spence C., 2017).

  Moreover, another great example used in order to compare these two correspondences is an installation conducted by Bompas and Parr, in collaboration with Professor Charles Spence, titled Flavour Conductor. This project features an immersive multi-sensory live performance with the intention of stimulating six different tastes for the audience while they sip their drink. As a form of cross-modal correspondence sounds were created from an organ that had different pitches but matched with what they saw on the sculpture-like instruments. The audience supposedly noticed several different tastes of whiskey at several different points in the performance, and so the project indicates that the use of cross-modal correspondences encourages particular flavours to be stronger at particular periods of time.

Flavour Conductor is a project that was designed in order to help a whisky drinker, both new and experienced, to better appreciate the flavour complexity of Johnnie Walker’s Blue Label. Also, it aimed to amplify the taste of the whisky in the glass through multi-sensory theatrical performance; this is when music is used in order to create visual, aromatic and haptic cues to different nuances of whisky.  

  In 2003, Professor John Edwards, a Food Quality journal writer, produced a similar project to Professor Spence, differing only in terms of its focus. Besides the combination of all senses, the study also emphasised the experiences people have based on their surrounding environment. On the back of his research, it has been suggested that environmental attributions, such as background music, the colour of lighting, ambient scents, temperature and the material of dinnerware can all influence how one perceives the taste of their food. Specifically, the study showed that the same dish can taste different depending on location; participants were asked to eat Chicken A La King at several different places, including at a boarding school, at a care home and at a four-star restaurant. The results of the study found that the dish that got the highest score was the one served in a four-star restaurant, while the lowest rating was when the dish was served in a boarding school.

Another example of cross-modal correspondence is MetaCookie. MetaCookie was also used as a research sample in order to explore how sensory modalities could improve the taste of food. According to the University of Tokyo, the experiment incorporated how virtual reality and olfactory information can change the taste of a normal cookie. In the experiment, the participants were asked to eat a plain cookie together with receiving a scent of chocolate chips from the tubes. As a result, the findings showed that most of the participants actually thought that they were eating cookies with chocolate chips inside. Without changing the texture, an ordinary plain cookie had turned into a cookie with chocolate chips (Narumi, Nishizaka, Kajinami, Tanikawa & Hirose, 2011).

MetaCookie, a project that aims to convince people to gain particular pseudo flavour experiences by using a VR headset and its scents.

   Another of the multiple intriguing examples of hose flavours of products can be varied based on olfactory cues and their design is the Right Cup experiment from 2016. The Right Cup experiment utilised different colours of mug in representing different tastes of drinks. Thus, in the study, a yellow mug was used to represent the taste of a lemon, whereas the green cup represented an apple taste. While conducting the study, the participants were asked to drink water inside the mug simultaneously with smelling a fruity aroma. The study’s results suggest that the colour of the mug and its aromatic scent are quite significant to one’s taste perception, since most participants indicated that it gave them a tasting experience that was real and authentic. Similarly, Aroma Fork, or a metal fork that has a liquid capsule underneath its handle, also provides a different sense to the tasters. Hence, these results again support the notion that individual taste experience greatly depends on how one sees the product and what the product smells like.

   From the aforementioned examples, it seems clear that there are several different goals expected by the user from the use of multi-sensory modalities. For example, Flavour conductor and Meta cookies, these two models are examples of an integration of multi-sensory cues, since they are created in order to convince people to have a certain perception towards a particular desirable flavour designed by a food designer. Hence, this is when the designers try to make their participant gain an experience that has already been set up. However, this idea appears to be the total opposite when considering the concept of the Right cup and the Aroma fork; these two experiments allow people to combine different scents and foods together on their own. By customising their own food, these experiences are helping enhance an individual’s flavour experience, whilst also helping them stimulate the sense of flavour exploration.

Synesthesia as a design approach

 To begin, Synaesthesia is a neurological condition in which one sensory cue has an effect on the other different senses with the use of cross-modal ideology. This happens automatically, involuntarily and irrepressibly. Simply put, Synaesthesia is the visualisation of sound, sensation of words, taste of colours, and the touch of tastes (Haverkamp, 2012). Therefore, the concept of Synesthesia has great potential to offer a multi-sensory design approach allowing intersubjectivity within the design process. Furthermore, the association of sensory modalities of imagination, flexibility and independence will also help in enhancing a more creative interaction and help in creating a unique multi-sensory experience for the participants.

 The concept of Synaesthesia emphasizes the approach of cross-sensory relations. Contrary to the concept of multisensory, Synaesthesia does not have to explicitly focus on possibilities of systematic cross-sensory connections. Furthermore, cross-modal correspondences are also sometimes synesthetic when the sense is shared with the majority (Spence C., 2017). According to the research considered in chapter 1, conducted by Charles Spence, humans tend to relate tastes, smells, and flavours with other unrelated sensory cues in a very consistent way; for instance, higher and lower pitched sounds of the piano also create different feelings for the listeners (Spence C., 2015).

 Spence (2017)’s recently conducted research on multisensory integration suggests that not only is a single perceptual mechanism applied in the integrating process, but also a variety of strategies can be involved (Spence, 2017). Despite the notion of synesthesia, as a methodology within a multisensory design, being hailed as the primary concept based upon those processes, combining all possible strategies of perceptual connections between each modality appears to generate a better result. (Howes, 2006).

 To illustrate the idea of Synaesthesia, Bompas and Parr’s Multi-sensory Fireworks (see figure 10) offers the perfect example. Within this experiment, different colours of fireworks were matched with the smell and taste of particular edible balloons; for instance, Red was matched with a scent and taste of Strawberries. Although systems of allocating colours to sounds, smells, or flavours were mainly dependent on the designer’s preference, there were several sets of colours; this was particularly true for the case of the sweet and fruity taste components, as they were selected based on confectionery items. Hence, this experience observably involves iconic colour associations according to the provided figure below, which creates an individual and unique reference of the colours to fruits.

     Furthermore, there was also an experiment that was conducted in order to explore the actual match between colour and smell, colour and taste, colour and emotions and colour and meaning. In order to explain this idea, figure 12 and13 should illustrate the concept in a clear and concise way.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

  According to Figure 12,  the association of visual cues, smell and taste of the multi-sensory fireworks contributes to the expansion of the participants’ multi-sensory and synesthesia-like experiences. Specifically, the participants were appreciating the visual cues of the fireworks and simultaneously were linking it with its taste and smell.

    In addition to the previous example, there is another interesting model coined the Squeeze music box in figure 14. This box is a jukebox maker that allows people to experience the actual ‘taste’ of the music by selecting a song. According to the process, the drinks are mixed automatically based on the mood of the song, after the song is selected. Thus, this machine allows people to actually drink their preferred music so that they will know how it actually tastes. Alternatively, another fascinating project based on the concept of visual Synesthesia is Boutique. This project allows people to experience various scents released from a cone-shaped instrument due to the colour of the vibrant posters; for example, red is paired with the smell of strawberry. Subsequently, this influences participants’ stereotypical idea of smell and colour.

 

Squeeze Music is a jukebox maker that allows people to experience the actual taste of the music by selecting a song.

Boutique by Niklas Roy

   In summary, all three experiments, fireworks, squeeze music and boutique, show that human senses are interconnected with one single input. In Fireworks, a taste is linked with its colour; the taste of the drink is linked to a mood of the song in the Squeeze Music Box paradigm; and, the smell is engaged with colours within the Boutique project.

  The synesthetic approach not only contributes to an experience expansion for participants, but it also influences how people think about particular senses; this somehow gives a new meaning to that particular object. To illustrate this, multi-sensory firework is an experiment that mainly focuses on how the colours of the fireworks change, and how all the colours are combined at once. Hence, this later makes the participants mainly focus on the visual and sound cues rather than anything else. However, this project also introduced other senses, such as olfactory and gustatory cues, for the fireworks. Hence, this experiment not only expands the perception of how people see the fireworks, but it also creates a new meaning of how one firework may taste sweeter than another.

  Moreover, when we talk about the use of symbolic cues, for instance, smell, colour and sound, most people use these in order to represent themselves, or to create meaning as a comparison; for instance, the smell of a rose is usually used in reference to the romantic sense of a wedding day. Hence, it seems fair to conclude that all of these scents do actually have a connection between both givers and receivers.

 So, in this thesis, the use of a synesthesia approach can be seen as an individual expression, but also can be seen as synesthetic interweaves of sensory cues in which all of them create a synergy, delivering particular flavours to the participants.

Theatrical approach

   On the back of these findings, the concept of multi-sensory modalities has been used recently in dining experiences. It can be clearly stated that ‘experience’ has now become a sort of performance at a dining table. Without any expectation, such a performance has turned out to be one of the main ingredients in enhancing the taste of food and the overall flavours. Hence, it seems viable to say that a dining table has become more and more like a stage, a place where people can be entertained and Chefs together with Waiters are allowed to conduct a show (Spence C., 2017).

   To illustrate this, Alinea restaurant in figure 18 is a fine example. This restaurant makes and serves their desserts directly onto the table right in front of the diners. This creates such a unique experience for the consumers since foods are normally served on plates. Hence, together with other

various sensory modalities such as scent, vision, and sound, this restaurant successfully enhances the multi-sensory immersive flavour experience for their diners.

   Moreover, the integration of multi-sensory cues, such as visual, smell and sound, are also used in order to present their dishes in a performative way. To exemplify this, Jelly of Quail, a tray of fish cream (see figure 19) made by Heston Blumenthal, was a normal chicken liver parfait, oak moss and truffle toast. However, with an additional use of dried ice in order to create a scented fog, the combination of a breathtaking scene together with the mesmerising scent of the food makes the diner’s experiences unique and remarkable. Pillow of Nutmeg Air (see figure 20) by Grant Achatz utilises the scent of Lavender to make people think of the forest, and the smell of lavender combined with the scent of the food itself creates enhances the eating experience for participants.

    These methods are mainly used in order to create a certain environment that helps enhance an immersive flavour experience. Hence, this food performance acts like a theatre where the chef acts like a performer delivering entertainment to their audience. These food performances have grown in popularity, with some receiving near-fame. Not only does a food-delivering-performance enhance the experience, the use of an integrated multi-sensory experience, along with innovative technology such as a 3D mapping, can produce a truly unique eating experience. These features also help flavour experiences to be even more immersive. Also, by using a scent as one of the elements of telling a story makes the consumer feel more connected to their food, through personal experiences and memories. What is left is the question of whether all these aforementioned elements are actually used in order to enhance the flavour itself, or just to entertain the consumers.

    In comparison to the performance-based dining experiences, the Dine in the Dark project provides a completely opposite approach to the idea of food. To illustrate this, this project works on the notion that eliminating all other cues, including visual and audio, tends to make the diners gain a deeper food experience without being distracted by any other cues. The eater mainly focuses on eating and experiencing the actual food. However, in my opinion, this practice does not take away all the cues like it claims, what is left is a clearer manifestation of the sound that we hear when we chew food, and a more focused touch when we take the food into our mouth. Hence, I believe that the Dine in the Dark project is also a project that uses multi-sensory flavour experiences in order to enhance the taste of the food, with the only difference being that they do not predominantly focus on using visual and audio cues as their main technique.

     Hence, for this thesis, as the use of multi-sensory performance is inherent within food performance, our main position for this research is that the multi-sensory is the main influencer influencing the taste of the food itself, rather than the entertaining side.

     As previously mentioned, presenting a food by integrating multi-sensory cues in these different techniques appears to make the diners feel more connected to the food; it creates an emotional engagement in which only the one who experiences that memorable moment can truly understand it. On the back of the experience is unique, the question has later come up of whether the diners should be more engaged in the experience, taking control of their food. Or, comparatively, should only the chef who entertains them be in control? In other words, what if the experience offers the feeling of equal parts between diners, performers and audience members at the same time? What if these experiences were specifically and specially created for each individual, where diners are able to share their initiative and creativity. What would the experience be like?

 

 

 

 

 

 

 

 

 

 

 

 

 

Unique individual taste

 Because different people have different degrees of receiving a taste, it is fair to say that a taste one has received could certainly be different from what another has experienced. This is especially true because taste mainly depends on how many taste buds on the tongue an individual has. It is said the number of taste buds affects the signals that are sent from the month to the brain; some people called super-tasters have 16 times more taste buds on their tongues than other individuals (Spence C., 2015). With these differences, each individual tends to enjoy their food in a different way. Thus, different people then have different flavours they prefer.

Definition of Personalisation

   Personalisation refers to a broad term that includes three types of behaviour: adaptability (also sometimes called customisation), context-awareness, and adaptivity. Adaptability refers to a number of options the user has in order to set up an application or system according to what the user likes. Furthermore, context-awareness refers to a system-based ability to sense the current state of the surrounding environment and to respond accordingly. Last, but not least, adaptivity refers to a system that maintains a dynamic model of ongoing interaction. So, this means the human behaviour is likely to change dynamically due to changes in the situation. (Not & Petrelli, 2018).

    With the surge in several advanced technologies, it is viable to suggest that it is quite easy for people to now analyse how other people think, like, and feel. Machine learning, for instance, uses a data algorithm analysis to allow the user to analyse personal information, and to formulate the preference of individuals through different means of interaction they provide the system. Spotify and Youtube would be the best examples of machine learning, since both of them always keep the set of songs a user has played, whilst then suggesting any songs that the user might also enjoy.

   Furthermore, in the context of flavour experience, Bombay Sapphire Distillery is running a project where participants have their own personal drink created by a system. Within this project, participants were asked to select any gin-making ingredient, this can be done via smelling, touching or seeing, in order to create their own personal drinks. Afterwards, the system would analyse that particular ingredient in order to create a personalised gin cocktail for each individual. This process is basically based on what kind of taste a person prefers so that it will later create a menu that has similar drinks for each individual. By the same token, a campaign conducted by Johnnie Walker (see figure 21) is a good example of personalisation. People were asked to choose a mood, environment, flavour and aroma they like, and then the system personalised a suitable taste of whiskey for them.

From the given example of Bombay and Johnnie Walker, both examples are the use of a personalised system, which analyses information to produce the one and only suitable choice for an individual. Hence, personalisation is certainly comparable to the use of multi-layers since all the personal information is flexible and can be changed over time. So, it is quite hard for certain information to always be accurate; this opens up multiple possibilities, rather than being limited to only one system of an algorithm.

    Not to mention this, research has also revealed that visitors who shape their own experience are likely to have a more personal engagement in the platform. To illustrate this, the IKEA effect is a great example of the concept of self-prioritisation. This concept indicates that those who cook by themselves are likely to rate the taste of their own food higher in comparison to another person’s cooking. Also, frying and preparing the meals makes participants feel more involved and engaged with the food. Thus, most of them ended up rating their dish better than before. Hence, it can be simply stated that the more a person is involved in the process of cooking, the more likely they are to feel more positive toward the food they made; this will subsequently lead to greater satisfaction towards the food they eat as they cooked it themselves. It is said that the more people feel empowered, or are able to take control in a particular task, the more they enjoy the outcomes and experiences (Spence C., 2017; Norton M., 2011).

   When we talk about dining, people are usually allowed to adjust their own food in order to meet their own preference. With these practices, the process is called “customisation”. An example of this is when people decide to add more seasoning ingredients in order to make the food match their own preferred taste. By adding spice, sugar and salt, an individual is customising their food. At this stage of the research, the food items Salads and Crepes offer great savoury and sweet examples of this concept. Whenever we eat a salad or crepe, it is our own decision to select which toppings we want to eat, a choice we make to satisfy our need. In this way, people either enjoy the choice they made as it is based on prior experience, or they experiment and find a combination they had never expected. To illustrate the idea, Pepsi allows their consumers to create and mix their own Pepsi drinks by selecting up to three flavour shots, which subsequently allows for the production of 1,000 different combinations.

   With the aim of not controlling or limiting possibilities, the Cocktail Machine project was established. This project was conducted in order to test the idea of customising flavours, as well as in order to encourage people to interact with what they drink. While conducting the research, atomisers were used to generate six different scents of beverage, two examples being lime and orange juice. According to the study, the findings showed that most participants decided to select a cocktail ingredient based on the smell they had received.

  Even though the visual process of food creation allows people to partly know and understand the process, it is quite difficult for the individual to understand the taste of the final cocktail as the information is not explicitly delivered. The lack of communication leaves participants unable to formulate an expectation of what they experiencing. In this way, the experience of customisation has failed. Instead, something like the Sense the Scents project is much more advanced. This project aims to explore more about how synesthetic interaction can be used as another layer of communication, where people can share their preference and eventually take control of the system, possibly creating their own expected mixed scents as the outcome.

A cocktail machine, a project where people smell the beverages from atomisers in order to choose their preferred drinks.

Sense the Scents

   Sense the Scents, an interactive, olfactory installation where collective scents are being created, is the result of an investigation into the idea of customisation, collective preference and shared multi-sensory experience for participants. Given these, the project features three different juice smells: blueberry, strawberry and orange. These scents are released from each structure of the installation, with each of the smells matching with a colour of lighting; for example, strawberry with pink and blueberry with purple. Then, participants are able to choose their preference by activating the tangible interface which is connected to an operating light.

  The operation of lighting, the act of squeezing as well as the method of pressuring determines how much fog is generated. Meaning, the harder the press, the more fog that will come out. When turning on the light, people feel empowered as they actually have power over the system; at the same time, it gives participants an awareness of actions related to what is currently happening, and what should be an expected outcome. In this case, the use of mixed scents together with a performance of lighting and fog generates different senses for the participants.

   From the previously stated examples, different studies focus on different things; some of them use a system that will firstly analyse personal data in order to create a single option for that individual, a customisation that suggests the choose-and-control process. In my opinion, combining these two hypotheses, personal data and customisation, will provide a synergy that delivers a complex and extended emotional flavour experience. Thus, this will later open more possibilities for the participants so that they are likely to explore more unexpected flavours at a higher level of satisfaction.

Sense the Scents, installation with a hanging structure where different colours and scents are exhibited.

The proposed framework of personalisation in flavour experiences

  The study above indicated our approach to the concept of personalisation in an interactive flavour experience. There are two main features of personalisation that I will mainly focus on, adaptivity and customisation. Each of them has different techniques and systems that create different interactive multi-sensory performances. They are displayed as an overlay, a synchronisation of the same performance which greatly affects the flavour experience of the food. Given this, a multilayer framework was implemented in order to give an idea of how the entire interactive system operates, and what the desirable experience is expected to be. According to the process, the system moves from adaptivity to customisation. This study investigates how different personalisation techniques can be integrated to support several patterns of experience. Hence, this thesis is being conducted with the aim to accommodate different motivations, emotional attitudes and expectations of the participants.

4.1 Adaptivity: personal engagement and awareness of a presence

 Talking about an interpersonal skill, adaptivity, or the capacity of being adaptive to the change of certain situations has long been known as one of the main elements of personalisation. Hence, when considering food, adaptivity is used in order to raise an individual’s awareness so that they will feel more empowered and engaged with the platform (Not & Petrelli, 2018).

  Adaptivity, the capacity of being adaptive to the change of certain situations, has been known as one of the main elements of personalisation, Hence, for the proposed framework, adaptivity is used in order to raise an individual’s awareness through the system, adapting according to an individual’s current information. In this way, individuals will feel more empowered and engaged with the platform.

The interactive system’s data collection was predominantly derived from the user’s interaction; for instance, when a user interacts with the system interface a sensor detects the information manipulated either directly or indirectly by the user. Hence, a user’s standing positions, moving directions, or even their state of emotion are all involved in the adaptive environment.

      The system monitors how the user adapts and responds to the new condition. This condition may include various things, such as the actual and observable physical characteristics; for example, the movement of a user. So, it mainly presents specific information to a user based fundamentally on that individual’s current condition. In other words, this is a user-adaptive interaction which helps create a data visualisation. To exemplify this idea, The Scoop: A wonderful Ice Cream World is an inaugural show at the new British Museum of Food in London, created by the food architects Bompas and Parr. Brain activity was captured by EEG sensors while people were eating ice cream. In other words, all the visualised neurological responses to the thing they were eating were captured by a coloured laser. So, the more people were engaged with these sensors, the higher the degree of connection between the individual and the platform. Also, people were allowed to reflect themselves with particular types of platforms.

   The output of individual artistic expression can be in multiple forms, such as visual and auditory cues. Also, such performances are not pre-defined or set up in order to create a result, everything happens in a real-time process. Hence, this eventually makes the individual feel more engaged to the system as if they are actually a part of it. Thus, this is the first layer of interaction designed to amplify physical engagement with the artefacts.

   By considering this, the Sense The Scents project also explores this topic by using a proximity sensor in order to detect an individual’s physical presence. So, if a person gets close enough to the installation, the lights will be shown up as detecting an object. Furthermore, the lights that go on and off are accounted for by an individual’s movement, and this is known as one of the performative arts. Hence, this can be seen as one way to create an environment that facilitates and creates a sense of belonging for an individual.

4.2 Customisation: awareness of actions

      As previously mentioned, an IKEA effect is when a customer feels satisfied by the services since they are involved in the creating process (Norton M., 2011). Hence, on the back of this evidence, customisation appears to be the strongest explanation for this concept.

   The use of tangible interaction helps get more people involved, as well as providing several different ranges of experience to the users. Hence, people are allowed to either change their mind as they select more choices, or select all the given choices at once. Thus, it seems apparent that empowering an individual with a choice of flavour based on their preferences will help improve and stimulate a personalised flavour experience.

    By considering this, both the Cocktail machine and Sense the Scents projects used a squeezed ball wrapped with a sensor as a tangible interface. So, the harder the ball was pressed, the more fog was generated. As this action turns on the light, people will feel empowered as they actually have a degree of power over the system. At the same time, it gives the individual an awareness of actions in what is currently happening, and what would be an appreciate outcome. In this case, the use of mixed scents together with a performance of lighting and fog generate different sense to the participants. Also, multiple squeeze ball also makes the food more interesting as it simultaneously activates at the same time

  According to both projects, Cocktail machine and Sense the Scents, an awareness of actions is the most important thing; if people do not pay attention to their actions, then anything can happen. So, creating an awareness of an individual action is the most important part of this Sense the Scents project. Hence, this is why lighting is used in indicating how soft or hard the user is squeezing the ball.

  A good example of such a topic is MixPlore, a project that explores how cocktail mixology, taste, sound and technology can be mixed in order to create a new performance. The main elements used in creating this live synesthetic digital performance is the act of several gestures, for instance shaking and stirring. By doing so, every single step is shown in great detail, with every gesture emphasised and focused; starting from the change in positioning, moving towards rotating the shaking tube and pouring it into the glass. Hence, everything is deliberately emphasised; this experience has later been titled as a medium of performative art (Lee, Chang & Lim, 2011).

By using an adaptive system, the performance will constantly change according to the current state of the individual response. However, a customisation system will react only when a person takes control or has a direct interaction with the system. So, this is quite deliberate; for instance, Mixplore involves several interactions which may lead to a confusion of the system. So, different levels of an interaction must also be fixed with different aims. This requires a deeper understanding of the actual person, in order to develop the system into its best use.

    From the Sense the Scents project, specifically, a squeezing action also influenced how the lighting would be operated. Thus, the harder the squeeze, the brighter the light. Also, there was a synesthetically association with different scents in an intuitive way; for example, during the presence of the purple colour one was more likely to smell a grape. Hence, by combining all of these techniques, adapting multi-sensory cues can manifest in several multi-sensory experiences for the individual in the food itself. So, the combination of an individual’s initiative, together with autonomous decisions made by the system, will create complex scenarios where expanded personalised experiences can be introduced. The interweaving of two types of personalisation features will be layered and give complex scenarios.

    As stated previously, there are two main frameworks in personalisation: adaptivity and customisation. By considering this, the synergy of the two of them helps one another influence and improve a flavour experience in a multitude of ways. Adaptivity mainly focuses on communicating through a personal interaction by using a system to detect personal information from each individual. Thus, this is when people are reminded to create their own self-awareness, and this later makes them feel more engaged with the platform. On the other hand, customisation mainly emphasises the importance of sharing experiences and preferences in order to create the most satisfying result. According to the findings, both techniques actually help in the enhancement of how people experience the taste of food, and this is the reason why this research is being conducted. From the diagram, it is apparent that there are two completely different routes to this experience of taste: adaptivity and customisation. By considering this, each of them uses information in different ways to trigger a different interactive sensory performance. However, both of them create a similar overlay that is integrated into a multi-sensory interactive food performance, embedded in physical artefacts; in this case, it is the food itself. Hence, these synesthetic multi-sensory cues have later influenced flavour experiences and contributed to a personalised flavour unique to each individual; this makes sense as different people have different preferences in terms of taste. Thus, all of these are expected to deliver emotion hyper-relevant to each individual.

Conclusion

    With the emergence of technology, together with personalisation which now plays a major role in several industries, this thesis aims to be the foundation for others to explore personalisation within a flavour experience. By doing so, we have proposed a framework to study the topic in all its aspects, starting from a scientific approach and moving into a psychological and interactive domain.

     Also, the system of personalisation, as mentioned in the previous framework, is also involved in the study. So, we hope that this framework will act as a starting point that initiates the optimal practical design of personalised food experience. The desired experience is contemplative and emotional, not something energetic and interactive. It is expected that people will appreciate the change of the flavour as it is specially personalised and created for that individual, and it will most likely give a hyper-relevant emotional flavour experience to each individual.

   The system of personalisation that we have designed is expected to be open to all possibilities so that people are able to share multiple preferences, as well as to combine or overlay different information from different people. This will later create a completely complex scenario for the participants, as well as to encourage a sense of exploration. Not to mention this, people will likely feel more engaged and empowered as they feel like they are being a part of the system. With high expectations, this research aims to be one of the main pillars of knowledge contributing to the academic framework of flavour personalisation, in the hope of actual implementation in the food industry where dining and performing co-exist in one place.

Bibliography

Diaz, M.E. (2004). Comparison between orthonasal and retronasal flavour perception at different concentrations. Flavour and Fragrance Journal. Retrieved from: https://onlinelibrary.wiley.com/doi/abs/10.1002/ffj.1475

Goldberg, E.M., Wang, K.m Globerg, J. & Aliani, M. (2018). Factors affecting the ortho- and retronasal perception of flavors: A review. Critical Reviews in Food Science and Nutrition. DOI: 10.1080/10408398.2016.1231167\

Haque U. (2008). Notes on the design of participatory systems for the city or for the planet. Retrieved from: https://www.haque.co.uk/papers/notesonthedesignofparticipatorysystems_eng.pdf

Haque U. (2016). “Technology is the answer, but what was the question?” On art, smart cities and bringing people together. City Metric. Retrieved from: https://www.citymetric.com/horizons/technology-answer-what-was-question-art-smart-cities-and-bringing-people-together-1951

Haverkamp, M. (2012). Synesthetic Design: a Handbook for a Multi-Sensory Approach. Birkhuaser. Retrieved from: https://www.amazon.co.uk/Synesthetic-Design-Michael-Haverkamp/dp/3034607156

Howes, D. (2006). Hearing Scents, Tasting Sights: Toward a Cross-Cultural Multi-Modal Theory of Aesthetics. Concordia University. Retrieved from: www.david-howes.com/WebHowesMulti-modalAesthetics.doc

Hornecker E. (2006).  Getting a Grip on Tangible Interaction : A Framework on Physical Space and Social Interaction. ACM Press. DOI: 10.1145/1124772.1124838

Hornecker, E. (2011). The Role of Physicality in tangible and Embodied Interactions. ACM Interactions Magazine. doi>10.1145/1925820.1925826.

Hornecker, E. (n.d.) Physicality in Tangible Interaction : Bodies and the World. University of Sussex. Retrieved from: http://www.ehornecker.de/Papers/TangibleBodies.pdf

Kakutani, Y., Narumi,T., Kobayakawa, T., Kawai, T., Kusakabe, Y., Kunieda, S. & Wada, Y. (2017). Taste of breath: the temporal order of taste and smell synchronized with breathing as a determinant for taste and olfactory integration. Scientific Reports. DOI: 10.10038/s41598.017.07285.7

Lee, Z., Chang, S. & Lim, C.Y. (2011). Mixplore : a digital performance using tangible user interfaces based on cocktail mixology. International Journal of Arts and Technology. DOI: 10.1504/IJART.2011.039841

Narumi, T., Nishizaka, S., Kajinami, T., Tanikawa, T. & Hirose, M. (2011). Meta Cookie+: An Illusion-Based Gustatory Display. Graduate School of Engineering University of Tokyo. Retrieved from: https://doi.org/10.1007/978-3-642-22021-0_29

Norton, M. (2011). The IKEA effect : When labor leads to love. Retrieved from: https://doi.org/10.1016/j.jcps.2011.08.002

Not, E. & Petrelli, D. (2018). Blending customisation, context-awareness and adaptivity for personalised tangible interaction in cultural heritage. International Journal of Human-Computer Studies. Retrieved from: https://doi.org/10.1016/j.ijhcs.2018.01.001

Small, D.M. (2005). Odor/taste integration and the perception of flavor. John Prescott. DOI: 10.1007/s00221.005.2376.9

Spence, C. (2015). Multisensory Flavour Perception. MEDLINE. DOI: 10.1016/j.cell.2015.03.007

Spence, C. (2015). Just how much of what we taste derives from the sense of smell?. BMC.  Retrieved from: https://flavourjournal.biomedcentral.com/track/pdf/10.1186/s13411-015-0040-2

Spence, C. (2017). Enhancing the experience of food and drink via neuroscience-inspired olfactory design. The Senses and Society. DOI: 10.1080/17458927.2017.1270800

Spence, C. (2017). Gastrophysics: The New Science of Eating. Viking. ISBN13: 9780735223462.

Spence, C. & Piqueras-Fiszman, B. (2014). The perfect meal: The multisensory science of food and dining. Chichester: Wiley Blackwell.

Spence, C. & Youssef, J. (2015). Olfactory dining: designing for the dominant sense. BMC. Retrieved from: https://flavourjournal.biomedcentral.com/track/pdf/10.1186/s13411-015-0042-0

Stevenson, R.J., Prescott, J. & Boakes, R.A. (1999). Confusing Tastes and Smells: How Odours can influence the Perception of Sweet and Sour Tastes. Chemical Senses. Retrieved from https://www.researchgate.net/publication/224983743/download

Visser, M. (1992). The Rituals of Dinner: the origins, evolution, eccentricities and meaning of table manners. A Penguin Book. Retrieved from: https://www.amazon.co.uk/Rituals-Dinner-Origins-Evolution-Eccentricities/dp/0140170790

Spence, C. & Piqueras-Fiszman, B. (2014). The perfect meal: The multisensory science of food and dining. Chichester: Wiley Blackwell.

Spence, C. & Youssef, J. (2015). Olfactory dining: designing for the dominant sense. BMC. Retrieved from: https://flavourjournal.biomedcentral.com/track/pdf/10.1186/s13411-015-0042-0