Title image:Transient from Pablo Gnecco

The main method of non-verbal interaction between individuals and the environment is re-channeling, namely the ability of collecting the data from the subject and turn it into the signal that can be demonstrated by the medium that can act on the environment and react to the individual. Human beings have a natural desire to communicate with the unseen interior psychological state. In addition, Emotion is the inner thoughts of human beings. And also emotion is an invisible response to a stimuli. In conclusion, in order to re-channel the interaction between emotion and environment, we should change the invisible inner emotion into a visible and measurable data. In this article, we choose two ways of expression that identified by Darwin in the book ‘The expression of the emotion in man and animals’—-the body movement and facial expression. For the environment expression, we choose the projection mapping to create a space for the ‘communication’.

The experience of emotions normally includes thought, action impulse and somatic disturbance [Ekman, 2003]. Though emotion cannot be seen, still naturally react to the spatial changes, color flicker and brightness changing in the environment around the individuals. We intend to find the relationship between the environment and emotions. Especially, the interaction between the color and emotion, and also the connection between the body movement and spatial changes.

The article is to find a visible way to measure the emotion of the individuals through the theories developed by other scientists. The measurement of the emotion change the external environment and create a new stimuli. And the stimuli can act to the emotion in the induvial in return. This circulation can be represented as the emotion talking with the environment through certain installations and sensors.

METHODS OF EMOTION EXPRESSING

Emotion is thought to occur in response to a social event [Ekman, 1993]. What’s more, Emotion is broadly regarded as the universal interface mediates between the constantly changing environment and the individual’s behavioral response. [Klaus R. Scherer, 1984] According to this, emotion is not just about individuals, it should have an interest in the social level. Emotion should be measured and then transmit to others, in order to interactive with other individuals or environment around them. What’s more, a common set of core emotions can be identified and expressed in spite of culture differences. [Ekman & Friesen, 1975]

THE ROLE OF BODY MOVEMENT IN JUDGING EMOTION

Body movement is thought to be the oldest expression forms to interpret emotions [Sachs, 1937].
Additionally, dance is thought to be a prime way to express and convey the emotions in non-verbal communication. In the poetics wrote by Aristotle, he describes the dance as “to imitate character, emotion and action by rhythmic movement.” However, here comes the questions. Whether the untrained individuals can convey emotion by their body movement? And what features of body movement can be utilized to express the emotions?

Ekman and Friesen previewed untrained clinical subject, and came into the conclusion that the position of head, force and degree of openness are important and general feature in expression certain emotions, such as depression and anxiety. Moreover, Walk and Homan (1984) found subjects can distinguish the emotions of the dancer, when being showed the record of different dancers performing basic emotions with only point light showing the figures of the dancers. This experiment inferred that the basic emotions (sadness, happiness, fear, surprise, anger and disgust) can be identified by their body movements [Ekman, 1975]. In 1983, Harrigan & Rosenthal demonstrate that the degree of openness of arm, the direction of head and other measurement of the features of body movement played an important role in judging emotions.

The researches done by the scientists above illustrated that certain features of body movement can be used to interpret emotions, such as degree of openness, direction of the head and trunk movement.

Transient from Pablo Gnecco

THE ROLE OF FACIAL EXPRESSION IN IDENTIFYING EMOTION

Facial expression is a usual way to judge emotion (Ekman, 1973). People smile with joy or cry with sadness. What’s more, in the experience hold by Ekman and Davidson, the subject was watched disgust or Duchenne smile while EEG data of the subject’s cerebral brain activity was involuntary shown [Ekman, 1999]. Since the relationship between the changes of emotion in physiology pattern and facial expression is given, a certain association between facial expression and emotion can be recognized.

How to use facial expression in identifying the emotion? How to measure the facial expression? The term ‘smile’ or ‘sad’ is Ambiguous. So the using of nonaffective terms to describe and distinguish facial action becomes inevitable.

Reliable and voluntary emotion expression should contain a muscle movement [Ekman, 2003].In return, those expression are thought to be more trustworthy. The table below shows such a reliable muscle relate to sadness, fear, anger and enjoyment [Ekman, 2003].Then the Facial Action Coding System (FACS) is developed by Ekman and Friesen to measure the facial expressions by a set of action units (AUs). Each AU has is relate to a certain muscle in the TABLE 2. So by tracking facial features and measuring the movement of certain muscle can categorize different facial expressions. What’s more, Ekman identifies six basic emotion as discrete categories—-happiness, anger, sadness, fear, surprise, and disgust.

TABLE2                                                                                                     FIGURE 1 AU in the face
Expect for using facial expression to judge emotion, emotions can also be changed with a continuous dynamic movement occurs between the individual and the environment. Interpreting the emotions of others in environment around from body movement cues are a considerable way for effective social interaction (eg Dittrich et al 1996; Wallbott 1998). Rather than thinking of emotions as an intrapersonal thing (eg Ekman 1984), emotionality can be regarded as a dynamic relationship that occurs between the individual and the environment around (Camposet al 1989; Blair 2003).

Rana el Kaliouby: This app knows how you feel- from the look on your face

TWO-WAYS INTERACTION BETWEEN EMOTION AND ENVIORNMENT

For perception, when the image in the eyes are read by the brain, with no delay, brain will make certain behavior to react to the information got from the present situation. Nonetheless, we believes that emotions is partly made of the features in the external environment, when those features play a right role in stimulating individual’s emotions [Andy Clark, 1998].

THE INTERACTION BETWEEN COLOR AND EMOTION

From the point of view of psychological science, color vision only appears in the mammals that up to the privates. [Gregory, 1997]This can be thought as a special visual aesthetics. After all, the perception of color profoundly affects our emotional state.

In 1943, Schachtel proved the existence of relationship between color and emotion depending on a subjective experience. However, he failed in explaining the specific connections between certain emotion and color. However, then Gerard (1957) used common physiological reaction patterns illustrated some of the precise link between color and emotion. Meanwhile, in the psycho-evolutionary emotion theory that proposed by Plutchik, each emotion is connect to the different hue value, while the extent of the emotion changes the saturation and luminance of the color(Figure 2).

FIGURE 2 Plutchik’s wheel
Utilizing the color to connect facial expression and environment, a sensor for tracking the muscle movement of the face and an installation to change the color is needed. And a continuous changing should be considered, since the extent of emotions changes continuously and each emotion is transmitted in a row.

THE INTERACTION BETWEEN BODY MOVEMENT AND SPACE

In a space, however, modest, details of the craftsmanship and colors are develop to tell a story of the interior of the space, just as movement of individual’s body speak of the emotions. For example, walls are always used to describe human’s territoriality and columns are surely a celebrations of special upside stance.

Geoffrey Scott (1996) observed that there is a difference between the appearance of size of a space and the feeling of it. Nevertheless, the latter one can be affected by bodily motion and locomotion. Body spatiality, in conclusion, refer to our inside emotions and affect visual perception of the environment outside our body boundary [Bloomer and Moore, 1935]. Moreover, the memory of the environment is formed by reactions to people, and thus is dependent on the basic body actions and desire of the individual. The different features of space functions as different stimulus for individual’s body movement.

Taking dancer for an example in the spacing sensing realms, they use body movement to speak of “feeling” space. After a set of haptic exercise, so the reaction to the environment will not be a vague indescribable set of reflex actions. With the help of the haptic exercise, we can articulately recognize the interaction of the body movement and the space. However, the environment must be felt before the dancer can move confidently in the space. This indeed reflect the interaction between the body and the environment.

THE INTERACTIVE ENVIORNMENT CREATED BY LIGHT

Light is a physical phenomenon that emitted by some objects. The light contains a huge rage of frequencies, but only a narrow part can be seen by our eyes (Figure 3). Those basic color with each other to create the colors in the Plutchik’s wheel that can interactive with the emotions.

FIGURE 3 Electromagnetic Spectrum and visible light
Moreover, According to Einstein, the world has four dimensions: height, width, depth and time. Nevertheless, at the view point of psychology, eyes act as a receptor surface to be mapped of the space to characterize the brain as a receiver to analyze the data of the space [Schwartz, 1977].In creating a real-time interactive space, projection mapping is a superb method. With the help of technical recording system, the reaction of the environment can be recorded smoothly in the projection mapping system. Just like, for expressing six basic emotions, there will be several certain features changing in our face and body. Then with the help of sensor and projector, we can talk with the environment. This can be thought of a two-way interaction. The space and individual

PEOJECTS

XYZT, Adrien M and Claire B
Adrien M and Claire B create a timeless interaction through a visual language based on the body movement of the dancer. X-horizontal, Y- vertical, Z-depth and T- time, those words describe the motion of a point in space, and to deploy imaginary territories. This project operates as a meaningful example that shows the experience of body movement expressing the emotions of the dancer through the projection mapping. The dancer uses body movement to express the emotions of herself. It utilizes projection mapping, through interactive with the body movement of the dancer, creating real-time and various changing visual space with simple projected surfaces. It helps understand and feel the affinity and interaction between the dancer and light when the volume and position of the projected surfaces changes with the body movement of the dancer. Moreover, when the dancer is changing the space the brightness (a kind of color) of the space changes with the structural changing.

From the audience’s point of view, according to the changes of the volume and direction of the surface of the visual space, and changing brightness, they are watching the projection mapping and dancer ‘talking’ with each other with body movement. Through projection mapping and body movement tracking working together, this project creates a real-time visual experience to preview the dancer to express emotions through the interaction between the body movement and projection mapping.

CONCLUSION

From the theories illustrated by Ekman, the facial expression is a reliable way to express the emotion of the individuals and certain muscle in the face can be used to measure the six basic emotions, according to the FACS. What’s more, the body movement interpreting emotion is discovered in 130 years ago by Darwin. Both color and spatial changing can have effect on individual emotions. With the help of projection mapping, real-time space changing effect and color transforming can be realized. The project ‘XYZT’ is an example of using projecting mapping to make the communication between the dancer and environment coming true. As a final comment, we think using the data of the changes of AU in the subjects face to identify the emotion and act to the color changing of the light in the environment. Meanwhile, utilizing the body movement to communicate with the projection mapping to create a real-time interactive changing space.
What is the next step of the work? With the help of those measurement mode of the emotion and the current affective computing assistant, we would like to put more attention on the making use of availability of different sensors and brain interfaces to get more precise data from the facial expressing and body movement. And for the projection mapping, we are thinking of making an interactive real-time 3D space in a certain dark space.

REFERENCE:
[1]EKMAN, P. (ED.) (1973). Drawing and facial expression. New York: Academic Press.
[2]EKMAN, P. (ED.) (1982). Emotion in the human face (2nd Ed.). Cambridge: Cambridge University Press.
[3]Ekman, P. (ED.) (1984).Expression and the nature of emotion, in Approaches to Emotion E.D. K R Scherer, P.Ekman (Hillsdale, NJ: Lawrence Erlbaum Associates), page 319-343
[4]Ekman, P. (ED.) (2003). Darwin, Deception, and Facial expression, in Emotion inside out: 130 years after Darwin’s the expression of the emotions in man and animals P.Ekman, P.Ekman, page 205-222
[5]Dittrich W H, 1993.Action categories and the perception of biological motion. 15-22
[6]Wallbott H G, 1998. Body expression of emotion, European Journal of Social Psychology, 879-896
[7]Blair R J R, 2003. Facial expressions, their communicatory functions and neuro-cognitive substrates,Philosophical Transactions of the Royal Society of London, 561-572
[8]Campos J J, Campos R G, Barrett K C, 1989, Emergent themes in the study of emotional development and emotion regulation, Developmental Psychology, 394-402
[9]Campos J J, Campos R G, Barrett K C, (1989).Emergent themes in the study of emotional development and emotion regulation Developmental Psychology ,Page 394-402
[10]ChungK Lee (2005). Using Neural Network to Recognize Human Emotions from Heart Rate Variability and Skin Resistance.
[11]Negroponte Nicholas [1975], Soft Architecutre Machines, The MIT Press, p.24
[12]Rudolf Arnheim. Art and Visual Perception: A Psychology of the Creative Eye. University of
California Press, Berkeley, CA 94720, U.S.A., 1974. New version; expanded and revised edition of the 1954 original. P.245
[13] Kent C.Bloomer and Charles W.Moore 1935. Body, Memory, and Architecture, Page45-56
[14] Richard L. Gregory (1998), Eye and Brain: The psychology of seeing, Page 23-40
[15] Clark, A and Chalmers, d.(1998) “The Extended Mind,” Analysis 58, Page 10-23
[16] XYZT, Adrien M and Claire B: www.youtube.com/watch?v=spclH4iGB2g