This thesis proposes a new form of theatre using the design of a spatial interactive installation that will assess the relationship between performance, space and audience, thereby providing the audience with a thoroughly immersive experience. Through the use of digital technology and computer programming (processing, Unity, Kinect, projection mapping, etc) a new narrative and immersive space may be created. I shall investigate the process of a performance based interactive installation which will support the development of the dramatic space.

For this project I have selected the traditional Chinese Kunqu Opera, THE PEONY PAVILION, as my research example. Kunqu Opera is an Eastern classical dramatic art form which clearly demonstrates the interaction between performance and space. THE PEONY PAVILION tells a love story through the medium of dream beyond space and time, even beyond life and death. Nonetheless, the employment of digital technology can result in a new and better theatrical interpretation; the coming together and fusion of classical drama and contemporary digital art has the potential to give the opera a new lease of life.

As well as using this spetial interactive installation in connection with this particular opera, it will unleash the potential in the dramatic possibilities of body, space and time. Furthermore, to achieve an interactive experience for the participant and immersion in the dramatic world, the installation can also be seen in a wider context and not only in the parameters of this one particular art form. Keywords: Dramaturgy, Interactive Performance, Immersion, Bodies, Space, Time and Technology

1. Introduction

1.1 Motivation and Background
Computing has become universal and pervasive in an increasingly digital world. The application of computer technology in contemporary drama is very common, with certain dramatic productions becoming increasingly experimental and technological. Using the specific example of this ancient traditional drama one can see how the art form can be adapted for today’s audience and how it can actually transform the opera by employing new techniques. The onset of the digital era promises amazing changes in the world of dramaturgy. Many contemporary dramatists and artists in the field of dramaturgy, such as the director, choreographer and composer Klaus Obermaier, recognize that digital technology plays an important role in theatre. Yet this remains a relatively unexplored topic, both in theory and in practice.

1.2 Research Questions
In this thesis, the principal question is how may one recompose and recreate the world of dramaturgy in the digital age to achieve the desired results of interactivity and immersion? I propose that we look more closely at the role of digital technology by asking three specific questions:

1. What is the relationship between body、space、time and performance in drama? How can they interact?

2. How may one create an immersive space in dramaturgy?

3. How can digital technology help the audience to take a more interactive role?

1.3 Scope
This thesis will principally place the emphasis on interactive performance and immersive experience in the world of dramaturgy in the context of this new, digital age. So, we have two main themes: interactivity and immersion.

Firstly, the investigation into interactivity will chiefly be about the relationship between actors and audience. I shall especially focus on the interaction between the real and virtual worlds and the improvisational feedback. The merging of these two worlds is creating a new drama environment where the coexistence and interactivity of real and virtual actors performing in real time is made possible. Secondly, when investigating the immersive aspect, I shall focus on the dynamic environment produced by lighting and visual design in performance.

1.4 Theories and Contexts
This report hopes to answer the above questions in three parts. Firstly, I shall explore the relevant theory and practice in both historical and contemporary fields of dramaturgy, discussing body、space、time and immersion in dramaturgy. Secondly, I shall present research on the interactive aspect of dramaturgy and how digital technology may be applied, before putting forward a new method of practice and analysis of a new dramaturgical form in the interaction between performance and space. Thirdly and most importantly, I shall present my design art installation of THE PEONY PAVILION, demonstrating with it how one may create an interactive performance and immersive experience.

For my research project I have chosen to investigate Chinese Kunqu opera so as to create a dream space, which might thereby enable an audience to immerse itself in the space and interact with the installation and feel itself to be part of the drama. I am seeking an innovative collision between Eastern and Western cultures, coupled with an artistic blend of the ancient and the contemporary.

This design project is named INNER AWARENESS: THE DREAM OF DU LINANG and in it I shall attempt to answer the three questions posed above. The project comprises three parts: space design, visualization and animation, and interaction.

For the design I have used the principles of classical Chinese gardens, as described in the text of the opera, to create an immersive dream space. The animations, on the other hand, have been visualized and generated via computing methods using processing and projection mapping, to simulate an immersive dream space, based on dramatic principles. Thirdly and most importantly, interaction with the performance is made possible using the technology of Kinect Xbox, capturing the participants’ movement . using sensors to generate feedback reaction to the environment. Finally, based on the interdisciplinary nature of dance performance, I have also done research on the collaborative process between the media designer, choreographer and performer. By using a wide variety of approaches, my project has utilised digital technologies to create distinctive forms of interactive performance and immersive experience.

2. Dramaturgy

2.1 What is Dramaturgy?
2.1.1 Definition of Dramaturgy
According to Turner and Behrndt (DRAMATURGY AND PERFORMANCE 2008), “Dramaturgy tends to imply an observation of a play in production, the entire context of the performance event, the structuring of the artwork in all its elements.” They regard dramaturgy as an exhaustive and complicated process, and define dramaturgy’s main characteristics as analysis, structure, and composition.
“By studying all the elements which contribute to a performance , one may analyse the structure by which they are managed into a whole.” (Marianne van Kerkhoven, 2008) Adam Versenyi defines ‘dramaturgy’ as ‘the architecture of the theatrical event, involved in the confluence of components in a work and how they are constructed to generate meaning for the audience’.

2.1.2 A New Perspective of Dramaturgy
As Causey states in his book, THEATRE AND PERFORMANCE IN DIGITAL CULTURE (2006): “Technology tends to form a new perspective of dramaturgy beyond the traditional theatre, and performance can be transformed from two aspects: first, technology simulates the art of performance into an interdisciplinary format, opening minds; and mix-technology can combine performance, dance and installation art into a new performance space.” In this case, dramaturgy becomes a practical bridge between art and technology, and expands its potential uses beyond the confines of theatre. In other words, it breaks down the disciplinary boundaries of art to and allows it to expand into other areas. Interactive performance has become more significant in contemporary dramaturgy. An overview of the history of interactive performance art reveals Myron Krueger’s VIDEOPLACE (1975) to have been a pioneer in this field. Fifteen-odd years later, David Rokeby’s VERY NERVOUS SYSTEM (1991) and Rafael Lozano-Hemmer’s BODY MOVIES (2001) are two typical and outstanding investigations into interdisciplinary art. They focus on the exploration of performative technology together with physical bodies, using staging, stage sets, dance and music in a theatrical context. These artists are interested in the creation of interactive installations, intending to engage the public in participation with the performance.

2.1.3 Elements of Dramaturgy
In order to proceed with an investigation into this new type of interactive artwork, it is necessary to explore and name its elements. Salter (2007) labelled them as: body, space, time, and technology. (Figure 1) An illustration of this can be found in David Rokeby’s VERY NERVOUS SYSTEM (1991) , in which a computer observes the physical gestures of human bodies through a video camera. It translates them into an improvised music directly related to the qualities of movements themselves in real-time. This creates a direct and visceral relationship between body, space and technology (Salter, 2007).

2.2 The Body in dramaturgy
2.2.1 Body sense
The human body plays a significant part in the elements of dramaturgy. It enables the plot, creates memories, and has an impact upon the spatial environment. As the most essential part of dramaturgy, a sense of the body’s capabilities facilitates the best possible communication of the beating heart of the drama.

The application of photography and “moving-image-capture” systems transformed the perception of how body movement was understood: cutting and slicing up motion in time. The French physiologist Etienne-Jules Marey (Dagognet, 1992) tried to “track down the imperceptible, the fleeting, the tumultuous “, which escaped and tricked the naked, unaided eye. This is one of the earliest examples of this kind of body-based study. There are also more fantastic and wearable examples, such as the body extensions and prosthetic devices employed in the experiments of the early German visual artist, Rebecca Horn.

Performing artists have also experimented with technology as a means of extending the body. The choreographer and dancer Trisha Brown sought to transform the body’s relationship with weight and gravity. In her performance piece, WALKING ON THE WALL (1971), (Figure 2) she modified the commonplace relationship between the human body and gravity by having her dancers walk on surfaces perpendicular to the ground.

New electronic and computer-based technologies have had a marked influence on the use of the body in performance art: new bridges between physiology and technology are being built.

2.2.2 Body movement and Performance
Body movement and its use in performance have been studied from a dramaturgical viewpoint in relation to communication between actors. According to Waschmuth and others (2008) “parallel and highly interactive couplings between communication partners” are enabled by physical expression in a way which goes beyond mere verbal interaction. Their studies show how the body plays an essential communicative role, especially in performance.

Many artists have an interest in the potential of movement to create art. For example, the contemporary performance artist and dancer from New Orleans, Heather Hansen, elegantly captures choreography on large sheets of paper. She completes her kinetic drawings using charcoal whilst dancing, using the body’s movements to produce graphic art. Thus one art form gives birth to another. (Figure 3)

2.2.3 Real Bodies and Virtual Bodies
Following the development of digital technologies, performance artists and dramatists have become interested in finding ways to create an interaction between virtual and physical bodies. Today’s interactive dramaturgy is perhaps best defined by a performance which includes both a virtual body and a live actor. That is to say, in a live performance the audience sees real actors on stage, their bodies present in real time; while at the same time, there will be video, image, film or digital representations of the body sharing the stage. (Meike, 2006)

A very early example is Merce Cunningham’s 1997 project, in which he creates a virtual dance installation called HAND-DRAWN SPACES, using Biped and 3D Studio Max. He followed this in 1998 with a full-length live dance project called, appropriately, BIPED. (Figure 4)

In this concept, the physical body of the real actor and the virtual body are mutually dependent. The virtual body is an abstracted image of lines, shapes or other forms generated by the performing, physical body of the actor. In my opinion, the creation of this virtual body attempts to establish a form of communication with the performer himself; but this visualisation needs to be related to the text and to the drama itself: visual effects should not be indulged in for their own sake.

In my research I have done a number of tests whose aim was to establish connections between the organic body and electronic media (Figure 5), making use of the interactive relationships between performers and their environment. I used Leap Motion and Kinect to track the body’s movements; and TouchDesigner to generate the creation of colour, pixels and skeleton. In a series of experiments to simulate the physical body I set up a system which could generate images in virtual space. Using virtual body movements, light, colour and lines interacting with the actor’s performance, the space was transformed into a four-dimensional form which rotated and morphed, thereby challenging the audience’s perception.

2.3 Space in Dramaturgy
2.3.1 Spatial Perception
Robert Edmond Jones (1998) writes “I would rather see great dreams in a small place, than small dreams in a great place”. He implies that the performance space is a part of the narrative; and that different spaces tell different stories. As part of an artistic language, space becomes a core dramaturgical element. In this section I shall search for a spatial language based on both individual and collective experience.

Merleau Ponty (Capeto, 2015), proposes an understanding of human perception which combines all the sensory experiences with the human intelligence. The means by which we understand a space is innately connected with perception. For example, our perception of depth relates to visual information. Bachelard (2003) theorises that one’s reaction to a space is fundamentally related to one’s experiences through both memory and imagination. He emphasises that any given space draws out a subjective reaction; that the unique memories of an individual will affect the significance of a space. What is more, Lefebvre (2005), addressing this subject from a philosophical point of view, emphasises the importance of the body in one’s understanding of space; that the physiological rhythm of the body and social, political and economic dynamics will inform the body itself and its reactions to space. Feeling, memory, imagination, awareness, and intellect all combine to influence the sensory experience of a body. Therefore presence in a space is in itself a factor in one’s awareness of that space. (Capeto, 2015)

Historically, stage and auditorium have tended to be separate. In immersive and interactive theatre however, the space tends to be designed holistically: as an area shared by audience and performers alike. To my mind, this enhances the immersive potential of theatre. There have in recent years been some excellent examples of this approach: Punchdrunk, for example, founded in 2000, is a theatre company whose works have explored the relationship between narrative space and human body experience.

2.3.2 Immersive Spaces
The concept of immersive theatre emerges from the use of spatial perception in dramaturgy. Punchdrunk has been closely associated with the term “immersive theatre”. They transform abandoned buildings into immersive spaces to bring audience and performers together. Their approach to dramaturgy is a radical departure from traditional theatre. The progress of the drama is not defined solely by a dramatist and the performers, but is also open to the influence of the audience: they become active participants in the drama.

Asked how to define the term “immersive”, Punchdrunk’s founder Felix Barrett explains that it is bound up in allowing the audience to exist within the performance space, thus influencing the drama. “It’s the creation of parallel theatrical universes within which audiences forget that they’re an audience, and thus their status within the work shifts.” (Machon, 2013)

In immersive theatre, many elements allow an audience to participate in a performance rather than just observing it. These elements include lighting, sound, physical presence, space and technology; all of which can directly impact upon the experience and perception of the audience.

Punchdrunk’s dramaturgical approach has been demonstrated in many productions such as A MIDSUMMER NIGHT’S DREAM, TUNNEL 228 and THE MASQUE OF THE RED DEATH. In their successful production SLEEP NO MORE, （Figure 6）they broke with traditional dramaturgy and found ways to re-orientate the audience’s perception of reality so as to modify the actors’ performances and therefore the audience’s reception of those performances.Another very successful immersive space is the immersive space created by Ali Phi in 2016. （Figure 7）

This installation has multiple seasons co-existing in the space and they will shift gradually with time goes by. The life of nature will repeat the process from birth, growth, blossom to death. If the viewer stops, flowers nearby will grow more and if the viewer touches or steps on those flowers, they will fade and die. This work is not playing pre-produced videos, but instead generating these patterns and behaviours in realtime through programming. Overall speaking, the images of this piece is not duplicating what they were before, but constantly changing according to how the viewers interact with the system. The viewers will never see the same images for a second time if they miss the very second of that time. Teamlab has plenty of other installations which use similar ideas of making spaces immersive. They are different according to the site and client, but they share the attribute of getting people closer to nature in an architectural and interactive way so that people are both entertained and inspired by the ideas behind these artworks.

There are other projects that turn spaces into an immersive narrative for the audience. For example, Solenoid is a project which aims to create an Immersion Experience Symposium, and use the concept of magnetism as the element to achieve this intention.(Figure 8) Nimbes which is done by Lemercier and Ginzburg in 2017 also presents a good example of immersive space. The thesis of this work is to explore the ontology of observation and its relationship to cosmogony, notions of intelligence and individuality. (Figure 9)

2.4 Time in Dramaturgy
Another important element in dramaturgy is time. Its qualities of speed, duration, movement and rhythm can interact with the physical body, space and technology to evoke memories and dream states. Time is perceived in different ways: Aristotle emphasises the relationship between time and movement thus: “Not only do we measure movement by time, but also time by movement”. Sylviane Agacinski (2003) relates time to philosophy and metaphysics: “Existence is defined by its own limitations within time”, “Awareness of time is neither subjective nor absolute, and it cannot be separated from the empirical contents that appear to give it structure”.

2.4.1 Concepts related to Time
Concepts related to time include Media Time, Freezing Time, Pausing Time, Time Passing and Time Delay. In dramaturgy Media Time is an important component which uses video and digital methods. Dixon states in his book DIGITAL PERFORMANCE (2007) that the use of such media leads to a compression of time: real time is transformed into a contemporary experience of time.
In a media performance in which a live performer appears with his or her virtual self simultaneously, time is doubled as past and present are visually and physically merged. Agacinski describes this as “being in time and surmounting its temporality”. One’s body is “capable of simultaneously existing in time and recalling its own past”. Dixon (2007)

2.4.2 Generating Memories in Dramaturgy
The expression of memory is a significant factor in this type of dramaturgy. Dramatists are fond of making use of digital video technology as a means of expressing time and generating memory. By representing memory and the process of remembering, they establish through different media a simultaneity.

The director Cassiers, working primarily from a lyric perspective, produced a stage adaptation in 1992 of Proust’s SWANN’S WAY. In this he created a sensory inner world, animating the memories of the audience. He contrasted live video with the performance of the physical actor, thus creating a negative space to demonstrate imaginatively both what was remembered and what was forgotten. And so individual experiences could be prompted from the memories of the participants in the performance.

In my project I have made use of time delay: the projected virtual body of the actor is screened eight seconds after the real actor’s performance. （Figure 12）So, when the real actor ceases to move the projection begins. The actor’s performance is recorded and shown as a memory. Why eight seconds? The exterior time limit of the neurophysiological short term memory according to Graham (1974) amounts to eight seconds, giving instantaneously a memory of what we have seen, in the present. An actor’s performance of eight seconds before, shown on a screen, therefore seems to show no time delay.

My project was inspired by Dan Graham’s TIME DELAY ROOM of 1974. In this he built a closed-circuit TV installation in which two rooms are under surveillance from two connected cameras, two screens being on the wall of each room. Each screen shows the behaviour of the visitors to the other room with eight seconds delay. Participants get caught up in a feedback loop as they respond to the unpredictable impressions they are presented with by what they see on the screens. They are given an objective view of themselves as they observe themselves as part of a social group.To conclude, the pliability of time has always been of fundamental importance to the ontology of performance, whether in traditional, avant-garde, or digital form.

3. Interactions in Dramaturgy

3.1 Why Do We Create Interactions in dramaturgy?

1. Enhance the Dramatic Tension and Expression

There is a captivating and potentially deeply creative relationship between theatre space, performance and interactive design. Today, ever more designers are including interactive elements in their works, whilst still paying heed to the cultural status quo. The widespread availability of computer programming in theatre set design and staging means that more and more people come to see them, leading to a wider acceptance of its potential to enhance dramatic tension and expression (Benford and Giannachi, 2012)

2. Build a Better Relationship Between Actors、 Audience and Space

In terms of traditional theatre, the audience’s only experience is visual and involves no participation. The audience and actors are physically separated, and the fixed view of the audience only permits a single view of the performance. However, through using a more interactive style, the audience are able to directly participate in the creation of the drama, as well as directly affect the development of the plot itself which, in effect, generates a new and unique version of the performance and story. In addition, by being able to interact with the spatial environment and the interaction of virtual digital imagery, the performance becomes fully integrated and linked to the environment as well as immerses the audience within the performance itself.

3.2 Interactive Digital performance

1. What is digital performance

The performance studies and digital media scholar Steve Dixon coined the term ‘digital performance’ in his book of the same name. In it, he summarises the history of new media in theatre, as well as explains dance, performance and art Installation systematically. Rouse (2013) states that Dixon’s definition of digital performance is that it encapsulates all performance art which computer technologies have played a part in either in regards to its content, aesthetics, techniques or dramaturgy forms. As such, this includes various types of live theatre, performance, dance and projections which have either been digitally produced or expressed. For instance, art or virtual reality installations and performances, the use of robotics within an art context or computing theatrical works such as cyber theatre events, therefore essentially anything that has computerized components within the context of art.

2.How to Interact
Poggenpohl et al. (2004) defined interaction as a series of actions in succession, each of which is a response to previous actions as well as a precursor to and being responded to by the next action in the sequence. Through analysing the actual patterns of interaction, interventions to facilitate material support for the most efficient and desirable interaction patterns can be determined. This can include a variety of technological tools such as tracking systems, 2D or 3D projection mapping, robotics or various mechanisms. Due to the diverse nature and wide-range of functions of this disparate tools, they must be utilised in the most efficient way in correlation to their respective capabilities and context in which they are being used.

Rouse (2013） further states that these tools can also be used in relation to commentary, scenery, costumes and even props. The technology itself, due to it being essentially a dramatic tool, makes it an integral part of the overall performance, and even in some cases can play the part of a virtual actor. Being used in a didactic way allows expository material to become more relevant to the overall scene. It may also enhance the viewing experience, for instance a commentary function could allow the audience’s comments to appear actually within the performance itself. Technology can also be used for props or the costumes of characters which may enhance the quality of the performance or the overall impression of the audience.

Rouse (2013） further states that these tools can also be used in relation to commentary, scenery, costumes and even props. The technology itself, due to it being essentially a dramatic tool, makes it an integral part of the overall performance, and even in some cases can play the part of a virtual actor. Being used in a didactic way allows expository material to become more relevant to the overall scene. It may also enhance the viewing experience, for instance a commentary function could allow the audience’s comments to appear actually within the performance itself. Technology can also be used for props or the costumes of characters which may enhance the quality of the performance or the overall impression of the audience.

Through possessing an in-depth understanding of digital performance, its many facets and how it can be used to increase interaction with its audience, various different technological forms have been studied in this project to help facilitate a more interactive and immersive performance.

3.3 Spatial and Embodied Augmented Reality
As a result of the continued development of digital interactive technologies, a generation of artists who utilise such techniques are slowly emerging which is in turn, facilitating the transformation of live performances. Due to this emergence, the role of space in interactions, be they either physical or virtual, is becoming an important aspect in live art and performance. (Beira, 2016） The fusion of live performance with virtual performance is termed “mixed reality performance”, a phrase which conveys the meaning, not only of the coming together of the real and the virtual but also of the actor and the audience member. (Benford, 2011) Mixed reality performance represents a new development in the world of artistic experimentation, where it is combined the use of a plethora of digital technologies and differing performance styles to compare and contrast the real and virtual worlds, and allow them to influence each other. (Beira, 2016) Through analysing and streamlining the interaction between the individual and their respective surroundings within design-augmented areas, artists have created embodied interactions which are referred to as ‘mixed-reality performances’. (Beira, 2016) Live performance can be influenced by interactive media. Real and virtual elements collaborate to establish new perceptions of reality.The path which this type of performance has taken: utilising new technologies, forms and spaces, has created a demand for dramaturgical specialists who have moved the art form beyond the simply literary into areas which combine the theatrical and balletic with the technical and production elements. (Eckersall, 2006)

3.4 Interactive Performance and Technology
This research seeks to further explore technology as a means to augment sensory experiences in addition to exploring tangible and haptic interaction. Combining several technical resources such as physical fabrication, computing technologies involving capacitive sensors, Arduino, Kinect, motion tracking, visual computing and projection mapping was the aim of the practical investigation.

As a result of this combination, the research used both body and space, in the context of space aesthetics, to provide the presence of space with elements of rhythm, movement and timely dynamics. As a result of the rapid development of the use of digital technologies within the theatre, many questions and issues are still being considered. Rouse (2013) focused on three specific areas which were textual justification, aesthetic cohesion and lastly, functionality. The larger questions were how can technology aid or enhance storytelling, can it be utilised in regards to other elements of design, and whether or not it impacts or affects the performers. Perhaps the most interesting aspect is its ability to increase interaction between actors and their audience. This will specifically be looked at in further detail.

 

Combining several technical resources computing technologies involving capacitive sensors, Kinect, motion tracking, visual computing and projection mapping. Because of this combination, the research used both body and space, in the context of space aesthetics, to provide the presence of space with elements of rhythm, movement and timely dynamics.

3.4.1 Image Generating in Computer Programming
Computer programming allows us to create and generate visualisation through various means. Visualisation of movement specifically can be generative, for instance, augmentable visualisation that changes depending on movement. As such, designers are able to uncover new possibilities or dynamics through working with visualisations.(Hansen,2011)（Figure 15）

An important aspect is the flexibility of the visuals or sketches themselves as to continually create further engaging visualisations, the ability for them to be changed and re-assembled is necessary. Through the use of augmentable visuals, movement in regards to semiotics can be altered and further explored in relation to the context in which it is being used.

Additionally, interaction design is now extending past its previous limitations and boundaries, as such, designers can shape responses to movements within the design itself, therefore actual physical movement becomes an integral aspect of preliminary planning. This focus on physical movement within interaction design also enables further exploration of communications in general. In the process of a movement being visualised, certain aspects are focused upon and re-interpreted, which in itself provides further considerations to be made in regards to interaction design.

As demonstrated by Siobhan Davies’s web-based archive, advancements in digital technology facilitates immediacy and multimodality, which in turn transforms critiques into resources. In simpler terms, the creation of generative tools which can be utilised in design practice and research occurs.

3.4.2 Projection Mapping and Visualisation
The emotion which are expressed by a dancer during their performance provides valuable and unique data, which can then be analysed and later used within digital environments and provide visual artists with frameworks with which real-time events can be created. These can then be used within an augmented environment which are generated by video projections. Projector-based augmentations are comprised of video projections and aesthetic programming. These facilitate an immediate change in the relationships between body and space, both of which are integral aspects of dance and as such, allows the complexity of the performance to increase. This framework in particular has helped create the laboratory-based environment, in addition to creating a space which was digitally-enhanced that blurs the lines between the physical and the projected light and sound aspects. The projections are entwined with the human bodies that interact with them which allows the animated to become an extension of the human dancers.

The Austrian composer and visual artist Klaus Obermaier employed advanced computer vision tracking techniques and graphic simulations derived from a study of physics for his 2004 dance work, APPARITION. This involved two dancers performing in an immersive projection of moving projected fragments and images. （Figure 18）

3.4.3 Kinect and Body Tracking
Technology is an infinitely crucial aspect in regards to the continued research of body movement and motion tracking. Studies of the possibilities of the movement of the body simultaneously informs the design principles of ergonomics as well as the utilisation of digital technology, visualising and tracking. Within the realms of digital performance, the body’s movements are interpreted via the use of motion tracking. Simon Penny (2004) has stated the need for the development of an interactive framework that exceeds theories of visual art by instead utilising procedural interactive images.

Movement being processed by technology is becoming increasing common, especially in interaction design which can be seen most prominently in field of videogames such as Microsoft’s Xbox Kinect. Combining visual sensing with projection in live digital visuals is the modern day equivalent of the use of perspective within painting. It is essentially an illusion, but can also be the foundation and point of a piece of work itself, simultaneously an impressive sight to behold as well as a platform which facilitates a new form of art itself (Peter Kirn, 2011)

4. Installation Design Research

4.1 Concept of Reinterpretation Chinese Kunqu Opera The Peony Pavillion
Kunqu opera is a traditional Chinese performance art which has existed for more than 600 years. It encapsulates many different forms of art such as literature, music, dance, martial arts and drama, and as such, has greatly influenced the history of these respective disciplines. Due to its cultural importance, Kunqu opera was selected by the United Nations Educational, Scientific and Cultural Organization in 2001 as one of its intangible cultural heritages of humanity (UNESCO, 2001). One of the most famous plays of Chinese history is The Peony Pavilion which involves a young girl who dies as a result of the unrequited love of a stranger she has met only in her dreams. However, the power of another young man’s love revives her.

This story originated over 400 years ago during China’s Yuan dynasty and has since been recreated using contemporary art techniques, many instances do not focus particularly on the story itself, but more on the spirit and themes of the story itself. Namely the pursuit of ideals, individuality, liberation, pursuing ones dream and to awakening to the reality of life.（FIgure 23）

My project, which was based on one of the most exciting scenes of “Peony Pavilion”, is designed to bring new interactive and immersive art forms to the audience with a fresh experience that allows the viewer to be completely immersed in Du Liniang’s dream. Walking freely in the magnificent dreamscape as designed by myself, the audience can feel the spirit of the story and feel complete immersion.

4.2 Physical Installation design

The idea of space unit design is based on the storyline spot, Chinese classical private gardens, and it uses the principle of garden design to create an abstract spatial installation. The total area is about 15.6*10.7 metres, and it is composed of 99 vertical hanging gauzes, 0.6*4 metres each, a 2700-metre wide and 4-metre tall white string stage elevation, five HD projector and three kinects. It can be mainly divided into two parts: the main watching area and performance interaction area. The main watching area is in an enclosed space before entering the interaction area. The performance interaction area includes the central stage and audience interaction area, with an 8.4*8.4*4-meter space designed with three intercluded vertical gauzes. By using tinny strings and veils, the stage is divided into dream and reality spaces, and it interacts with performers by showing digital images through projection. The audience can walk through the gauzes around the stage, watching performance from different angles, and taking part in the drama. By proper using of kinect, TouchDesigner, unit programming and projection interaction, this device reimagines the relationship between performers, audience and space, thus to create a magical interactive theatre.

Site-specific performance generates a considerable level of interactivity, because the audience member is invited actively to participate in the drama, or dynamically to influence its progress, make decisions and come to conclusions about the action. This notion of the audience member being within — rather than outside — the action, confers responsibility onto the audience member, with regard to making meaning.

The idea of space installation design is based on the storyline spot, Chinese classical private gardens, and it used the principles of Chineses garden design, creating an abstract intention space.

On the stage, technologies, such as projec-tion on multilayered gauze screens, kinect posture recognition, projection mapping, computational interaction and so on, are ap-plied to achieve the interaction between the performance and the space — as audiences seek themselves in layers of projected gauz-es and find resonance in the performance, immerging in the dreamlike scenery, and interacting with it. This is a spiritual read to The Peony Pavilion, demonstrating heroine Du Liniang’s emotional attitude towards love, and the brave and persistent pursuit after ideals. Audiences are encouraged to walk into their own dreams, to reflect themselves in the dream, and to pursue it without fear.

4.3 Immersive Argument Environment of Chinese Classical Garden
In contrast to the traditional dramaturgy of the Peony Pavilion, I have created an immersive space which participants can interact with. The concept of the space itself is that of a classical Chinese courtyard, however in this case it is more abstract in style. The dream sequence in the garden which involves the characters Du Liniang and Liu Mengmei is the focus of the piece.

The representation of the garden is generated from a point cloud model reconstructing a section of 19th century businessman Hu Xueyan’s villa, including a classical Chinese courtyard garden. First, the photographic documentation of the actual site is collected through taking onsite pictures or extracting frames from camera videos. These pictures should have enough overlap and detailed information allowing a more inclusive and precise reconstruction. Then the stills are processed by software RealityCapture to build the point cloud model. With this model, the file can be exported as xyz format, recording each point’s three-dimensional coordination as an individual parameter that could be used in any other point based software. Thus, an abstract animation representing the Garden can be made to create an immersive argument environment. (Figure28)

4.4 Computing Visual Images and Animations
With the continuous advancements of digital techniques, computer programming in regards to the design processes of dramaturgy is becoming increasingly used by artists and designers, especially in regards to visual imagery and animation. As such, I have also utilised computing in regards to processing through which I have created an abstract visual animation based on the storyline. To create and sufficiently encapsulate the idea of a dream-like space, I have used various types of abstract visual effects.

4.5 Design of Interactivity
Throughout this project, I have utilized 3d projection mapping design within a three-dimensional space to convey the plot, in addition to giving form to the characters on stage. Through using Kinect to interact with the actors, as well as computer graphics, human-computer interaction technology, sensor technology, and artificial intelligence, these aspects combined provide a more visual, auditory, tactile, and sensory stimulating experience. Depth sensors have also been used alongside 3D projection mapping in the preliminary investigation of using augmented environments. Alongside this, current choreographic research in relation to the interaction design process has also been analysed which has helped formulate concepts relevant to such a design.(FIgure 27)

The software package Touchdesigner has been used as the main application due to its logical nature in regards to designing interactive performances. In addition, I have also engaged in experiments to calibrate specific aspects of the interactive performance such as colour, pixels, light on the body, skeletons and time delays.  Through sensing the body as well as working with virtual 3D environments, new layers of augmentation and interaction have been established as well as mixed-reality environments for embodied improvisational self-expression.

The audience are free to walk wherever they choose to within the space and whilst doing so, their bodies are tracked which allows them to interact with their virtual selves, the actors and even the projection effects. This means that the audience are aware and a part of creating the meaning of the performance itself. The dramaturgy of the piece itself creates the vocabulary of discussion alongside identifying various ‘register shifts’, in addition to negotiating deliberately ambiguous or unclear structures of contemporary work. Conversely, contemporary work could also be seen as exposing and expressing its own respective dramaturgical processes, and thus draws on and defines the dramaturgical sensibility of the writer and director as well as the audience themselves.

At its most basic level, the live encounter within the interactive space is potentially represented by the relationship between the actor and the audience which presupposes interaction and critique to a certain extent, in addition to shared imaginative engagement. This allows the audience to enter a state of imagination, and share the feeling of creation that exists within all art forms.

5. Conclusion

5.1 Conclusion
This thesis, entitled ‘Design an art installation to create an immersive space for Chinese Kunqu opera “The Peony Pavilion”’, proposes a new form of dramaturgy which is inspired by the digital age. Through utilising digital technology such as processing, Unity, Kinect, projection mapping and computer programming, I have attempted to redefine the relationship that exists between performance, space and the audience by creating and providing the audience with an interactive and immersive space.

This report was comprised of three sections, the first focused on a new perspective of dramaturgy alongside aiming to further understand the relationship between elements of the body, space and time. Through analysing from a theoretical, technical and artistic framework perspective, I have been able to explain how each element can breathe new life into dramaturgy.

The second section focused on interactive performance. Through sensing the body within virtual environments, new layers of augmentation and interaction are created, which ultimately creates mixed-reality environments determined by embodied and improvisational self-expression. This section also contributes to the field of media studies by identifying and generating new architectures and interactions of virtual embodiment and mixed-reality performance. In addition, the interactive systems which have been created can greatly contribute to the understanding and knowledge of improving the interdisciplinary practice of mixed-reality performance.

The last section involved the design of the installation dramaturgy, the concept of which was the creation of a new form of dramaturgy alongside the re-interpretation of the Chinese Kungqu Opera “The Peony Pavilion”. To achieve this, an augmented environment was generated via the use of computer programming, projection mapping to immerse the actors and audience alike, as well as the use of TouchDesigner to facilitate an interactive performance. All of these elements facilitated a thoroughly immersive experience for the audience.

In summary, this report focused on achieving interactive performance within an immersive environment via the utilisation of digital methods and visual art. It also aimed to create sensorial and experiential visual feedback through the use of advanced media design as well as co-creating an interdisciplinary exchange and link between the performance, space, audience and the dramaturgy. As a result of this study, I have reached the conclusion that to create a new form of dramaturgy, the use of digital methods is incredibly helpful due to its immersive and interactive nature which essentially opens up an infinite amount of new possibilities to this previously fairly rigid and traditional art form.

5.2 Further Studies
In the future, I aim to create a practical guide for the artistic, conceptual and technical implementation of mixed-reality performance. Within this guide, I plan to outline visual programming and media design relating to sensing technologies (input) and projection-based augmentations (output). In addition, a theoretical and critical analysis of the performances in dramaturgy via the use of reporting and analysing creative applications will also be included alongside mapping strategies of motion tracking systems as well as projector-based augmentations. Finally, I also seek to further explore augmented reality’s potential within the context of contemporary dance performance through projector-based augmentations to increase the level of interaction between the space, actors and audience.

Bibliography

Amerika, M.M.N., 2007. Meta/data: a digital poetics, Cambridge, Mass. ; London: MIT Press.

Aronson, A., 2010. “The Future of Scenography.” Theatre Design & Technology, pp. 84-87. Auslander, P., 2008.

Liveness: Performance in a Mediatized Culture. New York, NY: Routledge.

Baugh, C., 2005. Theatre, Performance, and Technology: The Development of Scenography in the Twentieth Century. Palgrave Macmillan: New York, NY.

Beira, J. F., 2017. 3D [Embodied] projection mapping and sensing bodies : a study in interactive dance performance.

Benford, S. and Giannachi, G., 2012. Interaction as performance. interactions, 19(3), pp.38—43.

Bühling, R. et al., 2011. Adaptive art – A shape language driven approach to communicate dramaturgy and mood. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 7069, pp.290—293.

Cao, M. et al., 2014. Engaging the body in immersive theatre: Sleep No More and the rhetoric of arrhythmic experience. Engaging the body in immersive theatre: Sleep No More and the rhetoric of arrhythmic experience.

Carroll, N., David, K. and David, Z. S. eds., 2006. Philosophy and Drama: Performance, Interpretation, and Intentionality. Staging Philosophy: Intersections of Theatre, Performance, and Philosophy. University of Michigan Press: Ann Arbor, MI, pp. 104-121.

Cash, S., 2013. The Shape of Space: Laban Movement Analysis as a Methodology for Dance Dramaturgy. Canadian Theatre Review, 155, pp.29-32.

Cathy, T. and Synne, B., 2008. Dramaturgy and Performance. New York: Palgrave Macmillan, pp.18. Causey, M.M.N., 2006. Theatre and performance in digital culture : from simulation to embeddedness, London: Routledge.

Chapple, F.M.N., Kattenbelt, C. and International Federation for Theatre Research., 2006. Theatre Intermediality Working Group: Intermediality in theatre and performance, Amsterdam: Rodopi.

Chen, G. F., 2014. Intangible cultural heritage preservation: An exploratory study of digitization of the historical literature of Chinese Kunqu opera librettos. Journal on Computing and Cultural Heritage (JOCCH), 7(1), pp.1-16.

Chinese Academy of Arts, 2011. Intangible World Heritage Resource Database Construction and Digital Standard Specification Formulation (First Phase). Retrieved April 28, 2013, from http://www.zgysyjy. org.cn/newart/ihchina.jsp?pageNo= 3&pageSize=5&choice=next, [Accessed 19 August 2011].

Coyne, R.M.N., 1999. Technoromanticism: digital narrative, holism, and the romance of the real, Cambridge, Mass. London: MIT Press.

Dixon, S.M.N., 2007. Digital performance: a history of new media in theater, dance, performance art, and installation, Cambridge, Mass. London: MIT Press.

Dubnov, T. and Wang, C., 2015. Free-body Gesture Tracking and Augmented Reality Improvisation for Floor and Aerial Dance.

Gough, K. M., 2016. The Art of the Loop: Analogy, Aurality, History, Performance. TDR: The Drama Review: A Journal of Performance Studies, 60(1 [T229]), pp.93-115.

Hansen, L.A., 2011. Full-body movement as material for interaction design. Digital Creativity, 22(4), pp.247—262.

Henkin, S., 1998. Illuminating Vienna. The World & I, 13(8), pp.87—93.

Huang, S.C.C.C., 2013. Proactive Home Furnishings: Inspiring from interactive art for designing functional aesthetics in a space. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 8015(4), pp.367—375.

Jorgensen, U.A.A., 2012. Bodies and real-time interfaces: In video performance and interactive digital 3D installation art by VALIE EXPORT and Jette Gejl Kristensen. Journal of Aesthetics and Culture, 4, pp.1—10.

Jung, C. and Kim, D., 2016. Analyzing Media Art Used Images Commercial: Projection Mapping. International Information Institute (Tokyo). Information, 19(8B), pp.3503—3508.

Kaduri, Y. and Meyer, P.M., 2016. Sound, Image, Dance, and Space in Intermedial Theatre. In The Oxford Handbook of Sound and Image in Western Art. Oxford University Press, pp. The Oxford Handbook of Sound and Image in Western Art, Chapter 42.

Kong, H., 2012. Transfiguration of Space: Paintings, Photographs and Digital Art Installation by Kong Ho, Self-Published.

Lawrence T.C. and Louis B., 2013. Dramaturgy in the internet era. Qualitative Research in Organizations and Management: An International Journal, 8(3), pp.308-325.

Masters, P. et al., 2016. Constructing the Sensorium: Neoliberalism and Aesthetic Practice in Immersive Theatre. Constructing the Sensorium: Neoliberalism and Aesthetic Practice in Immersive Theatre.

McCormack, J. and Bown, O., 2009. Life’s What You Make: Niche Construction and Evolutionary Art. Applications of Evolutionary Computing, pp.528-537.

Mor, L., Liu, C. and Von M.S., 2011. Interactive biomimetic space: An interactive installation to explore living architecture. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 6625(2), pp.364-373.

Moser, M.A.M.N., MacLeod, D. and Banff Centre for the Arts., 1996. Immersed in technology: art and virtual environments. Cambridge, Mass: MIT Press.

Murray, J.H., 1998. Hamlet on the Holodeck: The Future of Narrative in Cyberspace. Cambridge, MA: MIT Press.

Popper, F.M.N., 2007. From technological to virtual art, Cambridge, Mass. London: MIT Press. Ramos, J., 2015. (re-) constructing the actor-audience relationship in immersive theatre practice. (re-) constructing the actor-audience relationship in immersive theatre practice.

Rosenthal, S.M.N. et al., 2010. Move: choreographing you. London: Hayward Publishing.

Saltz, D.Z., 2001. Live Media: Interactive Technology and Theatre. Theatre Topics, Vol. 11. Johns Hopkins UP. pp. 107-130.

Salter, C., 2002. Entangled: technology and the transformation of performance. Cambridge/Massachusetts: MIT Press, 2010. Schechner, Richard. Performance studies: an introduction. London/New York: Routledge.

Salter, C.M.N., 2010. Entangled: technology and the transformation of performance. Cambridge, Mass, London: MIT Press.

Suvin, D., 1987. Approach to Topoanalysis and to the Paradigmatics of Dramaturgic Space. Poetics Today 8, no. 2, 1987, pp311-334.

Shi, F. et al., 2011. Gender, Body and Space: Classical Chinese Theatricality in Contemporary Intercultural and Transnational Performances. Gender, Body and Space: Classical Chinese Theatricality in Contemporary Intercultural and Transnational Performances.

Si, M.M.N., 2011. Interactive storytelling : fourth International Conference on Interactive Digital Storytelling, ICIDS 2011, Vancouver, Canada, November 28-1 December, 2011, proceedings. Berlin, New York: Springer.

Thies-Lehman, Hans., 2006.Postdramatic theatre. New York: Routledge, Turner, Cathy, and Synne Berhndt. Dramaturgy and Performance. New York: Palgrave Macmillan.

Waleson, H., 2012. Opera: Indoor ‘Pavilion’ Wall Street Journal, p.D.6.

Wilson, S.M.N., 2002. Information arts: intersections of art, science, and technology. Cambridge, Mass, London: MIT Press.

Wu, Z., Ashizawa, L. and Marsh, S., 2016. Audience Participation in Traditional Chinese Theatre. Audience Participation in Traditional Chinese Theatre.