This paper explores the semiotic link between architectural spaces and film spaces in order to find new methods of designing in the real world.  There are many elements in which architecture and film overlap such as space, movement, visuals, etc.  These elements will be explored throughout this paper on a semiotic level as they can be considered key elements in its understanding.  The research will be used in experiments that were conducted with regards to the semiotics of architecture and film. The results of these experiments manifests into a bigger understanding of semiotics and how it can be used as the new frontier for new designing methods in the physical realm of architecture.

1.0 Architecture in the Physical World vs Architecture in Film Environment

“A viewer constructs a general picture of space in a scene, not directly based on the detailed spatial structure perceived on the screen, but rather through more general mechanisms that support their construction of a contagious off-screen space, this space is not visible on the screen but still plays an important narrative role for the scene.” (Tseng, 2017).  The previous quote raises the question of whether or not the semiotic link between architecture and film would result in a new understanding of designing physical architecture.  How are the spatial elements in a movie scene being communicated to the viewer?  What is the difference between perceiving architecture in a film environment and the physical world from a semiotic level? What is the relation between both?  Since film requires architectural spaces to further illustrate the narrative of movies, would it be possible to use film to help in the making of architectural spaces?  That being said, this paper will tackle multiple movie genres, explore them from a semiotic point of view, and then comparing them with the semiotics of architectural spaces. This comparison will be explored through the lens of photogrammetry and machine learning to have a better understanding of the relationship between both mediums.

2.0 Semiotics 

“Semiotics is usually defined as a general philosophical theory dealing with the production of signs and symbols as part of code systems which are used to communicate information.” (Tobin, 1990).  Semiotics communicate information to people unconsciously, e.g. a door informs people that it is a way to go through.  That is a sign; however, it is not a definite sign, and in order to understand this better the period of the initial establishment of semiotics must be looked at.  Ferdinand de Saussure and Charles Sanders Peirce are two philosophers who helped in establishing the term “semiotic.” Saussure believed that there is no actual relationship between a signifier and a signified, i.e. there is not one absolute relationship between saying the word tree (signifier) and the actual tree object (signified).

Where Peirce believed that there are three types of sign systems (Figure 1) which are: icon, a representation of an object in some kind of visual resemblance; index, referring to the origins of an object through a physical or material representation; and symbols, a representation of a word which does not have a connection to what it signifies, (Darrodi, 2012), i.e. a heart icon is an indication of love, but it is not related to the actual heart. (FilmInTheMaking, 2016).  Moreover, since semiology is the study of cultural phenomena being communicated to people, then color can be considered a communicative factor in the language of semiotics.

2.0.1 Color Semiotics

Colors are considered a strong communicative factor which varies from one culture to another.  E.g. in western society, the red color is associated with danger, where in china it is associated with good luck.  Although colors are a vast complex topic, it will only be studied in terms of semiotics.  Colors are considered to be a powerful emotional trigger in any type of communication associated with past and present.  Maryam Darrodi used Keith Kenny’s logic on colors semiotics to ask the question: when looking at a printout paper with the color red for example, does it trigger a certain memory in people? Does it trigger a certain emotion? Do people relive an experience of the past? Darrodi further explains that colors help people’s understanding of the physical objects represented to them or in the understanding of a cultural idea. (Darrodi, 2012).Darrodi explains how there are at least three types of color semiotics which are innate, socio-economic, and socio-cultural. Innate is explained as having people born with a predisposition that allows them to have predetermined association for certain colors, e.g. red is danger. The second one, socio-economic, e.g. the representation of colors like purple with royalty in some cultures due to the fact that purple dyestuffs were once more expensive than gold. Lastly, socio-cultural, e.g. in china, having money being handed in red envelopes most likely resulted in them associating good luck with the color red.  (Darrodi, 2012).  Understanding this, there is no doubt that the meaning of colors will always be differentiatedfrom one culture to another.  That being said, not all color meanings are different as there are objects that are unified in color meaning.  Having understood the definition of semiotics and color semiotics, it is important to understand the semiotics in terms of space, with the understanding that space in this context is one where it is occupied by people and objects in order to analyze it with regards to architecture and film.

2.1 Semiotic Space

“The study of space as a semiotic phenomenon suggests that the meaning of space, as a sign is generally understood in relation to other concerns.  In everyday life space is rarely considered for its independent qualities, but is more generally taken as a category of conceptions that act as background to other objects and relationships,” (Gaines, 2006) e.g. the distance between objects whos

e meanings are dependent on spatial relations.  Based on the previous statement, Gaines mentions how an empty space does not present any form of communication to people.  That might be considered partially true as it is not definite.  Robert McMurtrie describes in his book, The Semiotics of Movement in Space, the difference between a semioticised space and non-semioticised one. He states that a space which is out of people’s reach, such as unreached galaxies or undiscovered areas of the earth is a non-semioticised space; however, a space that is occupied by people is semioticised since it is socially realized. (McMurtrie, 2016).  This is why Gaines’s theory is considered partially right, since an unoccupied space can be interpreted as an unreached galaxy or a deep ocean.  Further exploring McMurtrie’s theory on spatial semiotics, his deconstruction of space went to a deeper level which will be explored throughout this chapter. That being said, before dissecting his analysis of space the semiotics of architecture has to be studied, as architecture is considered one of the major deconstructed factors in McMurtrie’s theory.

2.2 Semiotics of Architecture

Architecture has been known as a medium in which one would transform an abstract idea into a physical object (Gardner, 1987).  When relating architecture in the context of the built environment, it is usually perceived in a matter of a function or a form. (Leach, 1996).  A school building gives the indication that this space’s function is to teach young people. A monumental building might evoke the feeling of it being a governmental building or a public building through the form of it, which could illustrate greatness based on the general current understanding of society.  Having said that, it is worth noting that one of the ways to understand the function or the form of a building is in fact done through communication. Since semiotics deals with having all cultural phenomena being communicated to people, a building is another medium being communicated. This public perception of buildings is based on a shared cultural understanding within society.  However, having the materiality and volume of the building being communicated to people is not the only way in understanding the building, as movement plays an important role in that aspect too.

2.3 Semiotics of Movement in an Architectural Space 

“We don’t just ‘read’ architectonic texts; we move in them, and we interact with objects in their spaces.  As we move through, we create movement patterns in relation to the architectonic and the curatorial texts, and it is this relationship that creates the grammar of movement in space.” (McMurtie, 2016).  As it can be seen from the previous statement, understanding architecture is not only done through what people perceive from materials, lighting system, and scale, but it also includes the movement of people through the architectural space.  The movement is not limited to the physical properties of one’s movement in relation to the space or the objects in the space, it also includes the movement of one person in relation to another or to a group of people.  It is through movement and interaction between people that a space becomes realized. (Revak, 2014).  It has been mentioned earlier that a semioticised space requires social activity, and social activity is associated through movement and interaction between people creating a structurally semioticised space.  In Figure 2, McMurtie’s further illustrates the decomposition of space, as he breaks down the structurally semioticised spaces into architecture and non-architecture.

His definition of architecture and non-architecture is debatable, where architecture is defined by internal spaces and non-architecture is not.  This debate however, will further be explored in 2.3.2.  Meanwhile, his definition of architecture does support the decomposition of space in his theory and in practice.McMurtrie explains how an internal architectural space is defined by both matter and void. Matter is the volumetric surfaces such as walls, floors, ceilings, which are impenetrable.  Void is the empty spaces or openings in these surfaces, which are penetrable.  If a space is structurally semioticised, that leads the space to other connected spaces.  The spaces are defined by matter, and the connection between these spaces is defined by the void in the matter.  The navigation through these spaces is done through ambient spaces in relation to hodological spaces.

2.3.1 Ambient Space vs Hodological Space

Ambient and hodological spaces are two types of spaces with direct relation to movement in an architecture space.  An ambient space is the physical spatial properties of a space which is defined by surface and objects.  This space is initially unoccupied with the potential to be moved through and occupied by people.  Hodological implies a path or a way taken from the Greek word “odos,” (Charitos, 1996).  Thus, a hodological space is the movement created by the person moving through the unoccupied ambient space, it is not the distance covered by the person moving, as it is the movement itself. (McMurtrie, 2016).  It is worth noting that a hodological space is also referred to as the shortest distance between two objects or two places regardless of the intention of moving from one point to the other. (Lee, 2017).“Your activity of walking is not in “space-of-possible-movement” but your own hodological (intention based) space.” (Charitos, 1996).  According to the previous quote, people’s hodological space through an ambient space is achieved through a previous experience of encountering other spaces in the past.  That forms an expectation of the destination, whether it is visually perceptual or not.  It also helps the person to generate the shortest path or shortest hodological space from one ambient space to another.  The concept of an ambient-hodological space is connected with people’s memory of encountering new spaces in the past in relation to creating the shortest path in the present.  It is important to understand this as this concept can be used to create a better connection between architectural spaces.

2.3.2 Connection between Architectural Spaces

Throughout this paper, the term “architectural space” has been emphasized in relation to the semiotics of space instead of using the term “space” only.  To justify the purpose behind this, an explanation of the basic fundamental principles of architecture must be presented.  Contradictory to McMurtie’s definition of architecture, it is a carefully designed structure that generates the form of an object to create a function. (Oxford Dictionaries | English, 2017).  This is not exclusive to buildings as any object with a designed structure can be considered an architectural object.  An example giving by Oxford Dictionaries is presenting the human brain as an architectural object.  The biological organs of the human brain are connected together to form the functionality of the mind.  The hardware of a computer consists of the motherboard, central processing unit (CPU), random access memory (RAM), etc. (Techopedia.com, 2017).  All are structurally designed to connect with one another to ultimately form the function of the computer, which is considered architecture or computer architecture.  That being said, if an element in the structure of the computer or the human brain was to be removed, then an error in the functionality occurs. This causes the computer or the person to stop functioning properly; therefore, a disturbance in the architecture occurs, and the same logic can be applied to architectural spaces. An architectural space is not one where it consists of one space but multiple ones.  These are volumetric spaces on a human scale level, which are connected together in a network system. (Ankerl, 1981).  If one space is less operative than the other in the same architectural building then the building is not architecturally realized.  In order to define a building as an architectural one the spaces inside the building have to be structurally semioticised and connected with one another.  The realization of such a concept would potentially be achieved through the understanding of hodological spaces with regards to the ambient ones.  Movement is also a significant factor when designing architectural spaces, meaning movement will be used as a key factor in the analysis of film semiotics.

2.4 Semiotics in Film

Film is a medium dealing with the idea of representing storytelling and narrative on the big screen. As a medium of communication, film is inherently semiotic and each aspect of the film transports a meaning of the scene to the viewer.  Stanley Kubrick’s Eyes Wide Shut provides  an example of this. (Eyes Wide Shut, 1991).  The first scene of the film (Figure: 3) depicts a wealthy family getting ready for an event. The context of the scene shows elements of wealth  such as the hotel suite the family is in, the suit the main character is wearing, and the wallet he is looking for. These are elements that communicate to viewers the social class of the family they are looking at. Another aspect in the scene, referring back to the philosopher’s theory on semiotics, is having the main character being called out by his name Bill, alluding to its resemblance to the dollar bill.  In this situation, the signifier is calling out the character’s name, and the signified is his actual name. (FilmInTheMaking, 2016). Similarities in the importance of how buildings communicate to people not just by their materiality, but also by the movement of people in the space, are evident in the semiology of film. It includes not just the static objects and environments on the screen, but also the movement of characters in the spatial scene to add depth to the flat screen of cinema.

2.5 Semiotics of Movement in Filmic Space

“The filmic path is a modern version of the architectural itinerary.… She who wanders through a building or a site acts precisely like a film spectator absorbing and connecting visual spaces. The changing position of a body in space creates architectural and cinematic grounds. The consumer of architectural (viewing) space is the prototype of the film spectator.” (Bruno, 1997).  Following the previous statement, both architecture and film incorporate movement. Unlike the physical movement in architecture, film involves more of a movement perceived in the mind.  A movement that is generated by what the camera of film is conveying adds a sense to the spatial properties of a scene.One of the aspects of movements in film is lateral movement, which is the movement of characters parallel to the flat screen in films.  Lateral movement is considered a significant aspect in film psychology. Throughout the years, numerous directors have used lateral movement in their films as a psychological elaboration factor on the narrative of their scenes.  In the psychology of films, it is believed that if a scene portrayed people moving on the screen from left to right, then that scene would be perceived to the viewer as a positive scene. It would not be the same situation from right to left. (Nugent, 2016).  One of the reasons for this is due to how people in western culture perceive time and language. Left to right indicates progression of time, literature of western culture starts from left to right, and from this the analogy about positive and negative perception from films came to be. That being said, it is not an absolute definition of how lateral movement works in film; it is however, how film scholars interpreted it in western films.Director Stanley Kubrick illustrated that in his film Full Metal Jacket (Full Metal Jacket, 1987). In the ending scene of the movie the soldiers are walking within burning buildings. The shot started with the soldiers walking from left to right (Figure: 4), then it showed them walking from right to left (Figure: 5). Kubrick’s intention of that is to give the illusion that the soldiers have an unclear object in that war by moving in both a positive and negative direction within the same scene.  It can be seen that the relation between the semiotics of movement and lateral movement is apparent, since lateral movement has been used to semiotically convey the seemingly unclear objective of the war in the film.  It is possibly an aspect that can be looked at when designing semioticised spaces in both physical architecture and film.

2.6 The Semiotics of Long Takes Vs Short Takes

Different methods in filming convey different meanings for particular scenes.  It is believed that shooting long takes of a particular scene in movies can add more to the realism of a scene than shooting short takes. (Filmreference.com, 2017).  There are a variety of reasons why a director would choose to film a long take.  That being said, it can be agreed upon that the main objective of shooting these scenes is to communicate a specific amount of information from the narrative of films.Referring back to the hotel suite scene from Eyes Wide Shut.  The scene is a continuous shot which lasts 1:00 minute exactly. Having the scene being shot in one long take gives a better understanding of the spatial properties of the suite, especially the fact that the movement of the camera framed the space entirely.  Another movie that showed one of the longest takes in the history of films is Atonement (Atonement, 2007).  The beach scene in which the soldiers were waiting to leave the war zone and were shot in a take lasting a duration of over five minutes. (Figure: 6).  The five-minute scene depicted the causality of war such as injured soldiers lying on the ground, displaced families, and destruction of life and civilization.  The long take of the beach captured the suffering of these soldiers that resulted from experiencing a seemingly endless war. (Mosot, 2012).

It is worth noting that the sorrow in the soldiers was noticeable due to the camera movement throughout the beach, which also illustrated the enormity of the beach.  Showing the inescapable massive beach, the viewer semiotically understands the suffering of the soldiers being in the middle of nowhere with no hope for survival.   Semiotically, people’s understanding of architecture in the physical world is continuous, and is a major factor of interpreting realism in life.  Instead of visually trying to connect the shots of films to generate the spatial properties of a scene in people’s head, viewing a continuous shot in film can drastically help in understanding the scene.

3.0 Architecture to film

Understanding how the understanding of semiotics assists architecture and film in translating information to people, this chapter will focus on how the semiotics representation in architecture contributes to the translation of film’s narrative to the viewer.

Taking the movie Gattaca (Gattaca, 1997) directed by Andrew Niccol as an example will help elaborate further.  A sci-fi film set in a future, the plot of the movie illustrates the cruelty of the world where a person’s life is already defined by his or her DNA at birth.  It was important for Niccol to choose a type of architecture that can show off that cruelty of that world.  Therefore, Niccol shot his scenes in locations such as Marin County Civic Center by Frank Lloyd Wright. (Figure: 7).

The reason behind such a location is due to the type of architecture that this building speaks illustrating the narrative of the film.  The long corridors with four different lozenge shaped atria that gets wider going up the building, and the repetition of curves of the building both became a station to outer space. In the language of the film, this illustrates an escape from the strict laws of the film’s world. (Tobe, 2016)  With the understanding that was acquired from the semiotics of movement, Wright’s building with its long corridors and repeated curves throughout directed a repeated linear movement of the characters, which reflected on the narrative of the film of having strict laws in such a society.  This is an example of how the semiotics of movement in architecture of the physical world can be used to elaborate on the narrative of films.

Blade Runner is another Sci-fi film (Blade Runner, 1982) that depicts the city of Los Angeles in 2019 where humans have made “Replicants” to send on missions off the planet earth.  Eventually replicants became unwanted and were forced to retire if they were to return to earth.  The futuristic city is divided between the industrialized skyscrapers and the long horizontal sprawl of the low rise.  There is a definite differentiation in the film between the low class society and the high class.  The high class would live in towers and large scale houses.  The low class would dominate the streets of LA.  Taking an example of one of the high class people apartments,  Dr. Eldon Tyrell lives in an apartment on top of pyramid shaped building which was inspired by the Mayan temples. (Figure: 8).  The apartment represent some of the elements of a futuristic apartment, including high ceilings, oversized doors, and a big empty volume in the space. (Fortin, 2016).

A good observation to be noted in here is the movement of people in the apartment.  In Tyrell’s apartment, the scale of his apartment was easily noticeable with relation to his movement in the apartment. (Figure: 9). Humans are considered a major scale factor in architectural spaces.  If there was no one at the apartment while it was being filmed, then the scale perception of that space might have differed.

This is an example of how architecture elements such as scale have a great value to add for movie narratives.  Perceiving the scale of the apartment enlightened the viewer semiotically of the kind of supremacy Tyrell has over the dystopian city of Los Angeles.  It has become evident that the semiotics of architecture can add to the narrative of films.  The next question to be asked is whether the semiotics of films can add in the making of architectural spaces.

4.0 Film to architecture

In the ongoing link between film and architecture, one would ask the question of the possibility to learn from film semiotics to find new methods of designing in architecture in the physical world?  The problem with film is the fact that it is a 2D representational medium.  The viewer’s sense of depth in the movie is established through the spatial elements in the scenes.  The distance between objects and the movement of people in the filmic space are elements that help in imagining the 3D environment presented on the 2D screen.American philosopher Nelson Goodman proposed a method in his book “Ways of Worldmaking” in which he said “Much but by no means all worldmaking consists of taking apart and putting together, often conjointly: on the one hand, of dividing wholes into parts and partitioning kinds into subspecies, analyzing complexes into component features, drawing distinctions; on the other hand, of composing wholes and kinds out of parts and members and subclasses,  combining features into complexes, and making connections.” (Goodman, 1978).  Therefore, the major components in film such as spatial characteristics, materials, scale, and movement should be decomposed from film, analyzed, and then recomposed back again in the physical world. However, for the sake of this paper, the semiotics of movement will be the major component to be decomposed from film and translated into physical architecture.  This decomposition is due to the commonly important factor of movement between both mediums.  Jean Novel mentions that architecture and film do intersect with regards to the concept of movement, although this movement in both mediums differs drastically.  In an architectural space the person is physically moving through the space, where in film the person is sitting still as he or she views it. (Jadoon, 2015).  This concept of movement in architecture and film has been explored through the montage of Sergei Eisenstein and Le Corbusier.

4.1 Montage of Film to Architecture

The relationship between cinema and architecture has existed as far as the beginning of the 20th century. (Jadoon, 2015).  Le Corbusier, a well-known architect who is a fan of cinema, and Sergei Eisenstein, a filmmaker who has considered architecture as the only contemporary art along with films, have met in 1928 (Figure 10) to exchange work in a conversation (Jadoon, 2015). Both of them were exploring the idea of montage in each field.

Eisenstein believed in the montage of film being something that is perceived in the mind, which would occur through the illusionary path in film that encounters a multiplicity of phenomena and is aligned in a certain sequence. Learning from the montage of Eisenstein, Corbusier created his own version in architecture.  The montage in architecture which was viewed as the movement of a person through a sequence of architectural shots eventually led to the architectural promenade created by Corbusier. (O’Donnell, 2015).

Eisenstein was inspired by Auguste Choisy’s analysis of the Acropolis in Athens (Figure: 11), in which Choisy believed that the experience of the Acropolis is only done through a movement sequence created by the viewer.  Each visitor will have a different impression of the Acropolis depending on their body and head position, as well as the sequential movement through the space.  These impressions that are captured by each visitor are significant in relation to themselves, and in relation to other people capturing the same moment.  Using his analysis, Eisenstein interpreted the experience created by each person as the individual shot in cinema, where the relation between these impressions is represented as a cinematic montage. (Forget, 2013). Eisenstein was a well-trained architect who always explored the potential of architecture in film and the other way around. He did believe that the architectural experience in films would never be the same as of that of the physical world.  “… I would only ask you to look at it with the eye of the filmmaker: it is hard to imagine a montage sequence for an architectural ensemble more subtly composed, shot by shot, than the one that our legs create by walking among the buildings of the Acropolis.” (Eisenstein, Bois, and Glenny 1989). Even though Eisenstein did not pursue architecture directly, he did inspire architects with his cinematographic thinking. One of them was Le Corbusier. Le Corbusier created his “promenade architecturale,” which is the experience of walking through a building. (Blundell Jones and Meagher, 2015). This concept he derived from the montage in films established by Eisenstein.  His architecture promenade was first seen in his Villa Savoye. (Figure: 12).  The building which is centered on a ramp (Figure: 13) that connects all the floors is almost a filmic experience, as the building keeps on unfolding through time and space (Knox, 2007) similar to how a movie keeps unfolding throughout its duration.  Since the link between architecture and film throughout the years is evident, the next chapter will focus on applying the theoretical link between both mediums to create new designing methods in the physical space of architecture.

5.1 Design Experiments

To further comprehend the link between architecture and film on the semiotic level, the team conducted a series of experiments analyzing films to understand the hidden meanings behind films and what they can provide. These experiments included machine learning and photogrammetry as the main approaches to deconstruct films.

5.2 Machine Learning 

Machine learning is a type of artificial intelligence (AI) system that creates certain algorithms which are used to receive data, and then start analyzing this data in order to create a new output based on the inputted values. (Rouse, 2017). Three applications are categorized under the concept of machine learning. These are Deep Convolutional Generative Adversarial Networks (DC-GAN), Recurrent Neural Networks (RNN), and Reverse Image Search (RIS).

5.2.1 Generative Adversarial Networks 

In DC-GAN, by inputting a specific set of datasets, the systems combines two deep neural networks that are compared with others to hallucinate the original inputted pictures. (Geitgey, 2017).  The team extracted frames of The Matrix movie (The Matrix, 1999) as a dataset, which were inputted later on into DC-GAN.  By tweaking the parameters of the application, the machine took in these frames, analyzed it, and created the newly outputted values which are shown in (Figure: 14).

It is worth noting that the overall tone of the outputted values is green, a color that The Matrix movie uses indirectly throughout the movie. When the characters of the movie are in the matrix system, a virtual reality system, which is most of the duration of the movie, the color pallet of the film includes a mix of green in it (Figure: 15).  However in the normal world, the colors are standard with no color theme mixed into them (Figure: 16). The machine extracted the green color of the movie, which in color semiotics emphasizes the representation of green in the movie. The matrix system visually has a green theme to it, or is viewed from the monitor with multiple languages colored green.  Moreover, since the matrix system is associated with technology, the green color could also represent technology when it first started with its green color screens and monitors.  The matrix system color might have been based on the common cultural understanding of how technology started.  Using machine learning with regards to film, it could be possible to extract the meaning of colors that are used in films that would semiotically communicate the technological color of the matrix system or trigger certain memories associated with this color.  Due to the output generated from DC-GAN, the team explored other possibilities in machine learning one of which is reverse image search.

 

5.2.2 Reverse Image Search

Reverse image search is another type of a machine learning system that inherently outputs values similar to the values of the inputted ones (Rouse, 2013).  Using this search engine, we have inputted frames of the top 100 movies in IMDb website in order to have the computer analyzing these frames.  Later on we have inputted a video sequence of the renowned scene from The Shining where Danny runs into the twins in the corridor.  The machine started giving a set of 25 pictures for each frame of the video sequence. For each frame of the video sequence one picture of every 25 data sets was required.  Therefore, the selection came to what is most semiotically relevant in the original video sequence (Figure: 17).  Elements such as colors, spatial depth, spatial height, and character movement were observed to coherently create a new video that can represent the same semiotic elements.

[vimeo 234718631 w=640 h=360]A process video of how to create the new video with the help of Reverse Image Search.

5.2.3 Recurrent Neural Network

Recurrent neural network (RNN) is a third type of machine learning system that generates outputs based on the information of the input.  However, this system uses sequential information as a set of inputted values.  It is a looped system which keeps on feeding itself the information as it is learning it, eventually outputting newly predicted values based on the inputted ones. (Britz, 2015).  A 3D photogrammetry model was extracted from a specific set of sequences in The Shining, in which the character Danny using his tri-cycle through the hotel’s ground floor corridor of the kitchen, the corridor on the upper floors, and the Colorado lounge.  The camera position coordinates were extracted from the photogrammetry model, and the values of these coordinates were inputted into the RNN. The machine started learning from these values to later output a newly predicted set of coordinates within the range of the inputted values (Figure: 18).

The result was a new pathway system for the camera of the film, which can be perceived as a movie directed by the computer (Figure: 19) This could potentially raise the question of the possibility to break the rules of directing movies. Where a director tries to make meaning of a scene he or she is shooting, a machine understands numbers only and use these numbers to generate new camera direction for the movie.  

[vimeo 234718639 w=640 h=360]Explaining the process of making a video with the use of Recurrent Neural Network

This could result in new understanding for design in architecture.  Derived from Eisenstein and Le Corbusier techniques of montage, can the computer understands the juxtaposition of images and their coordinates to create a new type of promenade architecturale generated from the filmic path?  Would the new filmic path semiotically communicate a different meaning of the original intended scene or would it be the same?  Looking at the possible outcomes of combining recurrent neural network with photogrammetry, we started exploring photogrammetry on a deeper level to have a better understanding of this concept.

The three machine learning models are used in creating three different interpretations of films viewed from the machine’s perspective.  These different interpretations can be viewed in our website (neuralkubrick.com), and the following video is an introduction of our website.

[vimeo 234718644 w=640 h=360]Introduction video of our website, neuralkubrick.com

5.3 Photogrammetry

Photogrammetry is a 3D measuring technique that uses still photographs as its measuring tool to output a 3D digital point clouds of the 2D photographs. (Wheeler, 2016). Photogrammetry has been used across multiple mediums, starting with topographic mapping, and then spreading to architecture, engineering, industry, and many other mediums to produce 3D data (Pillai, 2015).  The process in which photogrammetry works is by taking multiple photos of an object or a site to generate 3D data. The camera shots of the specific object should be taken from different positions around the object in order to get as much detail as possible from the object (Figure: 20). The other method is to take multiple aerial shots around the architectural site to be generated in 3D.Using this technique, in addition to extracting the frames of film it was possible to have frames of film as camera positions and put it into the photogrammetry application to get a point cloud model out of it. This has been done before with regards to films in which famous movie scenes were extracted into the application such as Blade Runner shown in (Figure: 21). The generated point cloud, which can be navigated easily, was taken from the aerial view of Los Angeles 2019’s skyline scene.  From this, a viewer can break outside of the limitation of the 2D frames of films, and experience them on a much deeper level.Through photogrammetry, we have extracted scenes from the movie, The Shining, which were deconstructed and represented in the point cloud system.

This allowed the viewer to have a different perspective of the famously known scenes of the movie.  Depending on the medium the viewer wants to use to experience these point clouds, it would be placed in a game engine or a virtual reality (VR) system to allow viewers to wander in them freely. The images below show a frame of the famous elevator scene of The Shining (Figure: 22), 

and the generated point cloud of the elevator. (Figure: 23).  The generated point cloud version of the elevator scene allows the viewer to experience the scene from a different perspective; thus, having a better understanding of the spatial characteristics of the scene.

 

5.4 Photogrammetry Experiments

With an understanding of the way photogrammetry works, a series of experiments were conducted to find the hidden semiotics within movies and communicate them to the viewer.  These experiments included the deconstructing of one movie to analyze it in its entirety, combining different photogrammetry models into one model, and extracting character movements from films.

5.4.1 A Whole Movie through Photogrammetry 

Having the capability of recreating the spatial characteristics of movies in photogrammetry, the first experiment’s purpose was to explore the possibility of experiencing the semiotics of the hotel after deconstructing the movie.  The variety in 3D models that were extracted were aligned together to recreate a realistic point cloud representation of the hotel.  The different representation models shown in (Figure: 24) were filmed using the same cameras that were used in the original film.  Since semiotics communicate the meaning of the movie through its cinematography, then would that meaning still be communicated through photogrammetry? Could photogrammetry semiotically deliver a deeper meaning for films?  This is where recurrent neural network was involved later on in the project to generate the newly predicted camera paths. This in return, could generate a new semiotic meaning of the movie.

5.4.2 Different Photogrammetry films in One Space

In the second experiment, instead of deconstructing one movie into a point cloud system, more than one movie was chosen to be imported into the photogrammetry application.  Two different genres of films were chosen, which were The Shining and The Grand Budapest Hotel. (The Grand Budapest Hotel, 2014) The reason for this is due to the type of architectural places that the scenes of the movies were shot at, as both films were featured in a hotel set throughout most of the movie (Figures 25 & 26).

Unlike their settings, the genres of both films are quite different. The Shining is a horror thriller, and The Grand Budapest is a comedy.  The hotels were deconstructed in the photogrammetry application Reality Capture, and then merged sequentially (Figure: 27).  The reason behind merging the two photogrammetry models is to highlight their semiotic significance.  The shining hotel is used as a mean to communicate a scary movie through a scary space, whereas the Budapest hotel represented the humorousness of the film.  The two models of the hotels’ corridors were analyzed from an eye level perspective (Figure: 28).  Since each of these movies serves a different semiotic purpose, would the combination of these two semiotically different corridors result in new understanding of semiotics?  Would it create a new semiotic meaning based on the mixture of these two different semiotic models?

5.4.3 Characters Movement in Photogrammetry

Since movement is considered a key element in understanding the spatial aspect of both architecture and film, it has been used as an analytical aspect when deconstructing movies into point cloud models.  The third experiment involved the extraction of actors from the frames of the film, The Shining (Figure: 29 & 30).

The purpose behind the extraction was to compare both scenes simultaneously and semiotically. The process was done by having a 2D sprite representation of the actors move through the point clouds in the unity application1 (Figure: 31); therefore, recreating the scenes of the movie and played at the same time.  Observing the different kinds of movements that occurred throughout the movie, both scenes represent a different pace in moving through the space.

The first scene consisted of Jack and Wendy being introduced to the Colorado lounge of the hotel.  The second scene consisted of the starting point Jack’s madness in wanting to kill Wendy.  The first scene had a much higher pace in movement through the space.  In the semiotics of movement, the characters’ movement pace can communicate the atmosphere of the scene in terms of it being a light or a dark scene.  Using this experiment of mimicking different scenes of the same space simultaneously, we noticed the ease in identifying the different movement pace of the space. This resulted in communicating the different atmospheres that are occurring in one space.Moreover, we started thinking about this in the sense of lateral movement.  In the psychology of films, the direction of people on the screen determines the vibe exuded from the screen.  In the first extracted scene taken from the previous experiment, the owner of the hotel is taking Jack and Wendy on a tour throughout the hotel.  When he showed them the Colorado lounge of the hotel, the camera sequence showing the movement of the characters was perceived for the viewer from right to left.  In the psychology of film that would be considered a negative scene, or at least leading to a negative scene that would happen later on in the movie, which is the case in this situation.  The lounge depicted Jack’s initial steps into madness when he started scaring Wendy and threatened to kill her, which was the other extracted scene from the movie.  Both of these scenes can be considered to be linked psychologically in terms of film.  The generated model allowed the viewer to wander and experience both of these scenes freely.  The person is no longer experiencing the initial negative scene from right to left.  He or she can visualize the link between the introduction of the Colorado lounge to Jack and Wendy, and Jack’s initial madness in the lounge.  Learning from this experiment, the viewer no longer needs the scenes of the movies semiotically presented to him or her, in fact, the viewer can be considered to be personally experiencing the semiotics at this point.

5.5 The Semiotics of Photogrammetry 

What makes a space semioticised in film’s photogrammetry?  It has been mentioned before that a non-semioticised space is a space that people do not have a spatial imagination of, while a semioticised space is one with a social interaction.  Since social interaction is associated with movement, movement was taken as a vital part in understanding the connection between film spaces.  Based on the previous experiments using photogrammetry, an approach was taken to connect movies’ sequences semiotically.  The purpose behind doing so is to observe the possibility of learning from the filmic space by utilizing the medium of photogrammetry to create new designing methods in the physical architecture space.  If films are considered to be comprehensively semioticised spaces, then it can be assumed that there are structured and non-structured semioticised spaces.  A structured semioticised space is bound by architecture, and a non-structured one is defined by the unoccupied open environment.  For the photogrammetry approach, the structured spaces of films will be the ones to be deconstructed into the point cloud system.  This is due to the fact that the potential outcome of the design project has to be influential to the architecture spaces of the physical world.

5.6 The Semiotics of Movement in Photogrammetry 

The next step of the project was to choose the film spaces that would be arranged together and aligned respectfully to each other.  The first selection of movies were based on the director Stanley Kubrick due to the strong narrative in his films which might have been seen as quite impactful.  His movies, The Shining, Eyes Wide Shut, and Full Metal Jacket were deconstructed and aligned in the Unity application.  Multiple scenes were deconstructed from each movie, but few were selected to be aligned with one another.  The selection came from the architectural point of view of having a functional architectural space. This is done by having different types of spaces connected with one another to form the functionality of the building. (Figure: 32).  The two centralized spaces, the sleeping room in Full Metal Jacket and the ritual area in Eyes Wide Shut, would be connected with the kitchen corridor of The Shining.  As it can be seen in (Figure: 33),

the corridor of the Shinning (Middle) is acting as the transition between the two centralized spaces of Full Metal Jacket (Right) and Eyes Wide Shut (Left).What is worth noting from this alignment is the visibility of the photogrammetry spaces to each other through the walls from an eye level perspective (Figure: 34). Since photogrammetry is a point cloud generating technique, it is not a solid geometry that covers the surface of walls, floors, ceiling, etc; hence, a person can see through the walls.  Additionally, as mentioned earlier, people use their previous experience of encountering places to hodologically move through a mind generated ambient spaces.  The ambient spaces in the photogrammetry model above are the ones from each film, where the hodological space is the path generated by moving from one space to another.  That being said, ambient spaces are usually perceived in the mind due to the separation of physical spaces by a fully opaque wall or the long distance between two places. The photogrammetry model breaks that rule with the gaps between the point clouds, and creates a visual connection between the ambient spaces that are waiting to be hodological. From the previous outcome, it can be safe to assume that a person would be able to visually connect between spaces in the photogrammetry model better than in a real life architectural space.  Nonetheless, understanding this concept can have an impact on the way people connect between spaces in the real world, especially when designing architectural spaces.

6.0  In Conclusion

Ultimately, photogrammetry can be a new direction that architects would benefit from when designing structures as well as spaces.  The importance of deconstructing the filmic space into a point cloud system is crucial, that is due to the way the narratives of films are being communicated to the viewer semiotically.  Understanding and learning from the semiotics of films would provide an advantageous contribution in the making of architectural spaces.  The understanding of semiotic movement in the filmic path can result in better approaches when connecting spaces in the design stage of the project, which would then lead to developing structurally semioticised spaces.

7.0 References

Tseng, C. (2017). Revisiting dynamic space in film from a semiotic perspective. [online] degruyter. Available at: https://www.degruyter.com/view/j/semi.2016.2016.issue-210/sem-2016-0050/sem-2016-0050.xml [Accessed 10 Mar. 2017].

Tobin, Y. (1990). Semiotics and linguistics. 1st ed. London: Longman, p.6.

Darrodi, M. (2012). Models of Colour Semiotics. Ph.D. The University of Leeds.

FilmInTheMaking (2016). Semiotics: What We Don’t See In Movies.

Available at: https://www.youtube.com/watch?v=3SIEnAXUC-k [Accessed 5 Apr. 2017].

Gaines, E. (2006). Communication and the Semiotics of Space. Journal of Creative Communications, 1(2), pp.173-181.

McMurtrie, R. (2016). The Semiotics of Movement in Space. [ebook] Google Books. Available at: https://books.google.co.uk/books/about/The_Semiotics_of_Movement_in_Space.html?id=QpCiDQAAQBAJ&redir_esc=y [Accessed 13 Jun. 2017].

Gardner, J. (1987). Computers and Video: Coincidental in Architectural Design. Master. Massachusetts Institute of Technology.

Leach, N. (1996). Rethinking architecture: a reader in cultural theory. 1st ed. Routledge, pp.173-186.

Revak, D. (2014). Un-semiotic space: symbolism, perception, and space. Postgraduate. University of Minnesota.

Charitos, D. (1996). Definining Existential Space in Virtual Environments. University of Strathclyde.

Lee, J. (2017). week 3: Hodological space. [online] Academic and design communication. Available at: http://2787806.weebly.com/week-3-hodological-space.html [Accessed 1 Jul. 2017].

Oxford Dictionaries | English. (2017). architecture — definition of architecture in English | Oxford Dictionaries. [online] Available at: https://en.oxforddictionaries.com/definition/architecture [Accessed 7 Jul. 2017].

Techopedia.com. (2017). What is Computer Architecture? — Definition from Techopedia. [online] Available at: https://www.techopedia.com/definition/26757/computer-architecture [Accessed 10 Jul. 2017].

Ankerl, G. (1981). “Experimental sociology of architecture: a guide to theory, research, and literature”.

Eyes Wide Shut. (1999).Directed by S. Kubrick. Hollywood: Pole Star.

Bruno, G. (1997). Site-seeing: Architecture and the Moving Image. Wide Angle, 19(4), pp.8-24.

Nugent, Jack (2016). Which Way Did He Go? Lateral Character Movement in Film.

Available at: https://www.youtube.com/watch?v=Ys8-a0yD-MM [Accessed 1 Jul. 2017].

Full Metal Jacket. (1987). [film] Directed by S. Kubrick. Hollywood: Warner Bros.

Filmreference.com. (2017). The realist tendency — Realism — film, movie, cinema, scene. [online] Available at: http://www.filmreference.com/encyclopedia/Independent-Film-Road-Movies/Realism-THE-REALIST-TENDENCY.html [Accessed 9 Jul. 2017]

Atonement. (2007). [film] Directed by J. Wright. Hollywood: Universal Pictures.

Mosot, M. (2012). Narrative in the long take » Film Adaptation. [online] Filmadaptation.qwriting.qc.cuny.edu. Available at: http://filmadatation.qwriting.qc.cuny.edu/2012/10/14/narrative-in-the-long-take-2/ [Accessed 6 Jul. 2017].

Gattaca. (1997). [film] Directed by A. Niccol. Hollywood: Jersey Films.

Tobe, R. (2016). Film, architecture and spatial imagination.

Blade Runner. (1982). [film] Directed by R. Scott. Hollywood: The Ladd Company.

Fortin, D. (2016). Architecture and science-fiction film. 1st ed. London: Routledge, pp.98-104.

Goodman, N. (1978). Ways of worldmaking. 1st ed. Johanneshov: MTM, pp.1-13.

Jadoon, M. (2015). Architecture, Film, and Movement.

O’Donnell, C. (2015). Niche tactics.

Forget, T. (2013). The construction of drawings and movies. London: Routledge.

Eisenstein, S., Bois, Y. and Glenny, M. (1989). Montage and Architecture. Assemblage, [online] (10), p.110. Available at: https://doubleoperative.files.wordpress.com/2009/12/eisenstein-sergei-m-_montage-and-architecture.pdf [Accessed 8 Jul. 2017], p. 118

Blundell Jones, P. and Meagher, M. (2015). Architecture and movement.

Knox, M. (2007). Rear Window Redux: Learning From the Architecture in Hitchcock’s Film Using 3D Modeling and Animation. Kansas State University.

Rouse, M. (2017). What is machine learning? — Definition from WhatIs.com. [online] WhatIs.com. Available at: http://whatis.techtarget.com/definition/machine-learning [Accessed 9 Jul. 2017].

Geitgey, A. (2017). Machine Learning is Fun Part 7: Abusing Generative Adversarial Networks to Make 8-bit Pixel Art. [online] Medium. Available at: https://medium.com/@ageitgey/abusing-generative-adversarial-networks-to-make-8-bit-pixel-art-e45d9b96cee7 [Accessed 13 Jul. 2017].

The Matrix. (1999). [DVD] Directed by L. Wachowski and L. Wachowski. Hollywood: Warner Bros.

Rouse, M. (2013). What is reverse image search? — Definition from WhatIs.com. [online] WhatIs.com. Available at: http://whatis.techtarget.com/definition/reverse-image-search [Accessed 2 Jul. 2017].

The Shining. (1980). [film] Directed by S. Kubrick. Hollywood: The Producer Circle.

Britz, D. (2015). Recurrent Neural Networks Tutorial, Part 1 — Introduction to RNNs. [online] WildML. Available at: http://www.wildml.com/2015/09/recurrent-neural-networks-tutorial-part-1-introduction-to-rnns/ [Accessed 1 Jul. 2017].

Wheeler, A. (2016). Photogrammetry Explained: The State of Reality Capture > ENGINEERING.com. [online] Engineering.com. Available at: http://www.engineering.com/DesignSoftware/DesignSoftwareArticles/ArticleID/13313/Photogrammetry-Explained-The-State-of-Reality-Capture.aspx [Accessed 1 Jul. 2017].

Pillai, A. (2015). A Brief Introduction to Photogrammetry and Remote Sensing ~ GIS Lounge. [online] GIS Lounge. Available at: https://www.gislounge.com/a-brief-introduction-to-photogrammetry-and-remote-sensing/ [Accessed 5 Jul. 2017].

The Grand Budapest Hotel. (2015). [film] Directed by W. Anderson. Hollywood: American Empirical Pictures.

8.0 List of Figures

Figure 1: From Left, symbol, icon, and index examples of a leaf and an airplane. From: Darrodi, M. (2012). Models of Colour Semiotics. Ph.D. The University of Leeds, p.68

Figure 2: McMurtrie’s system netweork of space diagram. From: McMurtrie, R. (2016). The Semiotics of Movement in Space. [ebook] Google Books. Available at: https://books.google.co.uk/books/about/The_Semiotics_of_Movement_in_Space.html?id=QpCiDQAAQBAJ&redir_esc=y [Accessed 13 Jun. 2017], p. 61-63

Figure 3: Stills from the opening scene in the hotel room. From: Opening Scenes (2014). Opening Scene of Eyes Wide Shut. [image] Available at: https://openingscenes.wordpress.com/2014/05/29/eyes-wide-shut-stanley-kubrick-1999/ [Accessed 13 Jun. 2017].

Figure 4: Soldiers walking left to right. From: Full Metal Jacket. (1987). [film] Directed by S. Kubrick. Hollywood: Warner Bros.

Figure 5: Soldiers walking right to left. From: Full Metal Jacket. (1987). [film] Directed by S. Kubrick. Hollywood: Warner Bros.

Figure 6: a collage of stills from the beach scene. From:  livejournal (2009). Atonement’s beach scene. [image] Available at: http://mindyourhead-ss.livejournal.com/56784.html [Accessed 7 Jul. 2017].

Figure 7: A picture of Wright’s civic center. From:  Modernist Architecture (2016). Marin County Civic Center. [image] Available at: http://modernistarchitecture.blogspot.co.uk/2016/07/a-visit-to-gattaca.html [Accessed 4 Jul. 2017].

Figure 8: An image illustrating the pyramid shaped building. From: Archdaily (2012). An image illustrating the pyramid shaped building. [image] Available at: http://www.archdaily.com/233625/films-architecture-blade-runner/blade-runner3 [Accessed 2 May 2017].

Figure 9: A still of Dr. Eldon Tyrell’s office. From: Mirror80 (2011). An image of Dr. Eldon Tyrell’s office. [image] Available at: http://mirror80.com/2011/12/mid-week-match-up-a-blade-runner-inspired-room/ [Accessed 3 May 2017].

Figure 10: The Charnel-House (2012). Le Corbusier with Sergei Eisenstein and Andrei Burov (1928). [image] Available at: https://thecharnelhouse.org/2012/11/16/le-corbusier-in-the-ussr-1928-1931/corbusier-eisenstein-burov/ [Accessed 4 Jul. 2017].

Figure 11: isenstein’s drawing analysis of Choisy’s work. From: Eisenstein, S., Bois, Y. and Glenny, M. (1989). Montage and Architecture. Assemblage, [online] (10), p.110. Available at: https://doubleoperative.files.wordpress.com/2009/12/eisenstein-sergei-m-_montage-and-architecture.pdf [Accessed 8 Jul. 2017], p. 118

Figure 12: COMPLEX (2011). Villa Savoye, France. [image] Available at: http://uk.complex.com/style/2011/10/gallery-5-great-le-corbusier-buildings/5 [Accessed 13 Jul. 2017].

Figure 13: Highlighted ramp (Red) of Villa Savoye.  From: Blundell Jones, P. and Meagher, M. (2015). Architecture and movement.

Figure 14: Generated Images from DC-GAN

Figure 15: The Character Neo in the matrix system. From: The Matrix. (1999). [DVD] Directed by L. Wachowski and L. Wachowski. Hollywood: Warner Bros.

Figure 16: Neo in the normal world. From: The Matrix. (1999). [DVD] Directed by L. Wachowski and L. Wachowski. Hollywood: Warner Bros.

Figure 17: Diagram illustrating the process of selecting the new generated scenes.

Figure 18: Diagram illustrating original & generated Path.

Figure 19: Visual Representation of the new camera path.

Figure 20: Illustration of different camera positions around an object. culturalheritageimaging (2016). Illustration of different camera positions around an object. [image] Available at: http://culturalheritageimaging.org/Technologies/Photogrammetry/ [Accessed 2 Jul. 2017].

Figure 21: A generated point cloud of the skyscrappers of 2019’s L.A.

Figure 22: Collative Learning (2008). A photo of the elevator scene in The Shinning. [image] Available at: http://www.collativelearning.com/the%20shining%20-%20chap%2012.html [Accessed 1 May 2017].

Figure 23: The constructed version of the elevator scene, was done using photogrammetry.

Figure 24: Stills form the generated photogrammetry of the hotel.

Figure 25: Danny Cycling in the corridor of the shining. From: The Shining. (1980). [film] Directed by S. Kubrick. Hollywood: The Producer Circle.

Figure 26: M. Gustave & young Mr. Mustafa walking in the Budapest corridor. The Grand Budapest Hotel. (2015). [film] Directed by W. Anderson. Hollywood: American Empirical Pictures.

Figure 27: Top and side views of the two corridors merging.

Figure 28: Eye level perspective of the two  hotel corridors

Figure 29: Jack and Wendy being introduced to the Colorado lounge.  From: The Shining. (1980). [film] Directed by S. Kubrick. Hollywood: The Producer Circle.

Figure 30: Jack’s starting point of his madness From: The Shining. (1980). [film] Directed by S. Kubrick. Hollywood: The Producer Circle.

Figure 31: 2D sprit representation with photogrammetry.

Figure 32: Orthogonal representation of architectural spaces.

Figure 33: Aligning the three photogrammetry models.Figure 34: Eye level perspective of the aligned models