Katriina Ilmaranta
Designing the Cues of Realism in the Virtual Set
University of Art and Design, Helsinki
Department of Design for Theater, Film and Television
L’Enfant et les Sortileges is an artistically oriented doctoral thesis. The main object of the research is to utilize the new digital set design technology in practice, and to locate some of the major problems associating with it from the viewpoint of the set design process. The research consists primarily of completing a digital set for a 50-minute- long television version of Maurice Ravel’s fantasy opera L’Enfant et les Sortileges. This TV -production will be directed by Marikki Hakola, a pioneering media artist and director of electronic moving picture in Finland. The digital set technique used in this research is specifically referred as the virtual set among the present film and television industry. The virtual set production differs from the mainstream special effects film in such a way, that the emphasis lies on not on the postproduced, but the realtime use of the computer graphics. The analytical discussion concerns to report the experiences related to the realization process of the virtual set. The virtual set is understood as being a technology of represention specifically designed to simulate the practice of the traditional cinema and to mimic the impression of a physically existing scenery environment. The virtual set is first viewed in its preproduction state as the digital image and the scenic model, and second in its production state through cinematic possibilites it provides for the (video) camera to access in it. The virtual set realization process thematizes specially the questions of realism: why it is still so important to create an impression of foundatorial perceptual realism in the filmmaking process even with the technique which is so obviously artificial in its way to construct a scenic environment? The key concepts used this research originate in film phenomenology, however, the argumentation is carried through the practical examples of the documentation of the set design process. The material conditions the virtual set depends on are interplaying with the scenic expression, and can be also linked with the common features of the computer graphics technology. The technological applications used in the L’Enfant virtual set vary in their ways of representing the perceptive acts of expressing and viewing the narrative world. There will be a comparative analysis within the singular production between the cinematic possibilities and limatations these applications provide. As a conclusion there will be speculation of the narrative cohesion the virtual set as a spatio-temporal representation provides.
2.1 Background and significance of the research
2.2 Previous activity pertaining to the scope of the topic of the research by the research team
4.1 Research environment, including the equipment to be used.
2. Background
2.1 Background and significance of the research
During the last ten years, the digital set design techniques - as a part of overall digitalisation of the film and television practice - have developed quite explosively. These new design techniques have especially been explored within two established fields of audiovisual culture, namely the digital effects film and the broadcast graphics. The applications of digital set design in film or in television differ in certain ways from each other, on the other hand they can be seen closely linked, and most often these applications also mix inseparetedly with digital animation and compositing. The virtual set, however, is even internationally relatively little utilized other than in most common ways in television news and magazine programmes, and especially true this is in the countries like Finland, where the film and television industry is small. The structure of the film and television productions here in the country of nationally oriented film and television culture is essentially different from that of the large scale international productions. Low budgets, short prepoduction times and relatively small production teams inevitably limit the scale digital set design or animation can be used in productions. Therefore there is a lot of possibilities to explore how virtual set, being one special form of digital set, could be utilized more extensively as a medium in the television productions. Relying on the interactive working process, the virtual set can be seen accessible in the small scale productions compared with the digital design techniques based on postproduction — and yet at the same time experimental production like L’Enfant could develop the knowledge and the insight not only enriching on the overall level the possibilities of television genre, but contributing the development of digital set design and animation in Finnish film. Accompanying this personal academic research there is a devolopment project carried by the producing party, the KROMA Productions Ltd, aiming at the specific application of the virtual set.
2.2 Previous activity pertaining to the scope of the topic of the research by the research team
Digital set design was launched in Finland in early 1990’s largely through the efforts in major television companies (YLE, MTV). In addition to this, digital animation and compositing has been paid some attention to in few commercial companies. Generally the broadcast graphics has been a firmer success in Finland than the digital effects film. As a set designer and art director I have been working with the virtual set ever since the beginning. At YLE I participated the Virtual Set Project in 1994, which was aimed to further the development of the virtual set in the Finnish television: the project consisted of several test programmes some of which where actually aired where as the other ones were used more for demo purposes. In 1994-95 I designed a digital set for YLE main news (on air 1995-99), this project was also accompanied by a series of testhootings, concentrating primarily on tracking the camera movement data in a virtual set system. Overall the YLE research was applied research for the company purposes focusing on the aesthetics of the digital and virtual set as well as on the practical working process with the new set design technology.
Since1997 I have worked in the UIAH, first as a researcher in the ELOMEDIA postgraduate school (1997-99) and then as a professor in design for film and television (2000-05). One of my main areas of specialization has been digital and virtual set, and I in addition to my own academic research I have furthered the research, the education and the commercial activities in the virtual studio at the LUME Audiovisual Center in the UIAH.
In addition to the mainstream film and television some new media companies have shown interest in digital effects video. Among them is the KROMA Productions Ltd, a production company for audiovisual and multimedia especially spesialised on media art, computer animation and graphics and multimedia. KROMA´s productions are distributed world widely and broadcasted by various tv-companies around the world. The works are also among several art museum collections.
Director Marikki Hakola is a pioneering media artist and director of electronic
moving picture in Finland. She is also a postgraduate student in the University of Art and Design, in Helsinki aiming at the doctoral dissertation. She has been working 20 years on the field of moving image, multimedia, performing arts and experimental television productions. Her works have been shown internationally in art museums,
festivals and television channels.
The main objective for my research is to design a virtual set for a television version of Ravel’s lyrical fantasy opera L’Enfant et les Sortileges and to focus the problems in the set realization with the new technology. As Thomas Elsaesser has pointed out, the traditional film technologies has mixed in various ways with the new digital ones. L’Enfant, however, is a project, with the minimized use of live-action footage. This means a virtual set technology plays very important role in a production as an expressive whole and the medium in itself won’t be unnoticed. In the mainstream digital effects film there will be still usually a considerable amount of non-digital footage. Also, the traditional cinematic location consists of lived-world spaces, either being constructed or discovered places. The digital set will work out more as an addition to these spaces. L’Enfant is designed to be entirely based on the virtual set technology, where all the characters are shot in the bluescreen studio and realtime composited against the digital backgrounds. This means that the cinematic world the production suggests is uttermost artificial being fully digitally generated. While there exists a situation, where whole the set seems to be most constructive in its realization, there arises also a question, how it can provide us a meaningful sense of being somewhere?
The analytical discussion is thus linked to the practice related issues within this specific set realization process. Obviously the virtual set not seen here as an isolated technology, but within the frame of the representational function of the narration. The comprehension of the scenic interpretation is, however, seen more as "given" (here I am not suggesting it as unimportant, but only as a subject demanding so broad treatment as itself that it won’t be possible to realistically cover here). Since the virtual set exists only as a digitally generated image and thus cannot be bodily inhabited, certain predefinitions are there for us to encounter the virtual set. What are the "cues" the virtual set provides as a digital image to create (or not to create) an impression of realism? Furthermore, how does the virtual set allow the filmmaker explore it, to be present in it? And finally, as a "location" of the narration the virtual set comprehends a spatio-temporal representation function, but what is the world it refers to? Is there finally a sense of being meaningfully present in the world of the digital artificial reality? All these will be the main themes summarizing the curriculum of the set design process, which hopefully are shed some light through practical example the singular project.
According to the above given layout, the experimental part and the report is organized as followed. First the orientation is on the "identity" the virtual set primarily as a technique presetting certain material conditions relating to the artistic expression. The above mentioned material conditions of the virtual set can be linked with the common knowledge in the field of computer graphics. Worthwhile is also to compare the technique of the virtual set with the one of the traditional set. The second goal is step by step spectate and comment the main themes of the preproduction, production and postproduction phases. The emphasis of the analysis is slightly shifting through all these phases. The preproduction phase is concentrating on the image generating process, questioning how the specific technical application provides the varied realization of the set design ideas. Some special attention is given, how the digital and virtual set software is organized as the specific tools for imitate the physical phenomena of the lived world and how the virtual set process is structured in the utilization of these tools. The production phase is mainly about the problems there are while the digital set is shot with the physical (non-digital) video camera in the bluescreen studio. The inaccessible set means certain limitations in terms of the cinematic practice, which need to be compensated. When there cannot be actual physical presence, there can be some imitation of the presence, but how does the digital space differ from the one of the real space in terms of the camera practice (real space meaning here the physical solid-void space). How will the presence be grounded as the bodily based viewpoints? How does the camera movement further our willingness to perceptually accept the realism of the virtual set? The postproduction phase concrentrates on evaluating the relationship of time and space in the virtual set through the work completed. What kind of spatio-temporal cohesion is there within the narrative? While the previous phases mean to search answers within the terms of practical realization of the singular project, this phase is meant to be more of a feedback of the preceeding process and even leave some suggestions open.
3.2 Hypothesis and research methods
As mentioned, the virtual set is crucially different from the traditional one in the way it exists only as a digitally generated image that actually cannot be inhabited. Since the set design is an art form associated with the need to express meaningful presence in the narrative world, the process dealing only with non-habitable space is conceptually new one. Here I am not intending to forget that there exists a strong tradition of illusion techniques among theatre, film and television set design, and these techniques resemble to some extent the virtual set. However, the virtual set is not approached in my research primarily as an illusion, as something deceivingly meant to be taken as something else. I have intended from the beginning that the virtual set does not imply the predication of the world as "real". I rather see the virtual set as an intentional structure imitating our own world as well as akin to the world of realist cinema, challenging us to also as a perceptual experience accept the spatio-temporal cohesion it provides.
The general theoretical basis for these ideas is deriven from the recent film phenomenology, and especially the discussion here is based on Vivian Sobchack’s semiotic phenomenology as it is presented in The Address of the Eye: A Phenomenology of Film Experience (Princeton, NJ: Princeton University Press, 1992). The starting point for Sobchack is that the film spectatorship is based on "the structures and pragmatics of the existential experience", yet at the same time she also recognizes that the "signification and signification as immanent, given with existence". Thus Sobchack suggests the spectatorship to be a participatory semiotic dialogue. According to Sobchack the viewers are confronted with the representation of both the perceptive acts of viewing the narrative world as well as a representation of the expressive delimitation of this world. I am aligning with Sobchack when focusing on the perceptual encounter with the virtual set (nevertheless intending to claim there will not be the constructive comprehension). To some extent I have also followed Allen Casebier’s Film and Phenomenology: Toward a Realist Theory of Cinematic Representation (New York: Cambridge University Press, 1991). Especially Casebier’s notion of perceptual mediated realism seems to be useful with certain presuppositions especially thinking that the idea of the transparent, unmediated realism would be absurd with the production so obviously revealing the technique used. Furthermore, the conslusive discussion of the spatio-temporal representation is mainly argumented through the sense of existentially meaningful presence affirming the film spectatorship according to the Sobchack’s film phenomenology.
My goal has, however, been to minimize the argumentation based on theoretical references. Film phenomenology has more offered me some of the key concepts through which I have "tightened" my analytical discussion. To much extent I rely on on my professional experience in the field of theatre, film and television design.
I also wish to recognize to what extent my own design work relies on my perceptual relation to the world I live in — as well as I am assuming my design work with the virtual set to be related to the experience of the traditional film. Compared with the traditional set, it is most immediate, that the virtual set confronts the designer (as well as the artistic as a whole) with the lack of the direct physical relationship with it: in a way or another, on a singular or a general level, intentionally or unintentionally, is responded.
During the preproduction and production phase all the documents of the design process including the film script and the storyboard by director Marikki Hakola, the digital design sketches, notes from the design meetings, digital testimages, digital test animations, videos of testshooting will be collected. Also the corrections made by the artistic team and comments made by the director throughout of the artistic process are of special interest. This data is in the layout of the professional practice. The key examples pointed out from the L’Enfant design are meant to illustrate the intention to provide the cues of the perceptual realism. The qualification what these "cues" are is based on the common functions of computer graphics meant to create the realistic effect. Such functions are for example the creation of form, material, light, the simulation of the camera viewpoint and lens. The material preconditions also include the specific application used for calculating the realtime or postrendered animation. I am assuming that during the design process there are different technique related variations to solve the design strategies. These solutions also mean shifting knowledge of the different versions to simulate the perceptive acts. However, the digital set for me always negotiate of the success of the impression: If there is a complicated lightning solution, there usually cannot be complicated texturing solutions and vice versa due to the facts of the graphics generating process. This makes the use of the realistic cues as a very conscious choice, which can be traced quite obviously even in a singular project.
The postproduction phase is carried through the personal evaluation of the work completed and still progressing through the work of the rest of the artistic team. This evaluation is related to the film phenomenology. Previously I have described the representation of the lived world physical phenomenona in the virtual set and the simulative act of viewing this world through a camera. Now I wish to extend my discussion. Gone through all the trouble in order to understand actually in quite an mechanical way what the imitative and the simulative realism stands in relation to the technique and the expression of the virtual set, I wish also to speculate about it. I am not quite sure, whether the film phenomenology is enough to explain the film spectatorship, however, it seems to parallel in a very sensible way the film making process from the viewpoint of the set designer. According to my understanding, the film and television set design process suggests a strong sense of embodied presence and vision. I am very aware, that the virtual set compared with the traditional one is even more complex in its way to represent time and space. However, I wish to at least make some suggestions, how the sense of "the meaningful presence" is established in the world and the view of the virtual set.
4. Researchers and resources.
4.1 Research environment, including the equipment to be used.
This research is utilizing primarily the virtual set studio in the LUME media centre at the University of Art and Design Helsinki (UIAH). The virtual set system in this studio is 2-D virtual set system with two Silicongraphics O2:s. In addition to this the virtual set system of the Video Department at the Royal University College of fine art is utilized as the co-operation of the two above mentioned universities. This virtual set system is 3-D virtual set system with Silicongraphics Onyx. There is also a NT workstation in the use at the Department of Design for Theater, Film and Television with the digital modeling, animation and compositing software. The universities also provide some support personnel in the use of their facilities.
4.2 Division of research and the funding
The KROMA Productions Ltd will carry a separate part of the development process associating with the specific application of the virtual set. In my research, these applications are treated as a specific functions of imitation and simulation.
The KROMA Productions Ltd will be a responsible producing party. The collaborators are (in addition to the academic parties) the Finnish Broadcasting Company (YLE), The Center for Advancement for Audiovisual Culture (AVEK), The Finnish Film Foundation (SES). Foreign production parties are currently searched for. The overall budget is comparable of the average budget of an hour-long film in Finland. The research team also practically equals the production team.
5. Results
5.1 Research results to be expected and their significance
It can be expected, that the research produces valid knowledge of the structures and pragmatics of the virtual set related to the realization process. L’Enfant project will produce an systematic example of one set design process in the layout it utilitizes the functions of the digital technique. This example can be directly applicable in the virtual set design practice yet relatively unknown. As a demo project enlisting the limitations as well as advantages of the virtual set, it can provide some model how to "solve" a virtual set of a narrative production. Also, it is a nature of the digital production that they also provide more developed applications of the technique possible to utilize by a certain production unity, here in the first place the KROMA Productions Ltd and the LUME media center.
Also, the project can offer a platform for a deeper discussion what is the "world" the virtual set presents, what does it refer to and how it is possible to explore this world cinematically. To some extent the discussion will analytize of the reason for the "anxiety" of realism so inherently present in the current digital film and television practice.
5.2 Publication the research results
L’Enfant will be a television production for the YLE TV 1. The master tape will be digital beta. The analytical part will be published as a printed thesis accompanying the cd and dvd.
Bachelard, Gaston. The Poetics of Space. Trans. Maria Jolas. Boston: Beacon Press, 1969.
Branigan, Edward. Point of View in the Cinema: A Theory of Narration and Subjectivity in Classical Film. New York, Mouton, 1984.
Carroll, Noel. Mystifying movies: Fads and Fallacies in Contemporary Film Theory. New York: Columbia University Press, 1988.
Casebier, Allan. Film and Phenomenology. Towards a Realistic Theory of Cinematic Representation. New York: Cambridge University of Press, 1991.
Ihde, Don. Experimental Phenomenolgy. An Introduction. New York: Paragon Books, 1979.
Ihde, Don. Technology and the Lifeworld: From Garden to Earth. Bloomington: Indiana University Press, 1990.
Manovich, Lev. The language of New media. (puuttuu julkaistiedot).
Merleau-Ponty, Maurice. Phenomenology of Perception. Trans. Colin Smith. London: Routledge and Kegan Paul, 1962.
Merleau-Ponty, Maurice. The Primacy of Perception. Ed. James.M. Edie. Evanston, IL: Northwestern Univ. Press, 1964.
Woolley, Benjamin. Virtual Worlds. Cornwall: Blackwell, 1993.
Baudry, Jean-Louis. "Ideological Effects of the Basic Cinematographic Appararatus." Trans. Alan Williams. Film Quarterly 28 (Winter 1974-75), pp.39-47.
Casebier, Allan "Representations of Reality and Reality of Representation in Contemporary Film Theory." Persistence of Vision (Spring 1987) pp.36-43.
Copjec, Joan,. "The Anxiety of Influencing Machine." October 23 (1983), pp.43-59.
Allen, Richard. "Representation, Illusion, and the Cinema." Cinema Journal (Winter 1993) pp.21-48.
Elsaesser, Thomas. "Digital Cinema: Delivery, Event, Time" in Cinema Futures: Cain, Abel or Cable? The Screen Arts in Digital Age, ed. Thomas Elsaesser and Kay
Hoffmann (Amsterdam University Press 1998), 201-222.
Manovich, Lev. "Assembling Reality: Myths of Computer Graphics" [online]. San Diego: USCD, accessed 11 February 2001. Available from World Wide Web from http://www.jupiter.ucsd.edu/~manovich/text/assembling.html
Sobchack, Vivian. "The Scene of the Screen: Envisioning Cinematic and Electronic Presence" [online]. New York: University of Rochester, accessed 11 February 2001. Available from World Wide Web from http://www.rochester.edu/College/FS/Publications/SobchackScene.html