One of the cornerstones of ERC Advanced Grant FilmColors has been the development of computer assisted methods and tools for the analysis of film colors in the emerging field of digital humanities.
For the computer-assisted qualitative analysis of film colors of a large corpus of approximately 400 films, the research team has elaborated a workflow that consists of a video annotation system and various databases, currently implemented in FileMaker.
Screenshot ELAN template used in FilmColors
In this workflow the team segmented and annotated the films with the software ELAN, developed by the Max Planck Institute for Psycholinguistics in Nijmegen, and then imported the corresponding data into an analysis database in which a highly detailed protocol of around 1.200 items was used to structure the analysis process to yield standardized results over the whole range of the corpus from 1895 to 1995. The concepts of the analysis database are defined in an illustrated glossary.
Screenshot of the glossary database developed in FilmColors, entry cold-warm contrast.
Beyond the analysis of hues and color contrasts of figures and backgrounds, the protocol required the analysis of locations and periods, narrative and semantic concepts, characters’ affective and emotional states, image composition, lighting, patterns and textures, and surfaces and properties of characters, objects and environment. The concept of faktura is employed to analyze the material properties of the films as well, while the last topic addresses movements of camera, characters and illumination. An additional keyword database contains motifs and themes that describe characteristic elements of the story. Filmographic data, including film title, year and country of production, directors, cinematographers, production and costume designers, and not least technical information, such as color process applied, camera, lenses, etc., is organized in a corpus database and connected to the analysis database. In the final evaluation the collected data give insights into the diachronic evolution of aesthetic patterns and their shifts in connection with technical foundations, historical, cultural and institutional contexts of the films’ production, styles of filmmakers and collaborators or production companies.
Part of the analysis team working on their calibrated 4K monitors in custom-made darkening booths at the Department of Film Studies at the University of Zurich.
The analysis team consists of the PhD students Olivia Kristina Stutz (early films from 1895 to 1930, applied colors and early mimetic colors), Michelle Beutler (mid-century from 1930 to 1955, Technicolor, early chromogenic stocks and various other additive and subtractive two to three color systems) and Joëlle Kost (more recent chromogenic stocks), postdoc researcher Dr. Bregt Lameris with an emphasis on affective dimensions of film colors, principal investigator Prof. Barbara Flueckiger, and the student assistants Manuel Joller (working on Michelle Beutler’s period), Ursina Früh (for Olivia Kristina Stutz) and Valentina Romero (for Joëlle Kost).
Since March 2017 the team has been collaborating with the Visualization and MultiMedia Lab of Prof. Renato Pajarola, also at the University of Zurich, Dr. Enrique Paredes, Dr. Rafael Ballester, Noyan Evirgen and Gaudenz Halter. It is the aim of this collaboration to adapt the methods and insights gathered during the execution of the aforementioned analyses into (semi-)automatic tools based on computer vision and deep learning.
At the core of this development is the new video annotation tool VIAN by Gaudenz Halter. VIAN is a visual annotation and analysis environment that tackles some of the issues identified in the application of ELAN. While ELAN is a highly sophisticated software for video annotation, it was not developed with aesthetic analyses of full feature films as a target of its application. By contrast, VIAN allows, in addition to temporal segmentation and annotation, a wide variety of interventions with the audio-visual data such as adding visual annotation layers, a sophisticated tool for the creation and management of screenshots, automated segmentation and screenshots, colorimetry and most importantly a range of (semi-)automatic tools for the analysis and visualization of film colors.
Figure–ground separation and corresponding color schemes as developed for the project by the Visualization and MultiMedia Lab, University of Zurich.
Some of early tools were developed outside of VIAN by Noyan Evirgen with support from VMML, Dr. Paredes and Dr. Ballester, namely a deep learning tool for figure–ground separation and extraction of the corresponding color schemes. The tool combines an object recognition software with a contour extraction tool to automatically cut out characters from their background. Color schemes are created with adaptive clustering to produce finely grained representations that match the visual impression of the spatial distribution of hues (see conference paper presented at the Digital Humanities conference in Montreal, 2017).
Foreground visualization
Background visualization of temporal development on the x-axis and saturation on the y-axis for one entire film: Jigokumon (JAP 1953, Teinosuke Kinugasa)
VIAN contains the full range of analytic concepts elaborated in the project, assisted by the illustrated glossary for users to train their understanding and perception of film color aesthetics and narrative functions. These concepts are intended to be organized in a modular way, so that each individual user can select groups of concepts and levels of complexity according to their personal requirements and interests. Furthermore, a large arsenal of visualization and analysis tools provide a variety of results that connect the analytical concepts to the visualizations. All these results have been connected to the Timeline of Historical Film Colors.
Current State of Development (2020)
For a more current update see our paper Flueckiger, Barbara; Halter, Gaudenz (2020): Methods and Advanced Tools for the Analysis of Film Colors in Digital Humanities. In: Digital Humanities Quarterly, 14,4, Special Issue Digital Humanities&Film Studies. Analyzing the Modalities of Moving Images,
(= http://digitalhumanities.org/dhq/vol/14/4/000500/000500.html).
Download and Coaching:
VIAN is available for free download. We recommend that you get in touch with us for support and free coaching.
Further Reading:
Blog post by Dr. Christian Gosvig Olesen: In September 2017 the research team at the University of Zurich organized a workshop with external experts to receive input and feedback. Pictured are Dr. Eric Hoyt, professor of Media Studies at the University of Wisconsin, and Dr. Everardo Reyes-García, Université Paris 8.
Flueckiger, Barbara (2011): Die Vermessung ästhetischer Erscheinungen. In: Zeitschrift für Medienwissenschaft, 5, pp. 44–60 (in German). Download PDF
Flueckiger, Barbara; Evirgen, Noyan; Paredes, Enrique G.; Ballester-Ripoll, Rafael; Pajarola, Renato (2017): Deep Learning Tools for Foreground-Aware Analysis of Film Colors. Conference paper, Computer Vision in Digital Humanities, Digital Humanities Conference, Montreal.
Flueckiger, Barbara (2017): A Digital Humanities Approach to Film Colors. In: The Moving Image, 17.2, pp. 71–93. Download on JSTOR | Free download of the preprint manuscript
Flueckiger, Barbara; Halter, Gaudenz (2018): Building a Crowd-sourcing Platform for the Analysis of Film Colors. In: The Moving Image, 18.2. Download on JSTOR
Flückiger, Barbara (2019): Avancierte Methoden der computer-gestützten ästhetischen Filmanalyse. In: Digital Humanities. Multimedial & multimodal. Konferenzband 6. Tagung des Verbands Digital Humanities im deutschsprachigen Raum. Frankfurt am Main, pp. 13–21 (in German). Download
Flückiger, Barbara (2020): Deep Learning in der Filmanalyse. In: Philosophische Fakultät. Digitale Lehre und Forschung,
(= https://dlf.uzh.ch/2020/07/14/deep-learning-in-der-filmanalyse/, retrieved 08/21/2020).
Flückiger, Barbara (2020): Digitale Werkzeuge zur ästhetischen Analyse von Filmfarben. In: Montage AV, 29,1. pp. 157–172,
(= https://www.zora.uzh.ch/id/eprint/189059/).
Flueckiger, Barbara; Halter, Gaudenz (2020): Methods and Advanced Tools for the Analysis of Film Colors in Digital Humanities. In: Digital Humanities Quarterly, 14,4, Special Issue Digital Humanities&Film Studies. Analyzing the Modalities of Moving Images, (= http://digitalhumanities.org/dhq/vol/14/4/000500/000500.html ).
Flueckiger, Barbara; Halter, Gaudenz (2020): Méthodes et outils numériques pour l’analyse avancée des couleurs des films. In: Everardo Reyes-García and Gwen Le Cor (eds.): Faire Image / What makes an image? Saint-Denis: Octaviana (Bibliothèque numérique Paris 8), pp. 63–130. (= https://www.zora.uzh.ch/id/eprint/199220/)
Halter, Gaudenz; Ballester-Ripoll, Rafael; Flueckiger, Barbara; Pajarola, Renato (2019): VIAN. A Visual Annotation Tool for Film Analysis. In: Computer Graphics Forum, 38,1,
(= https://onlinelibrary.wiley.com/doi/full/10.1111/cgf.13676).
Stutz, Olivia: (2016) Algorithmische Farbfilmästhetik. Historische sowie experimentell-digitale Notations- und Visualisierungssysteme des Farbfilms im Zeichen der Digital Humanities 2.0 und 3.0. Zürich (in German). Download PDF, 6388 KB
VIAN Documentation and Tutorials:
Documentation:
https://www.vian.app/static/manual/index.html
VIAN Video Tutorials:
vimeo.com/user/70756694/folder/1220854
Screenvideos VIAN WebApp: