Anders Bonde : Emergent Forms in Mediated Audio-Visual Texts
In recent years, the production and experience of meaning in audio-visual media texts has been a subject of growing attention in the humanities and its various disciplines, such as media studies, information and communication studies, psychology, literature and cultural studies as well as musicology. The concept “audio-visual media texts” covers all kinds of genres, combining images (moving pictures or stills) with sound (e.g. speech, music, ambient sound, and sound effects), and examples are TV programs, movies, documentaries, commercials, music videos, computer games, websites, and digital storytelling. The interdisciplinary attention is mainly focused on meaning as a phenomenon of interaction and synthesis which partly appears on the level of perception through the interaction of visual and auditory modalities (verbal as well as non-verbal), and partly on a cognitive level through the sense impressions on the one hand and experienced general and specific knowledge on the other (Langkjær 2006: 20-21). Furthermore, it appears to be generally accepted that meaning comes about as an emergent phenomenon (i.e. a kind of Gestalt quality), in which the synthesized whole is considered either more than the sum of the parts or different than the sum of the parts (cf. Cook 1998: Hull and Nelson 2005). However, the interesting question is how audio-visual emergence actually might be constructed, mediated and experienced – or to more precise: Is it possible to define the necessary and sufficient conditions for generating emergent phenomena of meaning in mediated audio-visual texts? Such a question will be the central problem and issue in this project.
The empirical material is a selection of Danish TV commercials since 1999, all promoting financial services such as loans, savings and pension solutions, and insurance. This choice is foremost due to the genre itself, but also to the particular category of advertising products. First of all, the pragmatic and “parasitic” nature of TV advertising, coupled with its highly concentrated formula, seems to be a perfect field for exploring audio-visual emergence. Given the very short amount of time (usually 20-40 seconds) to communicate the message, the genre is generally located in a tensional area between the trivialized (or well-known) and the new (or attention-attracting), as it has to be surprising, convincing and accommodating at once. In such a perspective it is certainly not hard to believe that music often has a key function; despite its attention-attracting potential, it consists of a series of semiotic codes which can be used in the “economy of expression” which defines the genre. Through its gestural content and potential referentiality (the latter being a socio-cultural matter), the music contributes to the shaping of the meaning of the visual narrative which, on the contrary, also shapes the meaning of the music (Cook 1998: 8). Secondly, in the last 10 years, ordinary financial serviceshave been promoted ever more as lifestyle products by associating “a series of values to money that modern people require to form their personal identity” (Bak and Nørgaard 2006: 7). This has been done through the growing aestheticization of the cinematic instruments, and not least through calculated musical choices through which the addresser seeks to create a sense of solidarity with the addressee (Stigel 1998: 25). Another side of the same coin is to let the audience infer the meanings themselves as a result of well-planned discrepancies, uncertainties or ambiguities in the audio-visual representation (ibid.: 23).
Investigating audio-visual emergence is a rather complicated matter which requires more than a single analytical approach. At the outset, I will undertake a structural-hermeneutic perspective since my intention is to map the structural organization of the various modalities and to interpret the conditioned possibilities of meaning realization through their interaction. Then I will carry out a series of reception tests using a quantitative statistical method. The two approaches will be put into perspective through comparison to recent research, and it is my intention to present a new theoretical model for classifying the multiple kinds of audio-visual emergence.
References
Bak, Annette Cecilie and Nørgaard, Jens Lautrup (2006) ‘Strategisk semiotik: Overvejelser om iværksættelse af kommunikation og valg af tegn i en professionel kontekst’ [‘Strategical Semiotics. Considerations on the Implementation of Communication and the Selection of Signs in a Professional Context’]. Copenhagen Business School, Working Paper 9: http://ir.lib.cbs.dk/paper/ISBN/x656517300 .
Cook, Nicholas (1998) Analysing Musical Multimedia . Oxford: Oxford University Press.
Hull, Glynda A. and Nelson, Mark Evan (2005) ‘Locating the Semiotic Power of Multimodality’, Written Communication 22 (2): 224-261.
Langkjær, Birger (2006) ‘Mediernes lyd – En multimodal analysemetode’ [‘The Sound of the Media. A Multimodal Method of Analysis’], MedieKultur 40: 14-26.
Stigel, Jørgen (1998) ‘Tv og tv-reklamens udtryksformer’ [Television and the Mode of Expression in TV Advertising’], Dansk noter 2: 16-27.
