Signifying codes of audiovisual products: Implications in subtitling for the D/deaf and the hard of hearing

Abstract

Audiovisual products are complex multimodal constructs that produce meaning through the interaction of all sign systems delivered both through the acoustic and the visual channel, either verbally or non-verbally (Delabastita 1990; Chaume 2004; Gambier 2013). Due to the multimodal complex nature of audiovisual products, when creating subtitles for the D/deaf and the hard of hearing (SDH), the audiovisual translator faces the responsibility to be aware of the existence and understand the interaction of signifying codes of the visual and acoustic channels to create subtitles that are relevant to the target audience. Little has been said about signifying codes and their implications on accessible audiovisual translation (AVT) (cf. Tamayo and Chaume 2016), hence the present article seeks to fill the gap and suggests an interdisciplinary approach to the study and practice of SDH that takes Film Studies and Translation Studies into account. It focuses on how signifying codes of audiovisual texts might affect subtitling decisions, taking into consideration the needs of the D/deaf and hard of hearing (DHH) audiences (for example, the need of making explicit in the subtitles some sound elements or the need for an adequate subtitling speed) as well as technical aspects and formal restrictions (Martí Ferriol 2010) of this AVT mode. Since the target audience of SDH has limited or no access to sound, special attention is dedicated to the implications of signifying codes in the acoustic channel (linguistic code, paralinguistic code, special effects code, musical code and sound arrangement code) and how their meanings and interactions with other codes can be conveyed in the form of subtitles for the DHH audiences. Although the approach in this article is mainly theoretical, the possible subtitling solutions are illustrated with real examples or possibilities mentioned in previous research and publications that include some creative subtitling options.