Spoken Language and Multimodal Corpora

It is getting widely recognized that human behavior essentially integrates both verbal and non-verbal events. The knowledge of the synergy between spoken utterances and gestures is a necessary prerequisite for a wide range of practical uses such as in building better social interactions in general, more successfully handling multilingual communication, improving the quality of life of the disabled, but it is also needed for the implementation of seemingly distant applications like security installments or human-machine interaction.

This two-week course offers the basic elements in the study of such verbal and non-verbal aspects of human behavior that, further enhanced, can be used in any of the above fields and beyond. For this purpose it will cover a comprehensive spectrum of technological and methodological aspects regarding the creation and exploitation of speech corpora. The content will therefore range from signal processing technology via transcription and annotation methods to issues of multimodal analysis.

In an initial part of the workshop, basic audio and video recording techniques, (meta-)data formats and ethical and legal aspects of field recordings will be outlined and then applied as the participants record sample data according individual research interests.

Based on these recordings, techniques and methods for the transcription and annotation of spoken data as well as domain-specific annotation schemes will be discussed. Furthermore, an in-depth introduction to analysis methods and tools will be given.