Title: Optimised preprocessing for automatic mouth gesture classification
Language: English
Authors: Brumm, Maren 
Grigat, Rolf-Rainer 
Keywords: Sign Language Recognition/Generation;Machine Translation;SpeechToSpeech Translation;Statistical and Machine Learning Methods
Issue Date: 2020
Publisher: European Language Resources Association (ELRA)
Source: Proceedings of the 9th Workshop on the Representation and Processing of Sign Languages: 27–32 (2020)
Abstract (english): 
Mouth gestures are facial expressions in sign language, that do not refer to lip patterns of a spoken language. Research on this topic has been limited so far. The aim of this work is to automatically classify mouth gestures from video material by training a neural network. This could render time-consuming manual annotation unnecessary and help advance the field of automatic sign language translation. However, it is a challenging task due to the little data available as training material and the similarity of different mouth gesture classes. In this paper we focus on the preprocessing of the data, such as finding the area of the face important for mouth gesture recognition. Furthermore we analyse the duration of mouth gestures and determine the optimal length of video clips for classification.
Our experiments show, that this can improve the classification results significantly and helps to reach a near human accuracy.
Conference: 12th International Conference on Language Resources and Evaluation, LREC 
URI: http://hdl.handle.net/11420/9735
ISBN: 979-10-95546-37-5
Institute: Bildverarbeitungssysteme E-2 
Document Type: Chapter/Article (Proceedings)
More Funding information: This publication has been produced in the context of the joint research funding of the German Federal Government and Federal States in the Academies’ Programme, with funding from the Federal Ministry of Education and Research and the Free and Hanseatic City of Hamburg. The Academies’ Programme is coordinated by the Union of the German Academies of Sciences and Humanities.
Appears in Collections:Publications without fulltext

Show full item record

Page view(s)

checked on Jun 19, 2021

Google ScholarTM


Add Files to Item

Note about this record


Items in TORE are protected by copyright, with all rights reserved, unless otherwise indicated.