Avatar-based language assistant for automated sign translation
Motivation
Are the approximately 70 million deaf people worldwide included in the rapid change of digital communication? Since text language is often a foreign language for them, the dynamic presentation of content in sign language enables digital participation. To meet the information needs of all target groups, digital accessibility is becoming increasingly important. With the help of artificial intelligence (AI) and digital avatars, automated tools support the realization of barrier-free communication.
Innovations and perspectives
In the joint project AVASAG (Avatar-based language assistant for automated sign translation), funded by the German Federal Ministry of Education and Research (BMBF), six partners from research and development are developing a novel sign animation method for 3D avatars under the leadership of Charamel (specialist for interactive avatar-based assistance systems). By combining machine learning methods with rule-based synthesis methods, texts are mapped into gestures. This is done under precise resolution of temporal and spatial dependencies of the sign elements for high quality results.
Goals and Procedure
The goal of the joint project AVASAG is the creation of a real-time controlled and AI-based 3D sign language avatar for the automatic translation of German texts into German Sign Language (DGS). The resulting realistic representation of the 3D sign language avatar is intended to enable digital and barrier-free communication as well as improved participation in the Digital Society, thus offering real added value to deaf people. To optimize interaction and communication on digital channels, companies can use AVASAG to generate their content automatically as well as dynamically and at the same time comply with legal regulations regarding digital accessibility. With the involvement of the sign language community, a sign language avatar will be implemented as a demonstrator for automated translation in the area of travel information and services with a focus on transport and tourism.
The project at a glance:
Project name: AVASAG (Avatar-based language assistant for automated sign translation)
Project volume: € 1.98 million (~74 % BMBF funding share)
Project duration: 01 May 2020 - 30 April 2023
Project partners:
- Charamel GmbH, Cologne - project management
- yomma GmbH - experts for Sign Language, Hamburg
- Ergosign GmbH - pioneer for user experience design, Saarbrücken
- German Research Center for Artificial Intelligence (DFKI) GmbH - research areas "Cognitive Assistance Systems", "Language Technology and Multilinguality"
- Cologne University of Applied Sciences / Institute for Media and Photo Technology, Cologne
- University of Augsburg / Human Centered Multimedia (HCM), Augsburg
BU: 3D Sign Language Avatar translates text into sign language.
Picture credits: Copyright Charamel
Further information:
Official project page: https://avasag.de/
Press release: Joint project AVASAG - together for digital barrier-free communication
BFSG and BITV: Meet updated digital accessibility legislation with sign avatar in the short term
Free webinar: New digital accessibility legislation
sponsored by
