Evaluate Possible Use of Avatars to Enhance Direct Communication Support for Deaf and Hard-of-Hearing Users

Avatars can potentially play an important role in offering a visual language alternative to displaying spoken and/or signed messages within educational settings or workplaces that include deaf, hard-of-hearing and hearing persons. The possibility exists of developing a signing avatar system that could be used in conjunction with a voice-to-text system to visually represent spoken messages between hearing and non-hearing persons. Although fully functional avatar representations of sign language based on voice-to-text systems are likely years away, we propose that the feature set of such a system could be investigated and modeled for initial comment and review by both experts and the general deaf community.
The fundamental question to be addressed in this strand is whether it is possible for signing avatars, generated from voice-to-text systems or other input processes, to be used on videophones or conferencing systems to provide communication support similar to that provided by live, professional interpreters.

Currently, faculty in the department of Computing and Information Sciences at RIT are investigating the potentials of this emerging technology though advanced research and development in Second Life applications and other imaging representation systems.
For this strand of research, a key developer will be identified to conduct a review of the current and near future state-of-the art of signing avatars related to telecommunications. Working with avatar and animation experts, this developer will pull together the key questions and possible answers in a whitepaper, which will then be disseminated to a small group of experts (6 persons) for review, to be followed by an intense discussion in a forum at RIT/NTID on the viability of avatar-technology for sign language to spoken language communication, and on setting the context for future developed of the technologies involved. Interpreters and real-time captionists will be present at the focus groups and forums to assure clear and comprehensive communication between all participants.

A professional evaluator and focus group specialist will attend this forum and document all meaningful discussion and suggestions at the focus group sessions and forums. All captioned files will be retained for analysis.

The concluding work products will include the whitepaper and an action plan of recommendations that will outline future research and development considerations.