oa Assistive technology for improved literacy among the deaf and hard of hearing

We describe an assistive technology for improved literacy among the Deaf and Hard of Hearing, that is cost-wise and accessible to deaf individuals and their families/service providers (e.g., educators), businesses which employ them or list them as customers and healthcare professionals. The technology functions as (1) A real-time translation system between Moroccan Sign Language (a visual-gestural language) and standard written Arabic. Moroccan Sign Language (MSL) is a visual/gestural language that is distinct from spoken Moroccan Arabic and Modern Standrad Arabic (SA) and has no text representation. In this context, we describe some challenges in SA-to-MSL machine translation. In Arabic, word structure is not built linearly as is the case in concatenative morphological systems, which results in a large space of of morphological variation. The language has a large degree of ambiguity in word senses, and further ambiguity attributable to a writing system that omits diacritics. (e.g. short vowels, consonant doubling, inflection marks). The lack of diacritics coupled with word order flexibility are causes of ambiguity in the syntactic structure of Arabic. The problem is compounded when translating into a visual/gestural language that has far fewer signs than words of the source language. In this presentation, we show how Natural language processing tools are integrated into the system, the architecture of the system and provide a demo of several input examples with different levels of complexity. Our Moroccan Sign Language database has currently 2000 Graphic signs and their corresponding video clips. The database extension is an ongoing process task that is done in collaboration with MSL interpreters, deaf signers and educators in Deaf schools in different regions in Morocco. (2) Instructional tool: Deaf school children, in general, have poor reading skills. It is easier for them to understand text represented in sign language than in print. Several works have demonstrated that a combination of sign language and spoken/written language can significantly improve literacy and comprehension (Singleton, Supalla, Litchfield, & Schley, 1998; Prinz, Kuntz, & Strong, 1997; Ramsey & Padden, 1998). While many assistive technologies have been created for the blind, such as hand-held scanners and screen readers, there are only a few products targeting poor readers who are deaf. An example of such technology is the iCommunicator™ which translates in real time: speech to text, speech/typed text to videos of signs, and speech/typed text to computer generated voice. This tool, however, does not generate text from scans and display them with sign graphic supports that a teacher can print, edit, and use to support reading. It also does not capture screen text. We show a set of tools aiming at improving literacy among the Deaf and Hard-of-Hearing. Our tools offer a variety of input and ouput options, including scanning, screen text transfer, sign graphics and video clips. The technology we have developed is useful to teachers, educators, Health care professionals, speech/language pathologists, etc. who have a need to support understanding of Arabic text with Moroccan Sign Language signs for purposes of literacy improvement, curriculum enhancement, or communication in emergency situations.