Dipartimento Salute Mentale
Unità Operativa di Neuropsichiatria dell'Infanzia e dell'Adolescenza - Pavia - responsabile dr Emilio Brunati
(Decreto cost. 70639 del 22-12-1997) Partita IVA e Cod. Fiscale N° 01748780184
Sede di Pavia, Viale Indipendenza 3, 27100 Pavia
tel. 0382 431808 fax 0382 431374
Voice Project activities: Status report
The Neuropsychiatry Unit of ASL in Pavia has always followed
with interest the progress of computer science and more specifically of speech
recognition, as a potential aid for hearing impaired subjects.
When informed of the research undertaken by JRC in this field, we followed
their activities and collaborated in the tests. More particularly, we organised
in our conference room two workshops in March 1999: the first of two days
addressed to the schoolteachers and the second of one day addressed to language
therapists, for a total of approximately 250 participants.
We then organised some tests - described hereafter - with some of these schoolteachers
and language therapists, on the use of the Voice prototype for the integration
of hearing impaired pupils in the normal class as well as in rehabilitation
sessions. We also opened an experimental information desk for people with
disability, where the Voice prototype allows subtitling the information given
to clients with hearing impairment or other disability.
Communication needs of hearing impaired children
In developing projects to support deaf subjects' autonomy,
particular importance has always been given to the possibility of accessing
verbal communication without the intervention of interpreters (sign language),
mostly in those situations where lip-reading is compromised by the distance
or by the absence of the speaker.
In planned situations, it was already possible to adopt subtitles: films,
documentaries, videos of conferences, etc. But this is not the 'normal' situation
a deaf subject has to face everyday, especially, at school. Even when lip-reading
allows the student to follow the entire lesson, the required level of attention,
in addition to proving extremely tiresome, does not allow to take notes and,
therefore, to develop all of those review and reflection activities following
an explanation.
Voice prototype's potentialities
We believed the Voice project to be an opportunity to make
everyday life easier to deaf students in schools.
In our opinion, the strengths were:
The analysis of the product used within the Voice project and the first tests, highlighted a number of limits:
Despite these drawbacks, we saw the need to use this technology
to pave the way for a new development in a reality, which appeared to be based
on two non-communicating approaches: lip reading and sign language.
In addition, we wanted to assess the reaction of a deaf child to the early
adoption of this technology (children were aged between four and six).
Therefore, we undertook two different approaches: the first
one involved four deaf students already attending high schools in our province;
the second one focused on five deaf children aged four, five and six. All
subjects suffered from precocious or perinatal neuro-sensory almost total
deafness. They wore a hearing aid, which worked merely as an environmental
control device, and provided no support to verbal decoding.
Also, none of the subjects had reached a sufficient degree of oral expression,
even if their communication level was fairly intelligible. None of the subjects
showed intellective deficits, other than a minimum cultural gap due to learning
complexities. Only one subject showed an intellective deficit, which made
him, in our opinion, inappropriate for the attended school. In this case,
instead, the family believed that the student could still follow his educational
path, even if at a lower pace.
A difficulty encountered, at the beginning of the project, by students already
attending school, was the change of almost all of the support teachers who
had been trained to use the software. This change was due to school organisation
and regional administrative problems, not related to that specific class.
The new teachers were not trained and, in one case, not interested in using
this tool.
No difficulty was highlighted in the five young children,
even if in this case the complexity, for them, was to realise that the text
displayed below the image was linked to lip movements. Generally, hearing
children associate oral speech and written text by developing a number of
hypotheses about the structure of writing. These hypotheses are crucial for
the development of appropriate learning skills.
A deaf child is deprived of this experience because, by using the visual channel
alone, he cannot follow both written text and lip movements, at the same time.
(Hearing children, instead, link voices and texts by using both the visual
and the hearing channels, at the same time). In this sense, the apparent system's
limit of presenting the written text a few moments after that it has been
pronounced, may become a positive aspect, even if requesting the speaker's
particular attention in controlling his fall of voice.
The Voice system was used to create some didactic units. Unfortunately, these didactic units were used individually with deaf students and not within a lesson in a classroom. Teachers' difficulties in organising the activity in the classroom were the underlying reasons provided for this choice.
The required computer workstation was installed in a therapy
room. Children were presented in various sessions with illustrated stories
created for test purposes. The text was partly pre-defined and partly invented
during the session, and generated, in this way, a variety of presentations
to children, well even maintaining some fixed elements (characters and names,
images of the situations, etc.).
The texts generated by the system were corrected and used as a paper support
for the traditional pre-schooling activities. Mutual benefit immediately looked
as extremely promising and accelerated the arising of those cognitive behaviours,
which are called pre-requisites.
The current level of voice recognition technologies and teachers' lack of familiarity with the new technologies, make the system poorly effective. Nevertheless it provided encouraging results and proves to be, with a better level of computer literacy in schools, a reliable tool for the future.
Despite the difficulties experienced in linking the text
displayed on the screen to the 'voice' of the therapist, reasonable pre-alphabetisation
achievements were recorded in children. In particular, stories with images
and subtitles led to an increase of the level of attention towards the narration
and its events. When confronted with an image, these children can normally
only name its single elements, without perceiving either the temporal elements
or the actions inherent to the image itself. After a test period, these children
showed a stronger attention towards the elements of the story, in addition
to the expected vocabulary enrichment.
The improvement of voice recognition and control techniques will lead to the
definition of new pedagogic methods to be adopted with these children.
The tests have been very interesting and stimulating and showed significant potentialities of speech recognition techniques. More particularly, the Voice prototype demonstrated to take care, in an effective way, of the specific needs for the use with hearing impaired children, while standard commercial speech recognition systems hardly can face such situation. The results achieved by the specific tests, even within their time and applications limits, are very significant and encourage continuing in this direction.
February 2001
Dionigi Ioghà
Neuropsichiatra Infantile ASL Pavia
