5th International Conference on Computer Helping People with Special Needs Johannes Kepler University, Linz 1996
Giuliano Pirelli
Istituto per i Sistemi, l'Informatica e la Sicurezza
voice@jrc.it
--> http://voice.jrc.it
Abstract: the paper presents an overview of the Joint Research
Centre activity related to computer-based applications for helping the
disabled and the elderly people, as well as for the development of hardware
or software aids, intended for people with or without special needs.
. The Joint Research Centre of the European Commission
The Joint Research Centre (JRC) is the corporate research laboratory of the European Union (EU). The mission of the JRC is to promote and carry out customer-driven research in support of Community policies. The provision of impartial and independent advice from the JRC is an integral part of the Community research and technological development system, considering ùits competencies, unique research installations, neutral and impartial status. The installations and abilities developed over the years include the study of supranational scientific problems, such as environmental protection, plant safety, technical standards, computer and information science on the European scale and energy resources.
A staff of about 1500 officials performs the work through 30 research programme themes currently, often in collaboration with research in the Member States or other states associated with the Framework Programmes. The JRC is already involved in collaboration with industry, notably through its participation in selected EUREKA projects and through execution since 1988 of work for third parties, within which industry represents 60% of the customer basis. The JRC organises workshops for scientific and technical workers in advanced sectors of science. The workshops are linked with Commission research and development programmes, especially those in which the Institutes of the JRC are involved.
1.1. The JRC Institute for Systems, Informatics and Safety (ISIS)
The mission of the JRC Institute for Systems, Informatics and Safety (ISIS) is to be the impartial centre of expertise of the EU in the science and technology of safety management, the multi-disciplinary analysis of industrial, socio-technical and environmental systems and the innovative application of information technology. With a staff of about 280 officials, ISIS develops and applies its expertise and unique test facilities in many fields, including: transport safety, seismic protection of buildings, natural resource and environmental management, preservation of architectural work.
A new activity will be the development of multimedia educational and training software including user-friendliness of the systems and the execution of demonstration projects based on advanced multimedia techniques (in training of aircraft pilots, space education, robotics technologies and multi-cultural education). In addition ISIS will investigate new applications in the care for elderly and disabled persons.
1.2. The ISIS Unit for Software Technologies and Automation (STA)
The activity areas of the ISIS Unit for Software Technologies and Automation (STA) include dependable software applications (safety critical computing systems, requirements engineering), multimedia network applications (animation in medical imaging, multimedia techniques in training and education), sensor based applications (surveillance techniques, 3D reconstruction of real environments, learning approaches for control), robotics and remote handling.
2. EU Projects and JRC Exploratory Research Programme
Information technology may help in the development of hardware or software aids for people with special needs, due to a physical or sensorial disability. Not only is the potential market large, but, by paying attention to special requirements, it is also possible to improve the quality of the products for all users. Nevertheless, problems still exist for developing new assistive devices and for converting of good prototypes into commercial products.
The EU and COST (European CO-operation in the field of Scientific and Technical research) have undertaken different projects in this field. In particular, TIDE (Technology Initiative for Disabled and Elderly people) is a Community pre-competitive technology research and development Initiative aimed at stimulating the creation of a Single Market in Assistive Technology in Europe.
HANDYNET, a major activity of the HELIOS programme (Handicapped people in the European community Living Independently in an Open Society), is a computerised information and documentation system on technical aids and the statutory conditions attached to the assistance.
The Educational Software Multimedia Task Force is encouraging research and industry to do more to meet the educational needs, since in the Information Society every citizen will need to continue learning throughout his lifetime. Multimedia can improve the effectiveness of education and training activities at home, at school, at work and address special needs, as those of disabled people, of children at risk of dropping out of the school system and of small and medium-sized enterprises.
ISIS decided to consider the interface between Life Science and information technology to provide help for the disabled and the elderly in its Exploratory Research Programme for 1996. In particular, two projects are carried out by the STA: Information technology aids for people with special needs - voice to text recognition, and Brain-actuated control - using EEG pattern recognition to help the disabled. The exploratory work will provide a better definition, from a technical point of view, of the requirements of people with special needs and allow a more collaborative work between technicians and non technician, paying special attention to the different aspects of the interdisciplinary activities.
2.1. Information technology aids for people with special needs: voice to text recognition
The computer is very useful for the hearing impaired, being used for development of communication of deaf children in rehabilitation sessions or pedagogic games, as well as for communication with other deaf or normal-hearing people by text-telephones or electronic mail. A widely used application is the subtitling of television transmissions, very powerful help for deaf people, particularly for the language learning and training for deaf children.
In the last years significant developments have been made in voice to text recognition systems, used both for direct dictation of texts to word processing systems and for the control of computer programs by doctors or researchers using both hands for other tasks. These systems are under test for wheelchairs control for the motor disabled and for helping deaf people to see on a computer screen the voice of a teacher in the same room or of a speaker at the other end of a telephone line.
An objective of the STA Unit's project is the set-up of a laboratory prototype of a computer-driven automatic answering telephone, converting voice to text, intended for the use of deaf people. The same system could be of use for television broadcasters for live transmissions subtitling, based on a human interpreter, in one language or for multilingual subtitled European television news. The subtitling facilities could be of prime importance in education and tele-education programmes, particularly important for the deaf.
The research will include: testing of voice to text converter systems, with signals arriving from a telephone line; studying of subtitles system of television broadcasters; analysing the difficulties in speech, the lack of communication and the slight differences between special and normal needs in information technology applications. Even some limited results could be of immediate use for the Associations and Organisations involved in the research.
2.2. Brain-actuated control: using EEG pattern recognition to help the disabled
Information technologies have the potential to help people to fit better in the information society, only when users can easily interact with computer-based systems.
Quadriplegics (physically disabled people who have no control over their motor responses), unable to use keyboard, mouse, or joystick, cannot benefit from recent advances in robotics technologies such as motorised wheelchairs and manipulators, which could highly improve their quality of life.
Since a friendly, easy use of computer-based systems by the disabled depends on the development of adaptive human-systems interfaces (AHSI), the STA Unit is investigating the use of brain signals as input to the AHSI. In particular, if several patterns can be reliably distinguished in electro-encephalogram (EEG) signals, then people can control robotics devices by composing sequences of these patterns. The approach consists in developing an adaptative brain interface. Briefly, the robot has a set of pre-learned basic behaviours and the interface has to learn to associate the desired behaviours with individual EEG patterns, by the use of neural networks techniques.
In the short term the objective is the development of an AHSI to control a mobile platform and a manipulator of the STA laboratory and to transfer the experience gained to disabled-oriented robotics devices (wheelchairs). An intermediate goal is the development of an AHSI to select different functionalities from a computer screen, this helping disabled people to use computers to manage their homes. The long term objective is the development of AHSIs based on EEG pattern recognition to allow disabled people to communicate with computer-based systems and so to participate fully in our information society.