E-accessibility to educational content for the deaf
Dr. Eleni Efthimiou [eleni_e@ilsp.gr]
and Dr. Stavroula (Evita) Fotinea [evita@ilsp.gr]
Institute for Language and Speech Processing, Dept. of Educational Technology.
Artemidos 6 & Epidavrou, GR 15125, Athens, Greece.
Dr. Galini Sapountzaki [gsapountz@uth.gr]
University of Thessaly, Dept. of Special Education, Greek Sign Language Lab.
Argonauton & Filellinon, GR 38221, Volos, Greece.
Abstracts
English
This paper focuses on the implementation of an e-learning environment for the educational support of the deaf, as a paradigm of deaf inclusion in the Information Society. The developed platform allows structuring and presentation of Greek Sign Language (GSL) educational material and linguistic resources, addressing the needs of GSL grammar teaching to early primary school pupils. Linguistic content presentation is realized by applying dynamic sign synthesis through the use of avatar technologies for sign representation. The platform utilises standard techniques for the presentation of educational material in deaf classes but also allows tutors to modify educational material dynamically. Acceptance by students and tutors has been tested under the supervision of the Hellenic Federation of the Deaf.
Greek
Η εργασία αυτή επικεντρώνεται στην υλοποίηση ενός περιβάλλοντος ηλεκτρονικής μάθησης για την υποστήριξη της εκπαίδευσης των κωφών μαθητών. Η εκπαιδευτική πλατφόρμα που παρουσιάζεται χρησιμοποιείται ως παράδειγμα για την ανάπτυξη εργαλείων ενσωμάτωσης των κωφών στην Κοινωνία της Πληροφορίας. Η συγκεκριμένη εκπαιδευτική πλατφόρμα επιτρέπει την δόμηση και παρουσίαση εκπαιδευτικού περιεχομένου και γλωσσικών πόρων της Ελληνικής Νοηματικής Γλώσσας (ΕΝΓ) και καλύπτει τις εκπαιδευτικές ανάγκες διδασκαλίας του μαθήματος γλώσσας στις πρώτες τάξεις του δημοτικού σχολείου. Η παρουσίαση του εκπαιδευτικού περιεχομένου στο περιβάλλον γίνεται με εφαρμογή διαδικασιών δυναμικής σύνθεσης νοημάτων που κάνουν χρήση τεχνολογιών εικονικού νοηματισμού. Το περιβάλλον εκμεταλλεύεται τυπικές τεχνικές παρουσίασης εκπαιδευτικού υλικού σε τάξεις κωφών μαθητών και προβλέπει δυναμική διαμόρφωση του παρεχόμενου εκπαιδευτικού υλικού. Ο βαθμός αποδοχής από τους χρήστες (μαθητές και δασκάλους) του προτεινόμενου πειραματικού πρωτοτύπου ελέγχθηκε με επίβλεψη της Ομοσπονδίας Κωφών Ελλάδος (ΟΜΚΕ).
Keywords
Deaf accessibility, sign synthesis, signing avatar, Greek Sign Language, educational scenaria, virtual tutor
List of Topics
1. Introduction
4. An educational platform for GSL
5. Grammar content of the GSL platform
5.1 Morpho-phonological components of SL grammar
5.2 GSL platform content organisation
6. Structuring of educational scenaria
6.2 Educational unit structure and content
6.3 Language use – thematic units
6.4 Visual non-linguistic means - visual mode of instructions
6.5 Visual means of transmitting information
6.6 Organisation of resources for scenaria content generation
9. Concluding Remarks and Future Development
1. Introduction
Sign Languages (SLs) are the natural visual languages used by Deaf Communities. They use three-dimensional space to articulate linguistic utterances. The fundamental semantic unit of a sign language is the sign, a symbol that exploits a gesture or specific movement as its core compositional element, and represents a concept, functioning much like word morphemes of spoken languages. Each sign is described as to its composition, which includes both the manual sign features of hand-shape, location, movement, orientation and number of hands, and all obligatory non-manual elements for its formation, which include mouth patterns, head and shoulder movement, facial expression and eye-gaze (Stokoe & Kuschel, 1978).
SL systems are heavily based on iconicity to convey a meaning. Within sign language systems, sentences are uttered in three-dimensional space, where syntactic-semantic relations are expressed via a morpho-syntactic structuring, organized differently to spoken languages. In this respect, SLs are autonomous systems, which do not exhibit direct shared properties and characteristics with orally articulated languages, although some elements of language interference can be found in data of (mostly) young informants.
A base sign is the basic grammatical unit of an SL and is composed of the following main manual components (Tennant & Gluszak Brown, 1998):
The hand-shape: the most apparent characteristic of a sign is the shape of the hand at the starting point and while signing
The palm orientation: the direction to which the palm turns during signing adds meaning to the formed concept
The location of signing: there are only certain areas on or near the body where signing is allowed
The movement: a considerable amount of the meaning transmitted in a sign is expressed through movement. The movement may convey information about the agent or the recipient of an action, or even the semantic class of the object involved in a given action. Repetition of movement may further declare frequency, plurality or grammatical category differentiation, like verb – noun. The expansion of the movement may indicate size or volume, whereas speed or vigour, in combination with the appropriate non-manual signs, may express a range of adverbial properties.
The non-manual elements of sign formation involve body position and facial expression that may function as indicators of thematic roles, syntactic agreement or adverbial qualities. They may also add semantic information, i.e. approval, exclamation, rejection, etc.
Traditionally, SLs use video to convey linguistic content. Video is, however, a static mechanism for the representation of linguistic utterance, as opposed to the dynamic nature of human language in real use situations. Participation in Information Society is based on the vast production of electronic linguistic content as well as unrestricted accessibility to available resources. Inclusion of SLs into information systems is restricted by the means of conveying linguistic content. To support dynamic SL content, a technological solution towards sign production has been explored over the last decade, which is related to the development of systems for sign synthesis, where signing is performed by a signing avatar (Virtual Reality Modelling Language-VRML model) exploiting Natural Language Processing (NLP) output.
2. Greek Sign Language
Greek Sign Language (GSL) is the natural language used by the members of the Greek Deaf Community, with several thousands of native or non-native signers and, respectively, it also constitutes the primary means of education for deaf students. GSL native users are estimated at about 40,600 (1986 survey of Gallaudet Univ.), whereas there is also a large number of hearing non-native signers of GSL, mainly students of GSL and families of deaf people. Although the exact number of hearing students of GSL in Greece is unknown, records of the Hellenic Federation of the Deaf (HFD) show that in 2003, about 300 people were registered for GSL classes as a second language. Moreover, a recent increase of mainstreamed deaf students in education, as well as a population of deaf students scattered in other institutions, small town units for the deaf and private tuition, may well double the total number of secondary and potential sign language users. Official settings where GSL is being used include 11 Deaf clubs in Greek urban centres and a total of 14 Deaf primary, secondary and tertiary educational settings.
In 2000, GSL was recognised by act of the Greek Parliament as the official language of the Greek deaf population (Legislative Act 2817/2000). The act implies that education curricula -as released by the Hellenic Pedagogical Institute of the Ministry of Education ( the official organisation that validates all educational programs of primary and secondary education in Greece )- contain a requirement that all educational material must be accessible to deaf students through the use of GSL. However, appropriate GSL educational resources have proved to be extremely limited, partly due to technical limitations regarding dynamic representation of linguistic content. It is no coincidence that the systematic analysis of language has been closely connected to the maturing of technologies, which enabled incorporation of three-dimensional representation in multimedia environments.
Even though electronic resources for educational purposes have been systematically developed since 1999 and serious attempts have been made in the field of lexicography in the recent past (see Efthimiou et al., 2004b, for a reference to: PROKLISI, a Dictionary of Computing Signs, NOEMA: a Multimedia Dictionary of GSL Basic Vocabulary and A Children's Dictionary of GSL), a complete decoding of language structure is not yet readily available.
On the other hand, a positive aspect of universal inclusion in the educational process is that, in accordance with EU guidelines for accessibility to information in special education (see also COM(2000) 284 final), all Greek schools have been equipped with unrestricted Internet access. This equipment allows an e-learning platform for GSL to be readily applicable to a real life school routine, because the demand for equal access to e-content, combined with the geographical spread of deaf students, underline the need for the introduction of e-learning practises to enable their unrestricted access to education.
3. E-learning for All
When the accessibility issue first emerged, it was mainly interpreted as a parameter, the fulfilment of which involved modification of already developed systems to meet the requirements that would allow their use by groups with specific disabilities (Story, 1998; Vanderheiden, 1998). Under current circumstances of computer use accessibility issues reappeared, to address the fact that significant user groups including the disabled and the elderly, should not be excluded from accessing computing devices. Under this perspective, Universal Access becomes a critical quality target in the context of the Information Society. The requirement for the development of Information Society Technology products and services accessible to all citizens, involves architectures based on the systematic effort to implement methodologies, design principles and tools in a proactive way (instead of a posterior adaptations). Such a priori approaches define the concept of Design for All in the era of Information Society. Universal Access has indeed become a central issue to Information Society technologies. In this sense, conventional computer meditated human activities (Pieper, 2001) as well as currently emerging services and applications such as access to on-line information, e-communication, digital libraries, e-learning, on-line communities, telework, etc (Stephanidis, 2001) need to be readapted to become accessible by all possible users. Therefore Universal Access is becoming predominantly an aspect of design. Accordingly, e-learning system architecture is directly connected with the important issue of accommodating the needs of all possible users when designing a system.
Currently, a most common type of e-learning application involves networking of schools (or individuals) located at a distance from urban centres. These types of networks make use of technologies which crucially depend on tutors' ability to update educational content and the students' potential to show and discuss their work in virtual class sessions, the design of which may exploit video conferencing and Whiteboard options to support interaction (Efthimiou & Fotinea, 2004). These tools constitute standard facilities offered by state-of-the-art Internet technologies, to be exploited in current e-learning platforms. But they also signal the dominant tendencies in educational practice within the framework of the Information Society, as it is gradually being established. Given that most groups of people with disabilities experience a similar isolation in respect to mainstream schooling, which in a number of cases may also be combined with geographical isolation, networking may well prove to be key to universal access to education, if system architectures and content presentation of educational tools integrate multimodal access devices, following the principles of multisensory learning (Hauert, 1990; Gaddes & Edgell, 1994).
Dynamic sign synthesis is currently becoming essential for unrestricted signing capacity of any application in the IT domain. Hence, in order to support access to e-content by the deaf in the environment of e-education, we have developed an educational platform that integrates avatar technologies and GSL electronic linguistic resources. The specific platform provides a test bed for evaluation of sign synthesis in the environment of e-learning for the deaf, allowing teachers to construct and present signing linguistic educational material according to lecture needs. Also it presents potential for the inclusion of the deaf in Information Society.
4. An educational platform for GSL
Design and development of educational content for teaching GSL to early primary school deaf pupils, has resulted in a platform that uses a signing avatar to present educational objects (Efhimiou et al., 2004a; Karpouzis et al., 2005), and is currently a unique tool for providing on-line educational content in GSL. The platform employs standard techniques used for educational material presentation in deaf classes, but also allows tutors the option of dynamically modifying educational material according to lecture planning and class needs.
As regards 3D sign representations, the platform makes use of GSL linguistic resources, comprising a lexicon and a set of structure rules (Efthimiou et al., 2006), fed into a virtual signer. The lexicon resources are annotated according to the Hamburg Notation System (HamNoSys) (Prillwitz et al., 1989), whereas the structure rules utilize strings of morphemes to compose core sign utterances. The virtual character (VC) of the system is compliant with the h-anim standard. In order for the content designer to interact with a VC, the STEP scripting language is used (Scripting Technology for Embodied Persona) (Huang et al., 2002), since scripted animation is an interchangeable and extensible alternative of animation based on motion capture techniques. In our case, GSL synthesis is the result of a process where a syntactic parser decodes the structural patterns of written Greek and matches it with equivalent patterns of GSL (Fotinea et al., 2005a; Fotinea et al., 2005b). These patterns are then fed into an automated system that decodes HamNoSys notation sequences and transforms them into sequences of scripted commands that "drive" the virtual signer.
Specifications for the formation of the GSL educational content are essentially based on extensive research on the official, recently reformed, guidelines for the teaching of GSL in primary schools for the deaf. Educational content formation follows the same guidelines as the hearing children's curriculum, so that the same grammatical and semantic units can be taught in both languages. Concepts such as subject-object relations, types of verbs and discourse functions of the language form the units of the curriculum, so that the same principles are taught under the same platform, but without projecting a mirror image of Greek grammar onto GSL. For the selection and arrangement of the educational material, the design resulted from close cooperation with the Hellenic Pedagogical Institute.
Evaluation of the platform was carried out by user groups consisting of native signers, GSL students and their tutors in real use environment. A detailed account of the evaluation procedure follows in Section 8.
5. Grammar content of the GSL platform
5.1 Morpho-phonological components of SL grammar
Phonology and morphology of signs comprise basic grammar components of any SL (Brennan, 1987; Stokoe & Kuschel, 1978; Sutton-Spence & Woll, 1999). As a consequence, they form the primary concepts of early years' curricula. Educational content places emphasis on hand-shape, location, orientation, movement and manner of movement, these being the basic parameters for structuring phonology and morphology of language, expressed by the signer's hands, whereas sign formation usually demands participation of the so-called non–manual distinctive features as well, such as eye-gaze and mouthing.
Thus, the manual grammar components are essential to aspects of teaching a sign lexicon, but also to sign utterance issues, which participate in simple vs. complex sign formation rules. It should be mentioned that a base sign is defined as being a gesture or movement that conveys a concept and functions much like a word in spoken languages. Therefore simple signs are the basic semantic units of sign language, composed of, in principle, unlimited combinations of the language's phonological elements. The use of any obligatory non-manually articulated element (e.g. mouth patterns, head and shoulder movements, facial expression and other non-manual features) in the process of sign formation is also governed by specific rules.
5.2 GSL platform content organisation
The grammar content of the GSL platform is organised in three parts corresponding to the grammar components of phonology, morphology and syntax respectively. The first part is devoted to GSL phonology and is divided into chapters, which present exhaustively the manual features of GSL. Every set of teaching units corresponds to a manual parameter, while every single feature instantiation to be found in language forms the educational content of a separate teaching unit. Figure 1 depicts an instantiation of educational content presentation for assessing pupils' ability to recognise different signs formed by means of the same hand-shape (hand-shape 'B' for the GSL signs "table", "house", "donkey" and "tent" in the example).
Part two is devoted to GSL morphology, which provides the linguistic means to compose various semantic units either in complex signs or in sign utterances. The internal organisation of part two follows the above mentioned pattern, where various manual parameters of derivational and inflectional morphology are treated in different chapters, each chapter comprising a set of teaching units for exhaustive reference to the morphemes which fall within each morpheme category.
Part three deals with GSL syntax issues such as the basic discourse functions of affirmation, negation and interrogation, dialogue formation, setting reference pointers to absent entities and changing reference in respect to existing settings in 3D signing space.
Figure 1. Instantiation of educational content presentation.
6. Structuring of educational scenaria
6.1 General considerations
When addressing the issue of educational scenaria in an e-learning platform for GSL (Efthimiou & Fotinea, 2004), standard approaches deploy a range of exercises to cover passive/receptive as well as active participation in the class. Passive/receptive participation scores comprehension declared through indication of acquired objects by students. Active class participation involves students' responses that generate new language items. In the case of GSL, the above requirements are strongly supported by integration of a sign synthesis tool that allows for creative exploitation of GSL resources in a controlled environment.
Conventional means for teaching GSL include -but are certainly not limited to- text, pictures/still images, pre-recorded videos, pantomime as well as signing of oral Greek utterances and written word transcription by finger alphabet. Expansion of the above to include live video (teleconferencing facilities) and a virtual tutor for the presentation of educational content in a dynamic fashion, complete the range of tools available to tutors, for combining any of the above to produce exercises that can be dynamically reviewed and serve different educational purposes.
The exercise models constructed for use in a virtual class environment of the platform under discussion are grouped according to content (restricted – unrestricted), type of activity (single user – user group), type of structure (game – task) and difficulty. following general educational procedures, irrespective of the visual-only mode of instruction. In the same environment, real time scoring by tutors allows for modification of lectures according to class member needs, during sessions. This function is available through exercises that can dynamically change content.
Drills of exercises also entail self-assessment, as students need to correctly complete a task, before being allowed to proceed to the next stage. An automatic corrective procedure employs display of tasks and of student responses by the virtual signer. Exercises for individuals or groups connected to the virtual class may be presented in the form of a "contest" among knots to raise interaction and active participation in the educational process. Furthermore, there is no clash between the new state curriculum for teaching GSL as a first language, and the structure of the educational content of the platform. The design of topic presentation and related exercises follows a bifold theoretical approach. In this sense, teaching units deal with (a) grammatical aspects of GSL and (b) semantics and communication in GSL. This binary orientation remains consistent wherever possible, even in single exercises of each teaching unit. To optimise exploitation of the platform, tutors are clearly guided through the platform's content. To further improve usability by tutors, the terminology used in respect to educational content is similar to that of the state curriculum. Cross referencing between units and clear reference of grammatical purposes and semantic units addressed aim at the same end as well.
6.2 Educational unit structure and content
All educational units are constructed according to a pattern of three unit components:
- introducing students to educational objects (most usually in two stages) to clarify grammatical function and use in language,
- repetition of educational object presentation in a different manner than in component (a), aiming at full integration of the presented topic by pupils, and
- testing pupils' acquired knowledge by means of various exercise drills.
The phonology of GSL, that is the basic articulation parts that comprise every sign in sign languages, i.e. hand-shape, place of articulation, manner and direction of movement (Valli & Lucas, 1995)are taught in a subtle way in the first units, using signs that refer to objects and activities familiar to young students as clarification examples.
Regarding single manual elements corresponding to basic parts of speech, both presentation and exercise vocabularies consist of a group of nouns, a set of verbs morphologically related to some of the nouns and a set of verbs commonly used in the first stages of language acquisition (Moerk, 2000; Barrett, 1999), as well as a number of commonly used adjectives and adverbials. All signs used in topic presentation or in testing components are drawn from a database of lemmas in their basic forms, with all acceptable inflections marked and superimposed where appropriate.
Basic clause structures taught include syntactic formations with members of two out of the three predicate categories in GSL, i.e. with plain and agreement verbs, with provision to include motion verbs in more advanced tutorials, as part of teaching classifier predicates. Clauses with agreement verbs are always transitive, so their structure includes both Subject and Object; clauses with plain verbs are taught in different units, depending on whether or not they take an object.
Basic discourse functions include affirmation, negation and interrogation, with superimposition of non-manual features and of free manually signed interrogators and negators. Finally, provision is also made for the inclusion of units dedicated to broader discourse markers, manual and non-manual.
6.3 Language use – thematic units
There are units related to the sign lexicon, where the focus is on objects and acts typically familiar to users (i.e. connected with real-life in the family, at school and at home). The signs taught in these units are selected from the database according to several parameters: Ease of articulation and morphological clarity is crucial; the first unit tutorials include only signs of single fold hand-shape, location or movement that occur commonly in GSL. Signs whose morphology is complex or rare in GSL are only included in later stages. Another parameter for including vocabulary in the first units is semantic clarity for educational purposes, so as to ensure understanding on the part of the student.
Furthermore, educational material includes phrases that enable students to express feelings and views related to basic social uses of language (i.e. politeness). Respecting the personality of students, and that of a group of linguistically underprivileged young people, it has been a main consideration to include GSL vocabulary that can be used as a tool by young deaf children to express and perceive in a linguistic fashion likes or dislikes, denial or appreciation of situations in tutorials.
6.4 Visual non-linguistic means - visual mode of instructions
It has been acknowledged as a fact that GSL should stand alone as a language and that it should be possible to teach GSL in a monolingual way as a first language (Kourbetis, 1999; Special Education Act 143, 2000). In agreement to first language teaching principles, the default forms of the screen do not display Greek text at all. However, a bilingual glossary can assist some secondary users of the platform, and can also be useful for extensions of the platform in order to support learning of written Greek as a second language.
There is already an option of displaying or hiding Greek vocabulary tags appearing underneath pictures and/or in line with signing by the avatar. In this way, lessons can adapt to suit deaf students who have mastered some Greek vocabulary before learning GSL. In a similar fashion, the provision for Greek text makes the platform easily accessible to hearing parents and teachers, or even hearing peers of deaf users, who can test their understanding of GSL vocabulary. It must be noted, however, that Greek text is limited to vocabulary level; there are no Greek syntactic structures available, as this would risk giving false information or even violating GSL grammar in respect to linear formations.
6.5 Visual means of transmitting information
Visual ways of presenting the platform's educational content were sought, by providing instructions for exercises, as well as feedback on the results to users. These stimuli have been tested by deaf users and have been adjusted according to the results of the testing procedure.
Visual stimuli were considered an essential part of the educational process for two reasons:
- The target audience are young deaf individuals who are in principle visually sensitive and perceptive.
- Visual, non-linguistic stimulation in a structured and systematic way massively improves understanding and learning, especially in children, and more so in children with low linguistic skills.
Such visual means are realised by the following stimuli:
- Indicating educational units and parts of units with specific icons to achieve clarity in navigation.
- Highlighting pictures that are the grammatical focus of an exercise at any given moment.
- Using consistent colour marking of grammatical and/or semantic equivalents.
- Using animated cartoons for positive or negative feedback to students.
- Using animal cartoons that sign in GSL, transmitting linguistic messages such as "correct", "well done", "wrong", "try again", "watch me".
Given that the GSL platform was intended to exploit a range of functionalities with regard to its pedagogical aspect along with its content presentation, provision was also made for increasingly complex pictures to be included in tutorials on more complex grammar structures.
6.6 Organisation of resources for scenaria content generation
As already mentioned, the conventional way to represent linguistic content in software applications involving any sign language including GSL, is video describing pre-defined meanings. However, in the case of dynamic sign synthesis the frozen video representations are replaced with a system that may convert text to signing in a relatively flexible way, based on the amount of available -appropriately coded- linguistic resources. An innovative consequence of integrating a virtual signer in any SL educational platform is that it may allow for dynamic content synthesis by tutors. In the platform under discussion, this is achieved by allowing tutors to modify the content of implemented educational scenaria, provided there are available resources which authorised tutors may access in order to create their own content in the patterns imposed by the scenaria. Furthermore, use of an avatar for sign synthesis allows word family grouping to follow not only semantic categorisation but also morphologically based group formation according to a base-sign and its variations and derivations in GSL.
7. The system's lexicon DB
The platform communicates with a lexicon DB of coded GSL knowledge, which enables educational content modification by tutors, according to audience and lecture needs. Coding GSL knowledge in the lexicon involves signs annotated as to the phonological composition of lemmas, which also supports avatar motion for signing GSL, and descriptions of grammar properties as to functional (semantic and syntactic) and morphological features on base sign elements.
Following general SL formation principles, GSL is structured not only 'by the hands'; facial expressions and turning of the torso, head and/or eye gaze constitute grammatical morphemes (Antzakas & Woll, 2002, on negation, supporting evidence from all known sign languages to date). These non-manual features can mark, for example, quantity or quality of a class of objects, or can be modifiers for verb inflection, so they are considered a crucial issue in GSL grammar learning. On the database, specific feature values indicate whether non-manual features are marked on each lemma and which kind, so that the corresponding set of morphemes can be super structured onto the manually signed elements either as compositional parts of a base morpheme or as modifiers. Such constructions form the content of tutorials on the platform, where authorised tutors may select from the DB grammar structures and corresponding examples that best fit lecture planning and class needs.
7.1 Lemma coding
The lexicon DB contains sign lemmas described as to their phonological structure (Efthimiou et al., 2004a), i.e. the hand shape for sign formation, hand movement, palm orientation and location in the signing space or on the signer's body. As mentioned above, for the representation of the phonological features of GSL, the extended HamNoSys annotation system has been adopted.
The phonological structure of lemmas reveals a number of interesting parameters of sign formation regarding morpheme combinations for the creation of lexical items. 'Breaking down' a default or base sign into its phonetic/phonological components allows the development of an unrestricted, avatar-based device for sign generation, which may compose a new sign, previously unknown to the system, as soon as it meets a string of symbols that dictate a predefined sequence of motions to the virtual signer.
Sign coding is further enriched to provide, except for HamNoSys symbols of motion, other non-manual obligatory features, which accompany hand action in order to make linguistic content fully acceptable to the community of native GSL signers.
The multilayer information coded in the GSL lexicon comprises mouthing patterns, facial expressions and body/shoulder movement, used also for the indication of phonetically (stress) or syntactically uttered (focus position in sentence) elements of the linguistic message in spoken languages. Glossing of mouth patterns follows the conventions suggested by Sutton-Spence & Day (2001), while eyebrow movement and eye gaze are also included, since they are significant obligatory parts of GSL sign formation.
Figure 2 displays part of the lexicon, where HamNoSys annotated lemmas are accompanied by information on obligatory non-manual features (and whether this is considered grammatically significant). In this example, coding involves various mouthing gestures, as for instance, for the verbs 'run' (id 358), 'scold' (id 227), 'accuse' (id 185), 'kiss' (id 379).
Figure 2. HamNoSys annotated lemmas with corresponding information on obligatory non-manual features.
An example of exploitation of lemma coding in sign formation is presented in Figure 3, where the sign for 'children' contains the eye gaze feature performed by the avatar during sign synthesis.
Figure 3. Eye gaze performance when signing "children" in a certain context.
Full feature coding enables tutors to retrieve educational content related to all levels of grammar. Figure 4 depicts a number of features coded with lexical entries in order to provide sign formation details on phonology, morphology and syntax. The 'yes' value dictates obligatory simultaneous performance with HamNoSys annotated hand motions, where 'no' dictates lack of obligatoriness of feature and the null value declares that the specific feature is irrelevant. For example, plural 'YOU' (id 107) as coded in the lexical DB, demands obligatory eye gaze performance.
The morpho-phonological properties of GSL are reflected in the 'real movement' feature (which indicates use of classifiers and real space representation, i.e. the composition of signs for vehicle and physical motion) and the features related to plural formation. The signer needs to know the correct option among those available when signing a plural form, where plural is indicated either by a numeric value or a quantifier on the base sign (i.e. '2, 3, … days', '2, 3, … pencils' etc), by movement repetition and/or change in space (i.e. as in the case of 'book', 'tree' and 'child') or by using '2hand plural' signing, for sign formed with one hand (i.e. 'airplane') (Valli & Lucas, 1995; Sutton-Spence & Woll, 1999). If the educational object is related to phrase formation, which necessarily involves inflecting a verb, tutors may retrieve educational content, as for example on aspect values of predicates. The aspectual values illustrated below do not form an exhaustive description of GSL verb inflections. The aspects below are chosen with educational as well as linguistic criteria, in an attempt to be practical (i.e. to describe the most frequently occurring aspects) and linguistically accurate: 'Durative aspect' indicates that the sign movement continues for longer than default; 'Diminutive aspect' signifies a small span of movement to indicate minimal action/event (i.e. with predicative base signs such as 'wind-is-blowing', 'I-walk', 'I-speak', 'I-eat' etc); 'Intense aspect' denotes greater span and abrupt pauses in movement (i.e. with signs such as 'feel-a-pain', 'it-rains' etc); 'Repeat aspect' indicates obligatory repetition of sign movement with interval pauses (i.e. with signs as 'ask' or 'travel'), whereas the feature 'syntactic movement' is related to verb declination of the so called verbs of agreement (i.e. 'ask', 'scold', 'pay' etc). Different verbal aspects can be mutually exclusive or can occur simultaneously on a given predicate (Sutton-Spence & Woll, 1999; Wallin, 1990).
Figure 4. Feature coding on lexical entries for educational content retrieval.
8. Evaluation procedure
An essential part of system development was testing technical and functional aspects of the educational platform. User evaluation took place under conditions that simulate real use by metalinguistically aware, native users of GSL, as in the definition by Rathmann & Mathur (2002).
8.1 Stages
Tests on the linguistic material were carried out by sign language advisors and native GSL users; this procedure followed an internal pedagogical evaluation of the project, i.e. testing learning processes, time needed to attain each learning target, methods of presentation of learning tasks, as well as usability of the system by its final users. Internal evaluation was carried out in three different stages of the platform's development.
8.2 Sample population
Different end users participated in the different stages of evaluation: Internal evaluation was carried out by experts in the areas of sign language and linguistics, IT technology, language teaching and deaf education. Evaluation by end users involved groups of teachers and pupils from the deaf community.
8.3 Materials and methodological considerations
The first evaluation procedure was carried out on a prototype version of the platform in an experimental environment by teachers and professionals in deaf education and by deaf pupils themselves, while the next two evaluation procedures used as a basis revised versions of the platform, modified according to evaluation feedback.
User evaluation was undertaken by the Hellenic Federation of the Deaf (HFD), which also hosted evaluation meetings. Use of trained sign language interpreters was one of the first prerequisites. Influence of the presence of teachers and/or parents in the navigation and learning procedure were observed in detail, so that navigation of the program by young deaf students could be monitored in different circumstances. Two versions of a structured questionnaire were distributed; the first was addressed to parents, carers and teachers, while a slightly different version was addressed to young deaf pupils. All questions were stated in clear and simple Greek and were interpreted to the pupils and to Deaf adults in GSL. The content of the questions related to the usability of the system as well as its appeal to students in respect of
- navigation in the system,
- learning targets (exercises) and
- graphics used.
It also contained a short section at the end with three open questions, where participants were invited to make personal comments.
8.4 Results and feedback
Comments were offered regarding the use of the platform in teaching and learning practice. Comments as well as responses to the questionnaires were coded in categories. Out of forty-six different comments and suggestions by end users, forty were taken into account in the revised versions of the platform, while six were considered minor and/or would be addressed in the future, due to time and/or technical limitations.
Teacher and parent responses were similar to a great extent and mostly reflected
- positive views of the use of the system,
- satisfaction regarding the level and standards adopted for pupil age groups,
- comments for improved presentation of the virtual signer and
- comments for improved presentation of individual signs.
Responses by pupils were encouraging in the following ways: all questions on the questionnaire were shown to have been understood – which was not self-evident in the case of some pupils with limited communication experience. Most pupils expressed their / a preference to navigate through the platform on their own, or with an adult, and in most cases preferred not to be guided. The potential of working independently was a major aim of the educational aspect from the beginning of the project. Moreover, pupils understood most of the signs used in the exercises. In this sense, the linguistic performance of the platform is satisfactory; signs or rules that were not initially understood were not considered as failure of the program or of individual students, but as further linguistic targets. Finally, aesthetics as well as graphics of the platform received positive comments but left some space for improvement in order to meet the expectations of young children. However, aesthetics is one of the less burdensome aspects of the project, as long as it proved to be of a good standard, from an educational and linguistic point of view. These were all viewed as encouraging comments, strongly suggesting expansion and further development of the platform in order to achieve the potential of connecting educational settings for young deaf pupils in Greece through properly structured use of GSL.
9. Concluding Remarks and Future Development
The work reported here focuses on an innovative approach to educational content organisation of GSL, where pupils receive information on/for GSL in a close-to-natural signing situation, without the limitations posed by the use of video for SL representation, while tutors may organise lecture topics and dynamically retrieve presentation/testing material. Both groups of users benefit from the coded GSL linguistic knowledge, which underlies interaction with an educational platform.
The platform in its current state serves for monolingual GSL language learning. However, moderate extensibility of current functionalities and addition of content-related options will enable its exploitation as a bilingual application, to also support teaching of written Greek as a second language to deaf students. The linguistic resources to support this bilingual application, may well be exploited in the foreseen implementation, where GSL will be taught as a second language. Both environments may be implemented by use of a Greek-to-GSL conversion tool (Fotinea et al., 2005b) that will support lexical and structural matching between the two languages and enable dynamic sign synthesis.
References
[1] Antzakas, K. & Woll, B. (2002). Head movements and negation in Greek Sign Language. In: I. Wachsmuth, T. Sowa (Eds.), Gesture and sign language in human-computer interaction. International Gesture Workshop, GW 2001, London , UK, April 18-20, 2001 Proceedings. Berlin [u.a.]:Springer. P. 193-196.
[2] Barrett, M. (1999). The development of language, Studies in developmental psychology. Sussex: Psychology.
[3] Boyes Braem, P. & Sutton-Spence, R. (Eds.) (2001). The hands are the head of the mouth: the mouth as an articulator in sign language. International Studies on Sign Language and Communication of the Deaf; 39. Hamburg : Signum.
[4] Brennan, M. (1987). British Sign Language, the Language of the Deaf Community. In T. Booth & W. Swann (Eds.), Including People with Disabilities: Curricula for All. Open University Press: Milton Keynes.
[5] COM (2000) 284 final: Towards a Barrier Free Europe for People with Disabilities. Communication from the Commission to the Council, the European Parliament, the Economic and Social Committee and the Committee of the Regions, Brussels.
[6] Efthimiou, E. & Fotinea, S-Ε. (2004). An adaptation-based technique on current educational tools to support universal access: the case of a GSL e-Learning platform, Proceedings of the TeL'04 Workshop on "Technology Enhanced Learning", Workshop to WCC-2004 (World Computer Congress), Toulouse, France, 177-186.
[7] Efthimiou, E., Sapountzaki, G., Karpouzis, K. & Fotinea, S-E. (2004a). Developing an e-Learning platform for the Greek Sign Language. In K. Miesenberger, J. Klaus, W. Zagler, D. Burger (Eds.), Springer Lecture Notes in Computer Science (LNCS), Vol. 3118, 2004, 1107-1113.
[8] Efthimiou, E., Vacalopoulou, A., Fotinea, S-E. & Steinhauer, G. (2004b). A Multipurpose Design and Creation of GSL Dictionaries, Proceedings of the Workshop on the Representation and Processing of Sign Languages: From SignWriting to Image Processing. Information techniques and their implications for teaching, documentation and communication, Satellite Workshop to LREC-2004, Lisbon , Portugal, 51-58.
[9] Efthimiou, E., Fotinea, S-E. & Sapountzaki, G. (2006). Processing linguistic data for GSL structure representation, Proceedings of the Second Workshop on the Representation and Processing of Sign Languages: Lexicographic matters and didactic scenarios, Satellite Workshop to LREC-2006, Genoa , Italy , in print.
[10] Fotinea, S-E., Efthimiou, E., Karpouzis, K. & Caridakis, G. (2005a). Dynamic GSL synthesis to support access to e-content, Proceedings of the 3rd International Conference on Universal Access in Human-Computer Interaction (UAHCI 2005), 22-27 July 2005, Las Vegas, Nevada, USA.
[11] Fotinea, S-E., Kouremenos, D. & Efthimiou, E. (2005b). Generating linguistic content for Greek to GSL conversion, Proceedings of the HERCMA-2005 Conference (The 7th Hellenic European Conference on Computer Mathematics & its Applications), Sept. 22-24, 2005, Athens, Greece.
[12] Gaddes, W. & Edgell D. (1994). Learning Disabilities and Brain Function: a neuropsychological approach-3rd ed. New York; Berlin : Springer.
[13] Hauert C-A. (1990). Developmental psychology: cognitive, perceptuo-motor and neuropsychological Perspectives, Advances in Psychology, 64. Amsterdam; New York: North-Holland.
[14] Huang, Z., Eliens, A. & Visser, C. (2002). STEP: A Scripting Language for Embodied Agents, Proceedings of the Workshop on Lifelike Animated Agents.
[15] Karpouzis, K., Caridakis, G., Fotinea, S-E. & Efthimiou, E. (2005). Educational Resources and Implementation of a Greek Sign Language Synthesis Architecture, Computers and Education, Elsevier, in print, electronic version available on line since September 2005.
[16] Kourbetis, V. (1999). Noima stin Ekpaideusi. Athens, Hellenic Peagogical Institute.
[17] Meier, R. P., Cormier, K. A., & Quinto – Pozos, D.G. (Eds.) (2002). Modality and Structure in Signed and Spoken Languages. Cambridge: CambridgeUniversity Press.
[18] Moerk, E. (2000). The guided acquisition of first language skills, Advances in applied developmental psychology, 20. Stamford, Connecticut : Ablex.
[19] Pieper, M. (2001). Sociological Issues in HCI Design, in: User Interfaces for ALL: Concepts, Methods, and Tools, Stephanidis, C. (Ed.), Lawrence Erlbaum Associates, Publishers, London, 203-222.
[20] Prillwitz, S., Leven, R., Zienert, H., Hanke, T. & Henning, J. (1989). HamNoSys. Version 2.0. Hamburg Notation System for Sign Language: An Introductory Guide, Hamburg: Signum Verlag.
[21] Rathmann, C. & Mathur, G. (2002). Is verb agreement the same cross-modally?. In: R. P. Meier, K. Cormier & D. Quinto-Pozos (Eds.), Modality and structure in signed and spoken language. Cambridge University Press, 370-404.
[22] Stephanidis, C. (2001). Editorial, International Journal of Universal Access in the Information Society, 1/1, 1-3.
[23] Stokoe, W. & Kuschel, R. (1978). For Sign Language Research, Linstock Press.
[24] Story, MF. (1998). Maximising usability: the principles of universal design, Assistive Technol. 10:4-12.
[25] Sutton-Spence, R. & Day, L. (2001). Mouthnings and mouth gestures in British Sign Language (BSL). In: P. Boyes Braem & R. Sutton-Spence (Eds.), the hands are the head of the mouth: the mouth as an articulator in sign language. International Studies on Sign Language and Communication of the Deaf; 39 Hamburg : Signum. 69-85.
[26] Sutton-Spence, R. & Woll, B. (1999). The Linguistics of BSL- an introduction, Cambridge Univ. Press.
[27] Tennant R.,A. & Gluszak Brown, M. (1998). The American Sign Language hand-shape Dictionary. Clerc Books, an imprint of Gallaudet University Press.
[28] Valli, C. & Lucas, C. (1995). Linguistics of American Sign Language –an introduction. 2nd edition. Washington D. C.: Gallaudet University Press.
[29] Vanderheiden, G. (1998). Universal design and assistive technology in communication and information technologies: alternatives or compliments?, Assistive Technol. 10(1):29-36.
[30] Wallin, L. (1990). Polymorphemic Predicates in Swedish Sign Language. In: C. Lucas, (Ed.), Sign Language Research. Theoretical Issues. Proceedings of the International Conference, Theoretical Issues in Sign Language Research, II. May 18-21 1988. Gallaudet University. Washington DC.: Gallaudet University Press, 133-148.
Acknowledgments
The work presented here was partially funded by the national project grant SYNENNOESE (GSRT: eLearning 44). The authors wish to thank all researcher and developer teams that worked on the various modules integrated in the platform.