Preparing Students for Learning in an Online World: an Evaluation of the Student Passport to Elearning (SPEL) Model
Francesca Lorenzi (Francesca.Lorenzi@dcu.ie),
Kay MacKeogh (Kay.MacKeogh@dcu.ie)
and Seamus Fox (Seamus.Fox@dcu.ie)
Oscail - National Distance Education Centre, Dublin City University, Ireland
This paper was shortlisted for the 'Best paper award' at the
Third EDEN Research Workshop, Oldenburg, Germany, 2004
Abstract
This paper outlines the way in which Oscail (National Distance Education Centre - Ireland) has developed its technology strategy, from the initial research on student readiness to learn using technology to more recent studies which have identified the need for a programme specifically designed to prepare students for learning in the online world.
The skills for learning are not necessarily innate, and in particular, the skills for learning with technology need to be recognised and made more explicit. However, regardless of discipline, the development of preparatory courses for students to equip them with the skills for eLearning is essential if maximum benefit is to be garnered from the potential of this exciting means of teaching and learning.
The Student Passport for elearning (SPeL) programme has been designed as a learner-centred pedagogically driven module using a scaffolding approach to gradually introduce students to use of electronic media. Crucial to the success of the SPeL programme has been the level of support offered to students in pedagogical, technical and administrative terms.
Key words:
readiness to learn using technology, learner-centred approach, learning and learner support, learning to learn online, online learning pedagogy, tutoring online
Introduction
In the extensive and on-going discussion on the potential of ICTs in education, one major stakeholder is often overlooked, the student (MacKeogh 2003). Reports and documents promoting the potential of elearning are replete with references to 'users' of educational technology, referring to institutions, teachers, employers, and governments, while the readiness of students to learn using these powerful new technologies is frequently ignored. In this paper, we review the work of Oscail - the National Distance Education Centre in Ireland, in developing its student-centred strategy for introducing ICTs in its distance education programmes. Since its inception in 1982, Oscail has evaluated the potential offered by technology for the delivery of its programmes, in tandem with monitoring access to technology among its students. While ownership or access to PCs and Internet are now virtually ubiquitous among Oscail students, other divides are becoming apparent, in terms of knowledge, expertise and attitudes (MacKeogh 2003). The paper briefly outlines the way in which Oscail has developed its technology strategy. We then summarise the outcomes of research on student readiness to learn using technology. This research has identified the need for a programme specifically designed to prepare students for learning in the online world. The paper will conclude with some comments on the issues involved in supporting students in the new elearning environments.
Introducing technology - strategic considerations
Oscail has been the major provider of ODL in Ireland since 1982. While Oscail is a faculty of Dublin City University, it has a national remit to extend access to qualifications to adults throughout Ireland. Oscail provides graduate and post-graduate education both for those who have previously not had access to education (second chance students) as well as to those seeking to upgrade their qualifications. Qualifications are offered in information technology, the Humanities, nursing, and management. Undergraduate programmes are open to adults aged over 23 years, regardless of qualifications, although all students are required to complete introductory modules before admission to credit modules. Postgraduate students are required to hold undergraduate degrees or professional equivalent. To date, Oscail has used mainly 'second generation' distance education technologies in delivering its courses, utilising a mix of correspondence texts, video and audio tapes, some computer based learning and supported by face to face tutorials in a distributed network of study centres. This policy has been dictated by considerations of cost, accessibility and concerns about the pedagogical quality of most of the technologies which have been developed over the last twenty years.
Nevertheless, Oscail has been deeply involved in monitoring and analysing trends in technology in education for many years (see for example Curran and Fox 1999 and Curran et al 1995). At the same time, annual surveys showed that the level of access to technology by students, who are widely dispersed throughout Ireland and abroad, was insufficient to convert Oscail programmes to 'third generation' modes of delivery. To do so would have meant erecting substantial barriers to our students and reducing enrolments while possibly not adding significant value to our courses.
By the late 1990s national, international and global developments suggested that the time was at last opportune for Oscail to introduce ICTs into teaching its programmes. Since then, with the greater accessibility of virtual learning environments and expanding access to PCs and the Internet in the student population, technology has increasingly been used to enhance pedagogy. In 2001, Oscail introduced its Master of Science in Internet Systems programme. This programme is presented totally online, with students accessing course materials and online tutorial support via the WebCT virtual learning environment. With EU funding under the Minerva programme, an innovative form of structured support for online learning has been developed, initially for modules on the Bachelor of Arts in Humanities programme (Fox and MacKeogh 2001). This model aims to use the capability of the technology to develop higher order skills, in a cost and pedagogically effective manner. Since 2003, most modules at degree level in the BSc in Information Technology are presented online.
In the next phase it is planned to integrate the ICTs in all aspects of Oscail's programmes, administration, pedagogy and assessment, moving towards the ideal of a constructivist approach to learning, and enhanced flexibility for students. It is hoped that within the next five years, students can enrol online, access course materials and resources, and tutorial support, interact in a collaborative environment with their fellow students, and participate in range of assessment modes, including peer and self assessment.
The challenges to moving this strategy forward can be summarised in terms of quality access, skills, perceptions and cost. Among the ways in which the Internet and virtual learning environments can improve quality is by providing off-campus access to learning resources, both on the WWW and through online journals and full-text databases. All Oscail students can now log on to the journal databases provided by DCU library from their own PC and download a wide range of full-text articles and documents, thus solving one of the perennial problems of distance education students who cannot travel to the campus library. Nevertheless, many Oscail students operating from a broad range of environments have experienced great stress and frustration in trying to utilise these resources off campus, for a range of technological and expertise reasons.
Another way of using ICTs to improve learning is to tap into the potential to develop higher-order cognitive skills such as problem-solving, critical thinking, analytical skills and collaborative team working skills, all skills which are difficult to develop using conventional text based distance education delivery. The EU Minerva project referred to above was designed to test a range of online pedagogical techniques and methods which minimise demands on tutor time and simultaneously promote higher order learning. The report of the pilot phase of this project concluded that 'given the appropriate online pedagogical techniques, students will (more or less) by themselves ... display evidence of engaging in higher-order learning. However, further work is required ... before anything categorical can be said about the potential of eLearning to enhance higher-order learning at reasonable cost." (Fox and MacKeogh, 2001: 1). More recent extensions of this model indicate that while quality increases, so also does the cost of provision.
Another challenge to be faced is the readiness of students to engage with elearning. Earlier studies have consistently identified expertise as a key factor in creating positive attitudes to technology. In 2000 a survey of Oscail tutors and students found that proficiency varied with the type of technology (MacKeogh, 2001). Tutors and students rated their skills in word processing, email and WWW relatively highly, but had little expertise in data manipulation, including spreadsheets. A survey of students carried out in 2002 identified a similar pattern of varying levels of expertise. In particular low levels of expertise in computer conferencing were reported (MacKeogh 2003). With regard to attitudes to using technology, almost one fifth (19.2%) of students opted for traditional forms of ODL, without any technological element, while only 11.6% opted for elearning (defined as a mix of written course materials, online materials, online tutorial support and interaction with students and tutors). The majority favoured using ICTs to enhance existing modes of learning, and there was substantial resistance to removing face-to-face and personal contact from the learning experience, even among those who were supportive of technological enhancement.
Arising from the research on attitudes and expertise of our students, as well as evaluation of the experiences of students studying course modules online (see Fox and MacKeogh 2001), it becomes clear that if students are to benefit from the undoubted advantages of elearning, a system of support is required which integrates skills training with academic content, designed to prepare students to succeed in elearning, as well as to enjoy the experience. To this end Oscail has designed two preparatory online modules which aim to integrate the technological skills required for elearning with an introduction to the academic content students will encounter in either the Humanities or Information Technology undergraduate programmes. The Irish Higher Education Authority has funded development of these modules - the Student Passport to eLearning (SPeL) - as the aim is to extend the model to other disciplines in Irish higher education. The first modules were presented between December 2002 and January 2003. The outcomes of the evaluation of the pilot phase are outlined in the next section.
Evaluation of the SPeL modules
All students entering Oscail undergraduate programmes are required to take a short introductory module. This module serves a three-fold purpose, it helps students to update or acquire skills for studying at university level; it provides a 'taster' of distance education and gives students the opportunity to try out the format without committing themselves to a long course; it also gives a general introduction to the discipline. In the traditional mode of delivery, this module uses conventional text based material, supported by face-to-face tutorials. Students complete assignments designed to allow them practise their study skills within a disciplinary context. For the pilot version of SPeL, applicants to the undergraduate programmes were invited to volunteer to participate in the online version of the introductory modules. 40 students volunteered for SPeL Humanities (BA) and 32 volunteered for SPeL IT. Students ranged in age from 24 to 55 years. As might be expected, the majority of Humanities students were female (60%), while the majority of IT students were male (70%).
The Pedagogical Approach
Students on the SPeL modules were not allocated to face-to-face tutorial groups based in study centres. Instead students were led through a carefully designed journey using a detailed roadmap, carrying out specified tasks, and interacting with their tutor and fellow students within the WebCT environment. Students were required to read course materials, participate in online activities, post messages to a discussion area and carry out assessment tasks based on both their readings and online activities. Students enrolled were required to produce a final portfolio, which consisted of all the assessment tasks and print-outs of their online activities. The portfolio was submitted to the tutor who moderated the online interaction. The content of the assessment portfolio is outlined in Figure 1.
Figure 1: SPeL assessment process
The SPeL modules introduced students to the use of the online medium through carrying out tasks which gradually acquainted them with online resources such as e-mail, internet search engines, computer asynchronous conferencing communication, web-based library databases and basic software packages such as Microsoft Word and Excel. The SPeL modules can be described as task-oriented programmes. It was envisaged that these tasks would "accommodate objectives …seen by learners as directly in harmony with their own perceived and stated learning needs" (Breen, 1987: 28) while motivating them to engage with the course content while acquiring study skills. A scaffolding approach to task-design was chosen as the one more likely to succeed in gradually introducing students to the use of electronic media. Students started off with simple tasks such as posting a message to introduce themselves and progressed to more complex ones such as searching and reviewing websites, and entering data on the learning experience in an Excel spreadsheet and finally analysing the learning experience in both quantitative and qualitative terms. Scaffolding was applied to enable students to progress gradually and this helped to ease some of the initial anxiety associated with use of technology, particularly for the Humanities students who initially appeared to have less experience in using ICTs. In other programmes computer skills have been an add-on, often in the form of an ECDL-type separate course. However, it was felt that in order to optimise the acquisition of computer skills it was necessary to integrate technical knowledge acquisition with study skills. Furthermore technical knowledge and study skills were applied to course specific content and this gave students the opportunity to acquire an initial understanding of the subjects they wished to undertake at undergraduate level.
Students were also asked to reflect on the learning experience and to maintain both a learning log and a reflection diary. This added a metacognitive dimension to the experience and helped students to develop greater self-awareness in terms of time management and learning style. As Monteith and Smith suggest "Learning 'online' in an open-access resource centre, where there is the almost permanent option of crossing the boundaries between the social environment, requires students to adopt greater self-discipline and different study skills from traditional forms of learning" (Monteith and Smith 2001: 123). Having to reflect on a daily basis on the learning experience helped students to counterbalance this tendency and as one Humanities student reported
"On week one when I printed off the material I was overwhelmed. But now I am glad I did this course, because I have gained loads of knowledge with reference to my computer and of course the course content. This will help me with my future study skills… The daily log was useful as it showed me what I had done that day, and what other areas I should spend more time on. The study schedule was helpful but I had to be very flexible with reference to adapting my study time around my children, husband and other factors which influence my life".
Online student support
It can be argued that the factor which most contributes to the success of online learning is the level of support offered to students.Four broad categories of support can be identified: 1) support as learner-centred instructional design; 2) support offered by tutor; 3) support offered by peers; 4) technical support.
Learner-centred Instructional Design
From the outset, the design of the SPeL modules was not technology driven, rather it focused on learners' perceived needs as identified through evaluation of learner experiences. As Jung points out "It is instructional design not technology, that is at the centre of quality distance education" (Jung 2000: 1). Laurillard suggests that students in an elearning environment may need mediated access to information (Laurillard, 1996). Furthermore Coomey and Stephenson (2001:38) argue that the lesson learned from online programmes highlights "the importance of structuring the learning activity and designing the materials in order to promote dialogue, secure active involvement of the learner, provide personal or other support and feedback and enable the learner to exercise the degree of control expected".
Bearing those concepts in mind and given that no prior knowledge was assumed, students were provided with a manual with a detailed description of weekly tasks which framed activities in a general plan. This gave students a sense of structured progression. Students were also able to download from WebCT a Study Skills Handbook and programme specific self-instructional text. The content is presented in dialogical and student friendly form. The texts offer frequent opportunities for reflection and encourage students to self-assess their knowledge and understanding before progressing further. The course materials serve as the foundation for the online activities as online discussion topics are based on them. For instance one of the Study Skills Units deals with information gathering and evaluation and the online activity connected to it requires students to log on to the web to search for three websites related to a specific topic (for example, identity) and evaluate their quality and usefulness. The activity therefore links the skill (information gathering) with the course content with the technical knowledge (how to access a website). The integrated approach ensures that there is a parallel progression in, skill, content understanding and technical skill acquisition. Figure 2 summarises the interaction between course content, online activities, the assessment process and the development of technical skills.
Figure 2: SPeL integrated pedagogical approach
Tutor Support
The tutor is key to the effectiveness of online programmes. The tutor keeps the discussions structured and focused and motivates students at times when technical difficulties appear to prevail. In the SPeL modules, tutors could interact with students in the general discussion area or when more personal issues had to be addressed they could choose to email individual students separately. Figure 3 illustrates the distribution of messages by programme. Of the 584 messages posted in the HumanitiesHumanities discussion area, 26 were posted by the tutor, compared with 28 messages posted by the IT tutor out of a total of 103 messages. A closer look at interactions and tutor contributions reveals that the Humanities tutor used messages to direct and structure general discussion, leaving room for extensive peer interaction. In contrast, the IT tutor tended to reply to students on a one-to-one basis in the general discussion area with substantially less interaction between students.
Figure 3: Breakdown of tutor student interaction by programme
While tutors vary in their approach, the nature of the discipline is also of significance in determining the form of the interaction. The Humanities tutor invested in structuring and consolidating activities. This approach was dictated by the need to keep the discussion focused, which was often an issue given that Humanities students frequently failed to follow threaded discussions and posted long messages. The discursive nature of the Humanities course content favoured longer contributions and it was essential for the tutor to remind students of the topic and function of the discussion. A rather different scenario is offered by the IT interactions. The IT tutor spent time encouraging students to interact and this explains the high proportion of social and one-to-one contributions. The less discursive nature of the subject matter in the IT programme meant that students' contributions were shorter and often did not go beyond mere statement of fact. The tutor often reminded students of the task and primarily provided instructions and practical advice. Figure 4 represents interventions (%) according to the type of contribution/ support offered by the tutors.
Figure4: Analysis of tutor support interventions by programme
Support offered by peers
One of the most convincing arguments for introducing elearning is its potential to provide the social interaction missing from conventional text based distance education. One student at the end of the module wrote that she was "going to miss this little community", another added that she was going "to miss the discussion very much", one student felt that "reading other people contributions gave [him] confidence to carry on". Those comments summarise the general feeling that emerges from reading the messages posted in the general discussion area. Students truly engaged in the online discussions and a sense of community was generated by the constant interaction.
Analysis of the online interactions for both SPeL modules shows that most of messages were posted by students. The classification of messages is not straightforward. Often messages carry more than one communicative function. In analysing these interactions, a discourse analysis approach was taken (Brown and Yule 1983) and for the purpose of analysis the interactional value of the messages was chosen as a key to the interpretation and classification. Those messages with low interactional value, such as two-line messages accompanying posting of a task, were classified as "other" and were not analysed for the purpose of this study. Student interactions were classified according to Salmon's five step online teaching and learning framework (Salmon 2000, 2002). Messages could be classified as offering moral support and social interaction (28% Humanities; 50% IT); technical advice (11% Humanities, 7.5% IT); sharing information (12% Humanities, 13% IT); knowledge construction (12% Humanities, 0% IT); other (31% Humanities; 28% IT). From these figures it can be seen that a large proportion of messages served the purpose of getting students to know each other and the high level of social interaction was a motivating factor for more reluctant students. Moral support offered by peers was particularly important when technical or administrative problems appeared to undermine the smooth running of the programme. Knowing that the other students were experiencing similar problems helped some to stay on board and build bonds with peers. IT students appeared to provide each other with less technical advice, as most likely, they had sufficient technical knowledge to be able to deal with technical issues on their own. On the other hand, Humanities students often offered a word of advice to their peers particularly because their level of technical expertise varied greatly and those more experienced willingly offered help. While students were explicitly asked to offer feedback to their peers on activities posted in the general discussion area, the level of feedback offered is still quite limited and this might be explained by a certain degree of embarrassment in criticising their peers. In most cases students only offered positive feedback. It might be argued that peer feedback was not necessarily perceived as a form of support by students unless it was offering positive feedback.
Perhaps the second most important aspect of the interaction among peers was that of sharing information. The information shared ranged from valuable web addresses, to advice on how tasks should be carried out to simply sharing frustration when the postal delivery of one of the course texts had been delayed. According to Salmon (2002) when constructing knowledge students interact with course content and with each other in a critical, and creative fashion and the Humanities interaction offers evidence of exchanges moving beyond mere regurgitation of course content. Peer support in this case can be seen as progressive activity whereby the analytical and critical interaction with the course content is enhanced by interaction with peers. Humanities students managed to move to this level of interaction while messages from IT students offer little or no evidence of this type of interaction. This might be explained by various factors. The discursive nature of the Humanities might have helped and perhaps, in general, the greater level of interactivity in the Humanities area might have also contributed to building a sense of community which might have led to greater ease in interacting with each other at a more analytical level.
Technical Support
Students were provided with an online step-by-step manual to help them to use WebCT that could be downloaded and printed off. The manual was designed specifically for this programme and its simple and non-technical language made it approachable even by those students with no previous technical knowledge. Students were also provided with a personal email address, password and username and could access the Dublin City University library resources. Students could also contact the Oscail Technical support team by email or phone and this was a key element at a stage when students were unable to use the online databases for a brief period of time. However, as discussed above, in most cases technical support was offered by peers and students chose to contact Oscail when they could not resolve issues themselves or with the help of the tutor.
Outcomes
Students who had successfully completed the pilot SPeL module assessment tasks were eligible to enrol for undergraduate programmes. At the end of the modules, some 70% of IT students and 52% of Humanities students successfully completed the programmes. While the reasons for non-completion are the subject of more intensive investigation, this is, nevertheless, quite a satisfactory result and compares favourably with the completion rate of the traditional face-to-face version of the Introductory Modules. Some more advanced technical ability might have resulted in a higher completion rate for the IT students, however only a more detailed analysis should establish whether the technology, the pedagogical approach, content or other factors have influenced the retention rate. Interestingly, in following up the pilot SPeL students as they proceeded through their first undergraduate modules in 2003, a higher proportion of both Humanities and IT students successfully submitted their first assignment compared with students who had followed the conventional version of the module. It should be noted that the first assignment for undergraduates is due very early in the academic year and the input of credit module tutors is minimal. It could be suggested that the online introductory module may have contributed to enhancing students' confidence in their skills and ability to meet deadlines. A more detailed analysis of the student behaviour for the duration of the first academic year is underway.
The SPeL modules have been revised in the light of the evaluation of the pilot modules. The SPeL IT programme was modified to place less emphasis on content and more on activities in which students have to interact. The overall strategy is to finetune the approach in respect of Humanities and IT before expanding the model to other disciplinary areas.
Summary
This paper has indicated the crucial need to ensure that students are ready to learn through elearning. The skills for learning are not necessarily innate, and in particular, the skills for learning with technology need to be recognised and made more explicit. Of course these skills cannot be learned in isolation from the disciplines in which they are exercised and different approaches are required for different disciplines. However, regardless of discipline, the development of preparatory courses for students to equip them with the skills for elearning is essential if maximum benefit is to be garnered from the potential of this exciting means of teaching and learning.
References
1. BREEN, M (1987) Learner Contribution to Task Design in Candlin, C and Murphy, D (Eds) Language Learning Tasks Prentice Hall, London, 23-46
2. BROWN, G AND YULE, G (1983) Discourse Analysis Cambridge University Press, Cambridge
3. COOMEY, S AND STEPHENSON J (2001) Online learning: it is all about dialogue, involvement, support and control - according to the research, in Stephenson J (Ed) Teaching & Learning Online, (2001), Kogan Page, London
4. CURRAN, C AND FOX, S (1999) Telematics in Open and Distance Learning Dublin: National Distance Education Centre/Deutscher Studien Verlag Weinheim
5. CURRAN, C, FOX, S AND GOLDRICK, I (1995) Telematic Services for Tertiary Distance Learning (Final Report: 1 -mimeo) National Distance Education Centre, Dublin prepared for DGXII of the Commission of the European Communities
6. FOX, S AND MACKEOGH, K (2001) The PICTURE eLearning Project: Online Resources: Pedagogical Techniques, Higher Order Learning and Tutor Support Dublin: Oscail (http://www.oscail.ie/academic/picture.php)
7. JUNG, I (2000) Technology Innovations and the Development of Distance Education: Korean Experience Open Learning (15) 217-231
8. LAURILLARD, D (1996) Higher Education for Capability. The virtual university: Myth or reality Capability (2) 62-65
9. MACKEOGH, K (2001) Access, Expertise, Attitudes and ICTs in the Humanities: An Irish Case Study in EADTU (ed) EADTU Millennium Conference: Wiring the Ivory Tower: Linking Universities Across Europe Paris 28-30 September 2000 Proceedings Heerlen: European Association of Distance Education Universities pp 286-295
10. MACKEOGH, K 2003 Student perceptions of the use of ICTs in European education: Report of a survey Dublin: Oscail - Dublin City University http://www.oscail.ie/academic/picture.php
11. MONTEITH, M AND SMITH, J (2001) Learning in a Virtual Campus: The Pedagogical Implications of Students' Experiences Innovations in Education and Teaching International (2) 119-132
12. SALMON, G (2000) E-Moderating The Key to Teaching and Learning Online, Kogan Page, London
13. SALMON, G (2002) E-tivities The Key to Active Online Learning Kogan Page, London