Support Services in E-Learning - an Eevaluation Study of Students' Needs and Satisfaction
Professor Torstein Rekkedal (torstein.rekkedal@nki.no) and Svein Qvist-Eriksen
(svein.qvist-eriksen@nki.no)
NKI Distance Education
Box 111, 1319 Bekkestua
Norway
Abstracts
English Abstract
The article reports from an evaluation study among NKI Internet students. The main aim of the study was to examine students' need for support services in Internet based distance study and examine the students' satisfaction with support services. The study was carried out as part of the project, Student Support Services in e-Learning, supported by the Socrates Minerva Programme of the EU Commission. The other partners of the project have carried out similar evaluation studies. For reasons of comparison between different countries, institutions and systems, the quantitative part of the evaluation instrument was, except for language differences, identical with the one used in the studies conducted by the other four project partners, Ericsson Competence Solutions and Cork Institute of Technology in Ireland, FernUniversität in Germany, University of Rome III in Italy. The NKI evaluation study was designed to supply results to form the basis for planning quality developments of the learning management system, organisation and counselling services.
Norwegian Abstract
Denne artikkelen rapporter fra en evalueringsundersøkelse blant Nettstudenter ved NKI Fjernundervisningen. Hovedmålet med undersøkelsen var å få mer innsikt i nettstudenters behov for støtte- og oppfølgingstiltak (student support services) og få informasjon om studentenes holdninger til og tilfredshet med oppfølgings- og støttesystemene slik de fungerer i dag. Studien ble gjennomført som en aktivitet i prosjektet, Student Support Services in e-Learning, som er et prosjekt under EU-kommisjonens Sokrates Minerva program. De andre partnerne i prosjektet gjennomførte tilsvarende studier blant egne studenter. Prosjektet ønsket at disse parallelle studiene skulle kunne gi data som ga noe grunnlag for sammenlikninger mellom deltakeres syn på behov for støtte- og oppfølgingssystemer på tvers av land, institusjoner og e-læringssystemer. Derfor ble den kvantitative delen av evalueringsinstrumentet konstruert slik at det bortsett fra språkforskjeller skulle bli mest mulig identisk med spørreskjema benyttet av den andre prosjektpartnerne, Ericsson Competence Solutions and Cork Institute of Technology i Irland, FernUniversität i Tyskland, og Universitet Roma 3 i Italia. NKI-undersøkelsen ble designet med sikte på å gi informasjon som grunnlag kvalitetsforbedringer i vårt LMS, SESAM, NKI Fjernundervisnings organisering av nettundervisningen, og våre veilednings-, rådgivnings-, støtte- og oppfølgingssystemer. Studien omfattet studentstøttetiltak fra prospektivfasen til etter fullført studium. Et interessant funn er at studentene generelt er tilfreds med støttesystemene og at de faktisk er mest fornøyd med de støttetiltak som de mener er mest viktig.
Key words
e-learning, student support, empirical research, evaluation
Introduction
This article reports from an evaluation study carried out as one of NKIs work packages of the project Student Support Services in e-Learning supported by the Socrates Minerva Programme of the EU Commission. For reasons of comparison between different countries, institutions and systems, the quantitative part of the evaluation instrument was, except for language differences, identical with the one used in the studies conducted by the other four project partners, Ericsson Competence Solutions and Cork Institute of Technology in Ireland, FernUniversität in Germany, University of Rome III in Italy.
NKI decided to expand the questionnaire, originally containing mainly 5-point opinion/attitude scales with a number of open-ended questions. Thus, the survey was done by telephone interviews where the interviewer filled in the questionnaire and also carried out a structured interview when talking on the phone. The interviewers came from different departments at NKI. The selection of interviewers was done with the intention of covering different experiences, priorities and attentiveness to different aspects of student needs. The interviewers were instructed by the main researchers in a meeting before starting the interviews to secure a certain degree of common procedures between the interviewers. Except for a few interviews carried out by the main researcher to try out the questionnaire, all interviews were carried out during one afternoon. The interviewees were selected from students who had completed a study programme during the last 6 months and registered active students, who had submitted a minimum of one assignment during the last 6 months. The interviewers phoned students according to a list organised according to student numbers. Except that in some instances students are registered in groups, the lists can be considered to organise students nearly at random. The interviewers were instructed to phone students following the list, and at the same time try to include completed and active students, both genders and cover different types of courses at different levels. Those 47 students reached on the phone and who were also willing to answer the questions during one afternoon, were included in the sample. Only a few students asked to be excluded because of inconvenience/lack of time. No one expressed negative attitudes towards being interviewed.
The main researchers (authors of this article) have processed the quantitative data and translated and systematised the open answers.
Definitions
The project deals with the role of student support services in e-learning. In the literature there seem to be as many definitions of e-learning as there are writers on the subject (see e.g. Rekkedal & Qvist-Eriksen (2003), Súilleabháin (2003)). In the project 'e-learning' is defined as "...the provision of education or training electronically, via the Internet". The term 'student support services' is used for "...those parts of a distance or electronic learning system, which are additional to the provision of course content" (Thorpe 2001).
NKI Online Distance Education
NKI online distance education (or Internet/web based e-learning)
NKI was probably the first European online college, and it has offered distance
education online every day since 1987. Few - if any - online colleges in the
world have been longer in continuous operation.
NKI Distance Education has today well above 300 courses and more than 60 complete
study programmes on the Internet. In October 2003 we had 5,800 registered active
students. Contrary to many other educational providers, where the Internet is
used as a supplement to face-to-face teaching or other forms of distance education,
we have followed the philosophy that in principle all communication can be taken
care of through the Internet, and ideally no obligatory physical meetings should
be required. (This does not mean that the students are not free to communicate
by post, phone or fax or that study materials preclude print, audio or video
technologies.)
NKI Internet College '4 Generations' of development
The NKI Internet College has been developed through 4 systems generations:
1987 - 1994:
'1st generation' based on the conferencing system 'EKKO', a menu based conferencing system designed by our in-house systems developers as no other options were available that could be installed on the mini computer in NKI at that time. The idea was that we, through 'electronic means', could establish a virtual school and be able to simulate electronically all communication needs previously organised through solutions of combined distance teaching and local face-to-face classes.
1994 - 1995:
'2nd generation' - 'the open electronic college' with the underlying philosophy of offering a system as 'open as possible to other networks and services based on the Internet, e-mail and Listserv conferencing' system.
1996 - 2001:
'3rd generation' - the introduction of graphical interface and the WWW, taking the step from 'small scale experiments to large scale Internet based distance education', introducing courses and programmes below university and college level.
2001- :
'4th generation' - In our experience, it is the step from small-scale to large-scale operation that involves the greatest challenges. Putting some teaching material on the Internet and offering one single course is not a very difficult task. The great challenge is to develop and administer an Internet based teaching organisation offering a large number of courses with high quality to a large number of participants on a continuous basis. This is why we experienced a great leap forward when in March 2001, we launched what we characterize as the '4th generation' with the introduction of SESAM (Scalable Educational System for Administration and Management). SESAM is our internally developed learning management system completely integrating the teaching-learning system on the WWW with our overall student administrative system (STAS). The learning management system and the student administrative system together form the basis for the complete system of student support services.
Didactical solutions for distance teaching and learning
Based on theory and research from the field of distance education, including our own research, NKI has chosen this basic philosophy for the development of Internet based education at NKI: Flexible and individual distance teaching with the student group as social and academic support for learning. NKI recruits thousands of online students every year. These students may enrol in any of the more than 60 study programmes or 300 courses or in any combination of courses at any day of the year and progress at their own pace. This flexibility does not exclude group-based solutions in co-operation with one single employer, trade organisation or local organiser.
The overall choice of didactical solution does have some specific consequences for the design and operation of some aspects of the teaching-learning system. For instance, the NKI Internet courses generally put great emphasis on individual studies and individual work on exercises and assignments for electronic submission to a tutor for comments, evaluation and feed-back. Although it has been a main objective from the very beginning to exploit the possibilities for learning support from the learning group through interaction and discussion with fellow students, in most courses the majority of students have not taken the advantage of using the course Forums very actively.
Results from a number of evaluation studies have shown that individual flexibility is the most important and valued aspect for the majority of NKI online students (Rekkedal & Paulsen 1998, Rekkedal 1998, Rekkedal 1999). This means that learning methods heavily based on co-operation and collaboration with fellow students in real time and/or paced and rigid progression schedules do not seem to be in line with the needs of the NKI distance learner.
On the other hand, learning theories based on social constructivism (see e.g. McConnell 2000) have dominated the ground, at least in Scandinavian academic institutions, and have had considerable influence on higher education institutions when embarking on distance education. It is not difficult to agree with the ideal of theories of learning as a social process where students help each other to develop understanding in an enjoyable and stimulating context and not only as an individual pursuit concerned with accumulating knowledge. The learning is process-driven and learners must be involved in the social process and pay attention to this process to achieve their desired goals. The outcomes are not only academic, but involve increased competence in working with others, self-understanding and self-confidence. Learning is seen as a construction of meaning in interaction with others (teacher and fellow students). Knowledge is constructed in social groups.
However, according to NKI research and evaluation studies we must acknowledge that for most distance learners the primary social arena for learning is often not the group of fellow students, but their local society and environment including family, friends and colleagues. Our main challenge may be to find solutions in designing learning methods and support services stimulating individual learning and understanding constructed in interaction with the student's own social environment, colleagues, friends, family and others.
Student support in the NKI Online Distance Education System
The table below was presented by Rekkedal & Qvist-Eriksen (2003):
Time |
Support needs |
Component responsible |
Tools/applications |
Prospective phase |
Information about courses |
Administration |
Print, WWW, print/ broadcast media etc. |
Guidance concerning choice of courses and programmes |
Administration |
Phone, e-mail |
|
Financial questions, loans, grants |
Administration |
Print, phone, e-mail |
|
Guidance on practical matters |
Administration |
Print, phone, e-mail |
|
Start-up phase |
Dispatch of printed and other physical learning materials |
Administration |
Surface mail |
Registration/information/user identity, passwords etc. |
Administration |
||
Introduction to online learning techniques |
Administration |
Phone, e-mail |
|
Initial follow-up |
Administration |
Phone, e-mail |
|
Technical support |
Administration |
Phone, e-mail |
|
Learning phase |
Teaching/tutoring |
Faculty |
Phone, e-mail, Forum, WWW media |
Academic support |
Faculty |
Phone, e-mail, Forum |
|
Organisation of learning |
Faculty |
Phone, e-mail, Forum |
|
Social support |
Faculty |
Phone, e-mail, Forum |
|
Assessment |
Faculty |
Phone, e-mail, Forum |
|
Practical support, economy etc. |
Administration |
Phone, e-mail, Forum |
|
Follow-up |
Administration |
Phone, e-mail, surface mail |
|
Technical support |
Administration |
Phone, e-mail, Forum |
|
Resources/library |
Administration |
Print, WWW |
|
Learning group support |
Fellow online students |
Phone, e-mail, Forum |
|
Local learning support |
Local faculty |
Face-to-face |
|
Local administrative support |
Local administration |
Face-to-face, phone, print |
|
Local technical support |
Local faculty |
Face-to-face |
|
Local social/practical support |
Employer |
Face-to-face |
|
Graduation |
Diploma/accreditation |
Administration |
Print, face-to-face |
After graduation |
Counselling on further study |
Administration |
Print, e-mail, WWW |
Counselling on job opportunities |
Administration |
WWW, Forum |
|
Alumni services |
Administration |
e-mail, WWW, Forum |
Figure 1. Framework of student support services for NKI online distance students
The support services for NKI e-learning students have been described in more detail by Sjaastad et al. (2003), where the 'Prospective' phase in the table above was described as 'Information/Guidance Phase'.
Research Methodology
Qualitative and quantitative survey
The evaluation study was based on a questionnaire developed in co-operation between the project partners according to a "grid of possible student support services during the different phases of an e-learning process". NKI decided that instead of distributing the questionnaire in the post or on the web, we would design the study as a structured telephone interview. This was both for time reasons and because we wished to get more comprehensive information as a basis for quality development of our systems. Thus, we decided to construct a number of open-ended questions to get more substantial verbal responses. Consequently, we chose to collect both quantitative and qualitative data.
Collection of data
8 different interviewers with different functions, experiences and competencies concerning student support carried out the interviews. The interviewer group consisted of 4 people from the R & D Department, 3 from the Department for Student Support and Counselling, and 1 from the Marketing Department. Before starting the interviews all interviewers met for instruction and discussion to make the interviews as similar as possible. Before this preparation meeting the researchers responsible for the evaluation project revised the preliminary questionnaire based on a try out with 6 interviewees.
Selection of interviewees
We decided to interview a sample of students with recent experiences from the NKI Internet College and ordered a list of all active students and students who had completed an e-learning study programme during the last 6 months. 47 active and completed students who were reached by the 8 interviewers during a 3 hours period one afternoon were included in the sample. Only some very few students claimed to be too busy to be interviewed at that particular time when they were reached on the phone. Only one student gave a negative impression and refused to be interviewed, also with reference to difficulties with time available.
Data processing and interpreting answers
The student answers both on the quantitative and qualitative part of the interview were recorded on individual questionnaire word document forms during the interviews. The quantitative data was processed by the FOSS (File Oriented Statistical System) programme package (Amundsen 2001). The researchers have interpreted the open answers written by the interviewers during writing of the report.
Results - need for and satisfaction with student support services
Information phase
The quantitative questionnaire contained questions on the need for and satisfaction with support elements such as information about course availability as e-learning, about the total programme and individual courses, pricing, financial questions such as payment schemes, loans and grants, technical questions, possibilities of contact with student advisors, web information with FAQ pages etc.
The structured interview covered the same areas on how the students had experienced the information and their contact with NKI advisors. It was very clear that thorough information on all aspects of the e-learning programme before enrolling is seen as 'important' or 'very important'. Generally, the students were also very satisfied with the information they had received. Both the quantitative and qualitative part of the interview showed that the possibility for direct personal contact with advisors is considered to be important, and the students were also generally very satisfied with this support element, e. g.: "Very satisfied, the advisor was very nice on the telephone, and I received the help and everything I needed" or "I was in contact and got answers from people that knew their job".
Registration Phase/Start-up Phase
The questionnaire and interview on the start-up phase contained questions concerning real time technical support services, registration issues, and online learning techniques. Research has shown that the first phase of studies can be critical for study success or drop out from distance study (see e.g. Rekkedal 1972). There is reason to believe that a similar relationship is present in e-learning.
NKI has put large emphasis on technical support services by e-mail and telephone and on systematic follow-up of newly enrolled students. In this study a large majority found technical support to be important, but, in fact, quite few said that these services were 'very important'. Most of the students had not actually needed technical support. Those, who had used the support service, were generally satisfied. Concerning online learning techniques it seems that the students fall into two separate groups, either they find this support element to be really important or not to be important. Very few are indifferent to this question. NKI has for many years offered an introductory course in study techniques to students taking lower level courses. It seemed clear that respondents with low-level previous education and little experience from independent study would have liked to get more support concerning learning techniques: "I missed some information about study techniques. That was lacking". On an open question on whether the start of studies functioned satisfactorily, the majority answered positively. However, some had experienced problems. Especially, it seems to be frustrating if something is lacking in the package of learning materials at the start: "Some books were delayed. I lost some motivation as it took some weeks to get all the learning materials". Some students also miss a closer follow-up scheme in the start-up phase: "Some more deadlines and follow-up in the starting phase would have been good, so that one does get started. It is too easy to postpone the first necessary efforts".
Learning Phase
The support elements covered concerning the learning phase were access to and support from student advisors, tutor access in general, the possibility of contacting tutors on phone and e-mail, feedback on assignments, possibility to communicate with other students on e-mail and phone and discussion forums. This study confirms previous NKI research that easy access to the tutor and quick feedback on assignments are 'very important', while interaction with students is seen as less important by many. The students seem to be very pleased with support and help from their personal student advisor at NKI during their studies. Very few students find communication with fellow students as 'very important' for their learning. The students seem to be 'satisfied' and to a large degree 'very satisfied' with the services they have said to be most important, i.e. communication with their tutor(s) and speed and quality of feedback on assignments. It should be noted, however, that a large group of students who have experience with more than one tutor, report large differences, and often they have been dissatisfied with at least one of their tutors. It also seems that the tutors, who receive low ratings, supply low quality substance in their feedback, and they are also slower in returning assignments. There is certainly a challenge in following up tutors better to assure an evenly high and more predictable standard of tutor work. And, although communication with fellow students is rated as being less important, there is need for developments that make the course discussion forums function better that they do today for students who wish to take advantage of communication, discussion and collaboration with fellow students.
Summary and conclusions
The overall impression from both the quantitative and qualitative parts of the evaluation survey on support services in e-learning at NKI was that the students find all the support services supplied as important or very important.
The table below gives an overview of mean values for importance and satisfaction with all the support elements examined in the qualitative part of the questionnaire.
Table 1. Mean values for Importance and Satisfaction with the support elements.
| Support element | MImportance | MSatisfaction |
| Information regarding course availability | 4.47 | 3.39 |
| Information regarding course or module content | 4.56 | 3.47 |
| Information regarding the larger programme to which the courses belong | 4.41 | 3.25 |
| Information regarding pricing | 4.11 | 3.45 |
| Information relating to course costs, grants etc. | 3.49 | 3.18 |
| Possibility to contact NKI by phone, e-mail etc. | 4.41 | 3.39 |
| Information on the web on registration, access etc. | 3.72 | 3.17 |
| Access to real time technical support services | 4.04 | 3.11 |
| Support regarding registration issues | 3.84 | 3.43 |
| Information regarding online learning techniques | 3.91 | 2.79 |
| Possibility to contact tutors via e-mail telephone etc. | 4.65 | 3.59 |
| Possibility to contact other students via e-mail telephone etc. | 3.20 | 3.16 |
| Discussion forums/bulletin boards | 3.60 | 3.00 |
| Online tutorials | 4.18 | 3.39 |
| Tutor access | 4.70 | 3.31 |
| Feedback on assignments submitted | 4.89 | 3.52 |
| Advice on accreditation, certification and further study | 3.78 | 3.00 |
Importance Scale: Not important at all=1, Unimportant=2, Don't know=3, Important=4,
Very important=5, Scale Mean=3.00
Satisfaction Scale: Very dissatisfied=1, Dissatisfied=2, Satisfied=3, Very satisfied=4,
Scale Mean=2.50
Figure 2 below shows the relationship between mean values of answers to the questions of 'Importance' of each support element and the students' 'satisfaction' with the same support element. The figure clearly illustrates that the students express a high degree of satisfaction given by all the mean values, except one, being 3 or higher. The figure also clearly illustrates that there is a relationship between importance and satisfaction showing that generally the students are most satisfied with the support elements that they perceive to be most important.
Figure 2. Relationship between "Usefulness/Importance" of the support elements and "Satisfaction" with the support elements.
In line with the fact that the students express a large degree of satisfaction with the support services experienced from the prospective phase until graduation, the following quotations are typical answers to the final question on overall opinion of support services:
"I am very satisfied. I am impressed by the totality of services. I thought Internet based learning was complicated until I started my studies at NKI."
"All in all I am satisfied. According to my experiences I could take another course - and recommend it to others."
"I am satisfied. One can always take contact with the tutor and receive good answers. Specifically the tutor is a great help."
"I am very satisfied. I have received good information, and I know that there are people there who answer when I need it."
"I have absolutely been satisfied - I will gladly return to other courses if I decide to continue to study."
The survey ends with the following recommendations:
Support services offered to the NKI e-learners today are generally seen as important and satisfactory by the students - thus, the developments should continue according to the same philosophy of serving the individual learner and his/her need for flexibility.
Priority should be given to:
- Close follow up and support during the start-up phase concerning how to get started and how to make personal progression plans.
- Introduction to efficient techniques for online learning, specifically for students with little experience of independent learning.
- Follow-up and guidance of tutors who do not satisfy requirements concerning turn-around time and quality of comments/feedback to students.
- Developing course forums to function better for students who want more interaction with fellow students, without requiring too much participation for students who prefer more individual studies.
References
1. AMUNDSEN, A. (2001): FOSS - Fil-Orientert Statistikk-System. Veiledning for brukere.
2. MCCONNELL, D. (2000): Implementing Computer Supported Cooperative Learning. London/Sterling: KoganPage.
3. REKKEDAL, T. (1972): Correspondence Studies - Recruitment, Achievement and Discontinuation. Epistolodidaktika 1972:2, pp. 3-38.
4. REKKEDAL, T. (1998): Courses on the WWW - Student Experiences and Attitudes Towards WWW Courses. An Evaluation Report Written for the MMWWW Leonardo Online Training Project. http://www.nki.no/eeileo/research/eei/Rekkeval.htm
5. REKKEDAL. T. (1999): Courses on the WWW - Student Experiences and Attitudes Towards WWW Courses - II (1999). Evaluation Report Written for the MMWWW Leonardo Online Training Project. http://www.nettskolen.com/pub/artikkel.xsql?artid=147
6. REKKEDAL, T. & PAULSEN, M. F. (1998): NKI Internet Students - Experiences and Attitudes. The Third Generation NKI Electronic College. An Evaluation Report Written for the Leonardo Online Training Project. http://www.nettskolen.com/pub/artikkel.xsql?artid=149
7. REKKEDAL, T. & QVIST-ERIKSEN, S. (2003): Internet Based E-learning, Pedagogy and Support Systems. http://learning.ericsson.net/socrates/doc/norway.doc
8. SJAASTAD, J., AKRE, A.-K., L. & REKKEDAL, T. (2003) NKI Distance Education - Case Study Student Support Systems in E-Learning. http://learning.ericsson.net/socrates/doc/norwayp2.doc
9. SÚILLEABHÁIN, G. O. (2003): Principles, Structure and Framework of e-Learning. http://learning.ericsson.net/socrates/doc/cork.doc
10. THORPE, M. (2001): Learner support: A new model for online teaching and learning. Paper to the 20th ICDE World Conference, Düsseldorf.
Authors:
| Professor Torstein Rekkedal Director NKI Distance Education Box 111, 1319 Bekkestua torstein.rekkedal@nki.no |
Distance Education Svein Qvist-Eriksen NKI Distance Education Box 111, 1319 Bekkestua svein.qvist-eriksen@nki.no |