NEW

Best of EDEN 2018 Special issue
 

There are not any recent contributions.

There are not any recent contributions.

Archives

EURODL Mailinglist

2851 subcribers
 

EURODL Visitors

back

Developing an ODL Quality Assurance Model Using Students' Expectations of Tutor Support Needs: An outcome of the Socrates funded CEESOC Project

Keith Stevenson¹, Ulla Muda², Conny Karlsson², Annamaria Szeky³, Paul Sander⁴, Tim Read⁵
1. Leicester University, UK /Open University, United Kingdom; 2. SSVN, Norrkoping, Sweden; 3. KVIF Budapest; 4. UWIC Wales, United Kingdom; 5. UNED Spain.
Correspondence to be sent to Keith Stevenson, Leicester University, Department of General Practice, Leicester General Hospital, Leicester, UK LE5 4PW

© EURODL 2000

Abstract
Introduction
Initial Findings
CEESOC Project year two (1999/2000)
Outcomes of the research
Products
Value of the Expectations Driven Quality Assurance Process to students, tutors and ODL institutions
The importance of organisational partnership
Conclusions

Abstract

This paper outlines the thinking and methodology employed by a transnational team of six ODL tutors conducting EU funded research through the Socrates programme into the expectations of ODL students in 4 European countries. Initial findings about student expectations are reported. During the second year of the project the team concentrated on producing valid and reliable questions in questionnaires. The outcomes of this second phase are presented and a model of a quality assurance process using student expectations of tutor support needs is presented. The article ends with a call for organisational partnerships.

Key words: quality assurance, student expectations, EU funded research

Introduction

This paper outlines the thinking and methodology employed by a transnational team of six ODL tutors conducting EU funded research through the Socrates programme into the expectations of ODL students in 4 European countries. The team met up at the EDEN conference in Budapest in June 1997. After email discussions and a preliminary meeting in London they agreed to bid for funding to carry out research into European ODL students’ expectations of tutoring and tutor support. Funding was granted to pay for administrative and travel expenses incurred in the project. In October 1998 the team agreed an ambitious plan to collect and compare student expectations from samples of their students in each country.

The team agreed that expectations that students had about the course they were about to start had to be collected before their courses began. The team also agreed that they should consider the extent to which students felt their expectations had been met. These data could be collected at two points; shortly after the course had begun and also as the course completed. It was agreed that the 4 data collectors should try and collect information from 60 students in each country, (30 following a language learning course and 30 following political or social sciences). The data from students could be gathered through three common, mainly closed, tick box questionnaires with occasional opportunities for comment. The questionnaires were designed by the team to have common questions applied to all four countries. They were developed in English and translated into the three other languages involved in the project. The results of the questionnaires were coded onto a pre-prepared grid and then analysed centrally by the data analyst. The questionnaires were:

• SEDL: to record the students' expectations prior to commencement of the course.
• ERM: to measure the match between students’ expectations and the reality of the course, 8 to 10 weeks after the course commenced.
• EVAL: to evaluate the course against expectations on completion.

It was hoped that a consistent personal identifier would assist tracking individual students through the year while retaining anonymity to enable tentative hypotheses about expectations and attitudes to courses to be considered.
Number of participants returning questionnaires.

The findings from the SEDL questionnaire sent out in the weeks before students started their courses indicated that students were prepared to complete it and return it with a general response rate was between 60-70%. The ERM questionnaire sent out 8-10 weeks after course had started, achieved approximately a 50% return from those that returned SEDL and the least successful was the EVAL questionnaire with around a 25% return from those that returned SEDL.

Initial Findings

The most common expectations from SEDL were:

1. Most students expected to initiate contact with their tutor
2. Most students working at a distance expected contact about once a month
3. Books were expected to be the most important learning resource
4. The telephone was expected to be the main medium for communication
5. Expectations of assignment feedback differed between the 4 countries.

Important issues arising from year one (1998/99)

• Students had different range of experience of ODL and this may explain differences in their expectations
• Students in different cultures had different reasons for taking courses which could explain differences in student motivation.
• Project was ambitious but succeeded in creating, sending out and analysing 3 questionnaires measuring student expectations in 4 different languages in 4 different countries in one academic year.
• Interesting data were collected with some differences noted between groups.
• The use of closed questions with pre-specified response options were relatively successful but could be improved upon.
• Tracking students’ responses through the course proved very difficult and sometimes impossible.
• The SEDL and ERM questionnaires produced return rates that exceeded most post-course questionnaire return rates. The low EVAL questionnaire return rate suggests that a final end of course questionnaire is probably not necessary in establishing student concerns about the course which occur most importantly before, and soon after, the course starts.
• A review of the findings of year one concluded that because different countries deliver ODL in different ways further work was necessary to find the types of questions that fitted each country’s students’ experience more appropriately.

CEESOC Project year two (1999/2000)

The second year of the project has seen the team working further at producing valid and reliable questions in questionnaires that could be used for intra and inter-cultural comparisons. This involved developing the expectation questionnaires used in the first year and employing more open questions to check the range of student responses. The team concentrated on questions that could elicit "student expectations of ODL support " before a course starts and provide a "satisfaction with ODL course support" after about 8 weeks, matched against initial expectation. This process, the team realised, could become a quality assurance mechanism that an institution could adopt to help tutors support students more effectively. Using this simple process at tutor level, students’ support needs could be identified and tutors armed with the knowledge of what their students expected could support students in line with those expectations. Students who express inappropriate expectations of tutor support can be identified and more appropriate style of learning or more appropriate expectations of tutor support explained to them.

To establish the range of answers that exist and are meaningful to the students, we adapted the questionnaires used in the first year and used more open questions. This helped reveal student expectations better and we were also able to check the validity of the question style through selected student interviews.

Outcomes of the research

This project has shown that students do come to ODL with expectations, hopes and fears about frequency of contact with tutor, type of support that will be provided, style of teaching they will receive. Whilst it is difficult to know exactly why students drop out of courses we feel that unmet expectations of teaching and tutor support have an important effect. The expectations of tutor support that students bring to ODL is valuable information and we suggest that this information can be collected effectively and efficiently by institutions using the questionnaire styles that we have devised working to the model outlined below.

Figure 1: Model of quality assurance process using student expectations of tutor support needs

Products

The Ceesoc team has agreed to prepare the materials that should allow institutions to adopt this model. The materials will consist of a user pack containing a manual that explains the rationale for working with and responding to student expectations of tutor support. Examples of the process for developing the questionnaires will be included along with sample questionnaires that the team have found useful. Because ODL is delivered differently by institutions there can be no absolute set of tutor support questions.

There is however, a common need for tutor support. But the questions needed to reveal what those expectations are will be different from institution to institution and even course to course within an institution. The team feel that institutions or tutors using the pack must investigate the way their courses are delivered and then devise their own institutional questions that will make sense to students and provide valuable information to tutors. The manual will direct the reader to appropriate literature if more information is needed.

Value of the Expectations Driven Quality Assurance Process to students, tutors and ODL institutions

Working with student expectations of their needs for tutor support can help institutions improve the quality of the service they provide for the current cohort of students. Post-course quality assurance models tend to provide little advantage to the students who have provided the feedback on how tutor support could be improved. This may explain the willingness of students to complete pre-course questionnaires and reluctance of students to complete post-course questionnaires. Some examples of how this quality assurance system could assist ODL institutions are:

• New providers of ODL could use the process to establish what their customers expect. This could help them plan the type of support they need to provide.
• New tutors in an established ODL setting could be assisted in preparing themselves through having an awareness of what their students expect and through using the expectation-match questionnaire establish whether what the tutor provides is meeting the current cohort of students’ expectations.
• An ODL institution could use the process as a developmental tool to improve staff awareness of student needs and also a measure of how successful staff were at meeting those needs.
• Institutions might also see the process as particularly useful to investigate what expectations students have in areas of provision that have high drop out rates of particular types of students.

The importance of organisational partnership

Whilst the process it relatively straightforward there is still a crucially important "extra ingredient" that is necessary for the system to operate smoothly. We anticipate that there needs to be a culture of partnership between course managers, course tutors and course students where support and development are the primary shared focus. We recognise that the system to work well needs to be housed within an organisational culture that supports it.

Conclusions

Student expectations of tutor support are difficult to compare across culturally different institutions that employ very different methods of conducting ODL. Nevertheless individual institutions and individual tutors within those institutions can benefit from researching and reflecting upon their students’ expectations. We believe that the system we have outlined here for accessing and responding to student expectations is an important and valuable outcome of this study. We also believe that the appropriate use of the questionnaires operating in a professional supportive environment can assist in improving the quality of learning and tutorial support provided by ODL institutions (and experienced by ODL students) throughout Europe. We hope that further work using the system suggested here will demonstrate the value of this approach.

For opportunity to view the process materials as they are developed see CEESOC Web page: http://www.norrk.ssv.se/projekt/ceesoc/

27/11/00 (article submitted June 2000)