Access, Retention and Course Choice in Online, Open and Distance Learning
Ormond Simpson
(o.p.simpson@open.ac.uk)
The Open University UK, The Open University in the East of England, Cambridge CB2 1PF
This paper was shortlisted for the 'Best paper award' at the
Third EDEN Research Workshop, Oldenburg, Germany, 2004
Abstract
Course choice is an important part of the progression of transforming a potential student into a successful undergraduate. Yet the processes and materials of course choice have attracted little attention, it being assumed that the activity will involve an adviser discussing choices and directions with a potential students using course descriptions. Such discussions are expensive and there is evidence that potential students do not always take advice. In any case vulnerable students are often the least able to access guidance and tension between the recruitment and retention functions of course descriptions may make a course appear more accessible than it is.
This paper argues that course descriptions are inadequate to describe a course and that other materials are needed. These other materials are diagnostic, preview or 'taster' materials and students' views. Such materials have a vital role to play in ensuring that students get onto the right course for them thereby increasing their chances of retention. Courses will need such a 'set of competing perspectives' to describe them to any degree of accuracy.
There is also evidence that such materials have a demystifying effect on potential students' perception of courses, enhancing their self confidence in the possibility of their ultimate success
Keywords
Retention, course choice, students' opinions, diagnostic materials, course descriptions, costs and benefits
Introduction
Somewhere between a potential student's decision that they wish to re-enter learning and their transformation into a successful learner lies the process of course choice. It is a widely held view amongst advice and guidance workers that that process is an important factor in students' subsequent retention or dropout - see for example McGivney (1996). A student making an inappropriate choice - either the wrong level of course for them or the wrong content - is thought much more likely to dropout than a student who is on a course for which they are suited by both level and content.
There is considerable supporting evidence with respect to full time students where course choice has been found to be a very important cause of dropout - for example Yorke (1999). More recently Gibson and Walters (2002) identified inappropriate course choice as one of the four main reasons for dropout after access courses.
Amongst distance education students the clearest evidence comes from the annual survey of withdrawn students in the OU UK Institute of Educational Technology (2002). In this survey 'inadequate course choice guidance' is the second most chosen item after 'insufficient time' as reason for withdrawn students' greatest dissatisfaction - some 21% are dissatisfied with this aspect of their studies.
Making course choice decisions
There does not seem to have been much investigation into how potential students make their course choice. Clearly there will be students who have a clear idea of both the subject and level of their intended study and who will go on to study successfully. However there will also be students who have a clear choice which is not in fact suitable for them and students who will have much less idea of what they want. Initially such students will be relying on the course title and description for their decision.
Course titles and descriptions
All institutions provide course titles and descriptions to help potential students. But there are a number of issues around such titles and descriptions:
- Length. If descriptions are short then they are likely to be incomplete
in some respects or at least to lack comprehensiveness especially in the case
of long courses covering considerable ground. If course descriptions are long
then they may be relatively impenetrable or fail to give a view of the wood
for the trees.
- Vocabulary. The vocabulary available to course description writers may be
inherently imprecise. Authors often use terms such as 'explore', 'describe',
'advanced', 'elementary' and so on which allow for wide interpretations by
the intending student. Even where the descriptions are written in terms of
outcomes these may not be clear to students who will not necessarily understand
an outcome stated in terms they have yet to learn. It may be little help knowing
that a course will give a 'thorough understanding of vector algebra' if students
have only a very hazy idea as to what vector algebra is.
- Conflict between recruitment and retention (Simpson 2003). A course description
may be written with the subsidiary aim of encouraging recruitment onto a course.
In such cases there may be a conscious or unconscious temptation to emphasise
the accessibility of the course at the expense of its difficulty. This is
not to suggest any dishonest intentions on the part of the writers: merely
that there is a fine line between gently encouraging reasonably well-qualified
but un-self-confident students to take a course and allowing less well-qualified
students get the impression that a course is easier then it actually is.
- Assumed entry behaviour. Course descriptions can specifically address potential
students' entry requirements both in terms of the previous knowledge assumed
and the skills required and this may be one of their most valuable functions.
However, knowing for example that a course assumes that potential students
have an 'A' level in sociology does not tell them that the course is definitely
suitable for them nor does it help students in an open learning environment
who do not have that qualification but who might be able to study the course.
It may not be enough therefore to rely on course descriptions - however long and carefully written - to ensure that students have the best possible view of a course before deciding to take it. There may need to be other elements involved. Some possibilities are course choice guidance, students' comments on courses, preview and diagnostic materials.
Course choice guidance
The most obvious route to ensuring that students get the right course for them is to offer them advice from a course choice adviser. However again there are a number of issues around this approach:
- One to one advice is expensive for an institution to provide, particularly
in mass distance education. For example the OU UK has 35000 new students each
year and a staff of around only 35 trained advisers. So it is inevitable that
most new students will not be in a position to speak to an adviser at any
length if at all.
- Potential students may not find it easy to frame the questions they should
be asking about a course. A skilled adviser should know how to draw out the
appropriate questions and answers from a student about their needs and intentions
but this can be a lengthy process.
- Advisers may have the generic skills to advise on course choice but not
necessarily the detailed knowledge of a particular course to answer all the
possible questions that might be asked. In that case a process of referral
or information gathering may be needed which will add to the cost and length
of the process.
- Students in distance education often appear reluctant to seek advice. For
example the proportion of students seeking course choice advice from an advisor
by any route in the UKOU is probably only about 10% of the students choosing
courses in any one year. It can be argued that this may be a reflection of
the need but it may be more linked to the scarcity of advisers noted above.
It may also be connected to the reported level of dissatisfaction with course
advice from withdrawn students and the link with dropout already cited.
- There is evidence that even where students receive advice that they do not
put it into effect. Johnson (2000) reported on a follow-up of advice given
to a number of OU UK students whose course choice appeared to be unsuitable
for them. Although the advice had the effect of getting students to reduce
the number of courses they were taking it did not usually make them change
their course choice. Once having made a choice students were clearly committed
to that choice. This is hardly a new phenomenon - as Lord Chesterfield remarked
of advice 'Those who want it the most always like it the least'.
- Finally access to guidance may be difficult for some students - as Clayton and Mcgill (2000) note those who could most benefit from guidance are often the least able to access it for reasons of remoteness, unassertivness and so on.
Attention has therefore turned to alternative methods of providing course choice advice 'reflexively' - that is a potential student 'self-advises' as a result of working through different kinds of materials. There are a number of different methods.
Students' Comments on courses
It seems clear that potential students value the views of students who have taken courses previously in much the same way that people buying a car will seek the views of other people on the cars they drive. This is not to say that students will be guided entirely by such views but that they are likely to be factors in students' decisions.
There are a number of ways in which students views can be made available. In the OU UK a selection of students who had recently taken courses were invited to write a short note (100-200 words) aimed at advising new students considering taking the course they had just finished. These were lightly edited (largely to remove comments on aspects of the course such as tuition which would not be common to all students) and collated into single sets for each course.
The comments are now supplied both as hard copy and on the web - www.open.ac.uk/courseviews. Each of about 150 undergraduate courses has a set of comments - usually containing about ten or so individual comments from individual students thereby giving a number of alternative perspectives on the course. It is hard to select typical comments from amongst so many but these are two typical comments [out of ten] for course A210 'Approaching literature, authors, readers and texts':
"A highly rewarding and stimulating course, but demanding in its reading requirement. An open mind is needed to make the most of the course. Those who resist the course's call to approach and explore literature in a variety of ways waste energy and miss out on some of its riches. Those who tackle the course in a positive way will find much that is both challenging and enjoyable, and have their literary horizons permanently widened." "There is a very heavy workload with this course and a great number of texts; not only the set books, excellent and interesting though they are - but a thesis within the Readers and the Supplement. The Study Guides were on the whole helpful, although the first one seemed rather out of synchrony with the Reader and it was not clear where one stopped and the other took over. However, the range of texts was interesting, and the video performances thought-provoking. I would advise anyone taking this course to read all the texts before you start (except the anthology as only a few works are referred to in this) because it is nearly impossible to do as the guides suggest, to study the next text while completing work on the previous one and prepare the current assignment all at the same time." |
There were initially some concerns that comments might be overly critical, negative or simply unfair. This has not turned out to be the case at all. There are negative comments but they are almost always in the context of a balanced opinion. For example it is not at all unusual for students to criticise the work load of a course as very heavy but to add in the same breath that providing one can handle that this is an exciting and worthwhile course. Indeed the overall tenor of the comments is usually very positive whilst realistic about the challenges that the courses represent.
Evaluating the Students' Comments
It is hard to assess the effectiveness of the Students' Comments in terms of ensuring that students make the best course choice for themselves. There are various evaluations, however:
Site usage
The site has steadily increased in popularity and 'hits' are running at around 850 a day.
Student and tutor feedback
The site allows students and tutors to send feedback. This is consistently very positive:
Examples of feedback to the Student comments website: I found this website an excellent resource. I'll be referring other students to it as well. It's particularly useful for those who are undecided on their course of study. As a K301 tutor I think this website is a great idea; I shall refer current and prospective K301 students to it. An excellent idea! I wish it had been in place last autumn, it would have made my choice much easier I absolutely love this page and find it the most useful and interesting way to get a really good feel for the course and what is on offer. I shall soon compose and send my thoughts on a wonderful A103 year. Great website, I'M going to be starting DipSW in Feb 2003 , it would be nice to see course comments added on these type of courses to give us students like other students a feel for it before we start!!!!!!! |
Interestingly, some of the comments suggest that students are using the site for preparatory advice as well as for course choice. This may mean that the Students' Comments have a particularly important role to play for new students.
Students Comments - the political dimension
This has always been a difficult project for the OU UK as - in a typical recruitment versus retention dichotomy - course teams are uneasy with the idea that students' comments on their courses can be seen so widely. Indeed in order to protect the project the originator arranged that it should come under the aegis of the Open University Students Asspcaotion. Nonetheless the comments are still not linked from any one of the OU UK's official course information web pages presumably because of uneasiness that such linking would provide an official imprimatur.
However, the political environment is changing. Last year the Higher Education Funding Council for England and Wales produced a report ['Information on quality and standards in higher education' HEFCE 02/15 March 2002 informally known as the Cooke Report] which recommended that universities in the UK be required to publish information on the quality and standards of their teaching and learning. The information published should include the outcomes of student satisfaction surveys. This in turn was based to some extent on Australian practice which requires universities to place student feedback on the Web.
Subsequent to the report the HEFCE issued a circular to all UK universities in which it announced its intention to pilot a similar activity in the UK which would include student feedback ['Information on quality and standards: pilot of recommendations from the Cooke Report' HEFCE 04/2003, 7 February 2003]. There have been some problems with the pilot particularly the problem apparent from Australia that students' response rates there were sometimes too small - some courses having received only one or two comments thus potentially distorting the judgements. It is not clear how many comments are needed to give a fair judgement but the average number of about 10 comments on each course on the Students' Comments may be a good balance between fairness and overkill.
At the moment of writing this article it is not clear what HEFCE will do in response to these issues but it seems likely that it will press ahead in some way as the developments are clearly important to the government's drive to raise the quality of HE in the UK through consumer pressure.
Students' Comments - the costs
To some extent student comments become now self generating as sites can be designed to allow students to send their own comments to a designated email address. They can then be edited and placed on the site. For the 200 or so courses on the OU UKwebsite the annual costs amount to only about L1500 pa. There is therefore a powerful case both in student support political and financial terms to maintain and develop the Students Comments.
Students Comments - live
The use of computer conferencing can allow potential students to ask questions of experienced students directly on line. There is an interesting example of this activity on the Bart's Medical School Student site which hosts an admissions conference where 'A' level students can post questions which are answered by current medical students.
From Little-Samantha Status: Final Year School pupil (Y11) Hi everyone, My boyfriend is interested in medicine, but seems to have a little phobia of the good ol' red stuff. Small amounts are ok (I think), but he watched a surgery programme on TV once and felt sick to his stomach. So to those med students (or anyone with work experience): have you ever felt a little squeamish, and how have you dealt with it? Is it something you all just get used to? Thanks a lot. P.S. exams over with now. Just a long wait until August 22nd. ;-) From tig Status: medical student (yr-3) I can't watch surgery on tv..... in real life.. fine! on tv everything is distorted and I am very sqeamish! From azy_cool Status: 1st Year Medical Student at Manchester I was a little squeamish before I started, when looking at surgery on
tv etc, but I used to watch it cos it was intresting and it got 'easier'
to watch the more you watch it .. so he can try that, Operation comes
on the Discovery channel most nights I think? In medicine having to cut
up dead bodies .. that does sound quite bad, but really it was fine, even
the first lesson was fine, |
The OU UK has made similar attempts to use its 'FirstClass' conferencing facility to run a course choice discussion but with less success (Simpson, 2002). This may be due to the fact that there are several thousand conferences on the facility and students have little time to explore them all.
Students comments - the limitations
Students' comments are of course limited in value. They cannot tell students that they have the right background for a course and no matter how independent they will always represent partial views of courses.
Course Preview Materials - 'Taster Packs'
Taster packs have a long history in distance education and early examples were produced by the National Extension College in the early 1980's. They were introduced by the author of this article in the UKOU in 1986 for the then foundation courses.
The original rationale was a simple one: students considering a particular course should have the opportunity of surveying the materials and undertaking a short 'test drive' of that course. The reasons for using specifically designed materials for the purpose were two-fold:
- Cost - in theory in distance education all the course materials are available in written form there should be no barrier to potential students browsing through an entire course at leisure to see if it suits them. In practice the cost of sending an entire course would clearly be prohibitive and making them available for loan in some way is impossible in view of the number of potential students for any given course. Even attempts to make them available through public libraries are utterly inadequate given the scale of the need.
- Typicality - a complete OU UK course can consist of 30,000 words or more. Even if potential students could get their hands on a course it is clearly unfeasible to do anything more than read through a few of the pages of one of its constituent units. But that raises the problem that individual units may be untypical of the course as a whole.
It seemed better therefore to select short samples of the course which were thought to be reasonably typical of the course itself and capable of being cheaply reproduced in quantity. Whilst doing that it was also easy to add typical examples of assessment materials such as an assignment with a tutor's comments and grade and a specimen exam paper in order to give students some kind of feel for the level of work they would have to produce at some point in the course.
Originally it was intended to address the issue of the amount of time needed for study [withdrawn students' highest dissatisfaction factor] by ensuring that each pack was about an hour's worth of study. Over the years this has become a difficult criterion to fulfil but it still remains an attempt to give potential students a realistic sense of how long study will take them.
Once again there were recruitment versus retention concerns. For example it was thought that such packs would deter students from studying a course. In order to deal with this each pack came with an explanation of the pack's purpose and a 'health warning' pointing out that the pack was typically less the 0.5% of the course, that it might contain material that relied on knowledge gained earlier in the course and so on.
Of course clearly one of the intentions of such packs is to deter students from taking a course if either the content is not relevant to their needs or at the wrong level. There would be cause for concern if it was found to be deterring students from a course that was in fact right for them in both respects. In fact Adams Rand and Simpson (1989) found no evidence for that belief and student feedback was uniformly positive - see below.
Student feedback on taster packs
These are selected at random from several hundred comments:
Interesting, informative. Challenges ahead thoroughly paraded. Thank you for the health warning! Excellent resource, certainly helped me in making the right decision for my future. I found the taster pack both stimulating and interesting. I now definitely want to study A103 Very helpful - assisted in my decision to go ahead with this course this year. This was very helpful and I shall enrol next year. Couldn't see how it could be improved except by larger print. I struggled with this but am visually impaired. Many thanks for the taster pack. I found it very interesting, giving an insight as to the work involved. Enough to prepare - enough to scare! |
It is always difficult to evaluate such feedback as it may be partial with students who have negative responses not bothering to feed those back. But the uniformity of response is remarkable and it seems safe to assume that at the very least there are few negative perceptions. The feedback also appears to reveal that potential students find the packs reassuring rather than off-putting. It's not hard to see why this might be so. To a potential student a course represents an intimidating unknown and there must always be the question in many students' minds - 'can I cope?' To see a typical selection from a course and realise that although it may be challenging it is not completely unintelligible must help overcome the fear of that unknown.
Course Preview materials - the cost
One of the attractions of preview materials is that because they are derived from existing course materials they are inexpensive to produce. There are costs in deciding what selection of material should be used but these are one-off costs which can be amortised against the life of the course. Thus in the OU UK the editing cost of producing a Taster Pack for a course is around L100 and the reproduction costs are around L1 per pack. Since the course enrolment for such courses can be of the order of several thousand students who will be paying fees of several hundred pounds Taster Packs may represent a cost-effective reflexive advisory method. But further work is needed.
Course Preview materials - the limitations
Like the Students' Comments preview materials cannot tell students that they have the right background knowledge for a course and are limited in that respect.
Diagnostic materials
Diagnostic materials have long been used in open and distance learning to advise students about their study choices. There are essentially two kinds - generic, to test applicants' suitability for higher education and course-specific, to test suitability for a particular course. Both can be either externally- or self-assessed. In addition it is possible to argue that diagnostic materials can assess either skills or knowledge although it is not clear how easy it is to make this distinction in practice.
- Generic externally assessed diagnostic materials range from the massive externally assessed 194 point questionnaires such as the Noelle-Levitz test used in the US to identify vulnerable students, to the simpler test devised by Johnston at Napier University (described in Simpson 2003) which is used in conjunction with an interview with a student's tutor.
- Generic self- assessed diagnostic materials have been less used. An example is given in Simpson (2003) of a test developed from a statistical analysis of the characteristics of successful UKOU students in order to help potential students work out their predicted probability of success and see if they can change those characteristics [such as low entry qualifications] which could increase their chances of success. But this has not been evaluated.
- Course-specific externally assessed materials where a potential student takes a test which is assessed by a member of the institution are relatively simple to apply although for Open Learning institutions they may resemble entrance exams too closely for comfort. The will also be expensive and in trials in the OU UK where they were voluntary it quickly became clear that the potential students who submitted them were the students least likely to need to do so (Ashley 1986).
- Course-specific self-assessed materials appear to be more suited to mass
distance education. However there are still difficult issues. It is relatively
easy to design self-assessed diagnostic tests where the answers are clearly
correct or incorrect as in maths science and technology subjects. It may also
be straightforward to design tests where clear answers can be given such as
in language quizzes where for example simple 'cloze' tests (replacing blank
spaces in sentences with appropriate words) can be used. It becomes more difficult
where more advanced courses with some background knowledge and skills are
needed such as such as courses in higher level arts or social science subjects.
It is also possible that a 'recruitment versus retention' dichotomy can occur. It can be hard for a diagnostic test writer to give 'you shouldn't be taking this course' as a possible outcome when there is pressure to recruit students onto a course. There will be a temptation to argue that a diagnostic test result should be less directive and leave the door open to students by saying something along the lines that 'you will find this course a challenge unless you prepare for it carefully' instead. This may not be as helpful to a potential student. In one of the few evaluations of course-specific self-assessed materials Williams (2001) found that UKOU science diagnostic quizzes which used such rubrics were not as effective as she hoped in persuading students to change their course choice. But that finding may well be specific to the quizzes concerned.
Whatever the type of diagnostic material there appears to have been little work published on their effectiveness in getting students onto the correct courses. And diagnostic materials may not be sufficient in themselves as whilst they may tell a student that he or she is at the right level for a course they may not tell that students that the course has the right content for them.
Conclusion
Thus all the methods described here for course choice have their limitations either in terms of their cost or the partial view of courses that they provide. Perhaps the best route to a satisfactory course choice system is to borrow a concept from social science - that any reality can only be fairly represented by sets of 'competing perspectives' and that a concept as complex as a course needs all the perspectives of descriptions, previews, comments and diagnostic materials in order to describe it completely. How students will react to such a package of perspectives and whether it will be too much for any one student or whether different students will choose those perspectives that give them the best 'feel' for a course remains for further research.
References
Adams, C. Rand, V. and Simpson, O. (1989) 'But what's it really like? - the Taster Pack Idea' Open Learning 4 (3)
Ashley, C. (1988) 'Testing the Water' internal report Open University in East Anglia Clayton, P. and Mcgill, P. (2000) Access issues in adult vocational guidance and counselling for people at risk of social exclusion, Journal of Access and Credit Studies 2 (1) pp 4-17 Johnson, M. (2000, unpublished) Course Choice Advice Open University in London internal report.
Gibson, K. and Walters, J. (2002) Access and guidance in tutor-student relationships Journal of Access and Credit Studies 4 (1) pp 20-31
McGivney V. (1996) 'Staying or Leaving the Course' National Institute of Adult Continuing Education, Leicester, UK.
Simpson, O. (2003) Student Retention in Online Open and Distance Learning Kogan Page, London
Simpson (2002) Supporting Students in Online Open and Distance Learning (2nd edition) Kogan Page London
Student Research Department (2002), Withdrawal survey UKOU Institute of Educational Technology,
Williams, R. (2001) Science diagnostic quizzes in the UKOU, Internal report UKOU Science Faculty
Yorke M. (1999) 'Leaving Early' Falmer Press London.