Does embedding support devices have an effect in independent learning?
Author
dr. Rob Martens
Open
Universiteit Nederland, Expertise Centre for Educational Technology, P.O. Box 2960,
6401 DL Heerlen, the Netherlands
© 1998
Introduction
Research overview
Conclusions and discussion
References
Abstract
Traditional universities use ICT to improve their education. This use is often accompanied by more components. How to make study materials suitable for self study is an important issue. In electronic and printed learning environments of lot of support is build in for this purpose. Think of advance organisers, questions with feedback, examples and exercises. In electronic learning environments more support can be implemented in a flexible and interactive way. But do students really use these embedded support devices (esds)? How do they appreciate them? Does instructional material with esds lead to better study results? In other words: do esds really fulfil the high expectancies of course developers? In varying contexts different research methods were used ranging from questionnaires to eye movement registrations. In this article the intention is not to give a detailed research description but to present a helicopter view on these investigations and the results. Most researches were conducted at the Open Universiteit Nederland . The main conclusion is that esds are used at a deep level by most students, are highly appreciated and that the use of esds leads to better study results. The optimal adaptation of esds to individual student characteristics remains one of the main issues to be resolved.
A full version of this article can be downloaded by clicking here.
Keywords
embedded support devices, self study, course materials, research overview
Introduction
Self-study is becoming increasingly important, and it is a form of education which has many advantages. Crucial is the rise of information and communications technology (ict). Affordable, powerful computers, the increasing standardisation of platforms and the rise of Internet are all highly important factors which are changing society and which require a different approach to education. Distance-teaching universities have traditionally embraced this type of ict because it helps solving some of the specific problems inherent to distance education.
Alongside its advantages, distance education obviously has disadvantages as well. Important ones are the relative lack of support, guidance and interactivity, the fact that course material is often static and is not tailored to meet the needs of the users, and the lack of interim adjustments to take account of what students actually do. Such problems may express themselves in the form of students’ falling behind in their studies or dropping out. Distance education attempts to solve these problems by making use of embedded support devices (esds) in (written and electronic) course material. These devices consist of a whole spectrum of additions such as:
- examples (e.g. ‘The economic theory on the stock exchange stated above, can be illustrated by a description of black Monday, which was a…’)
- other changes in the lay out (such as bold or underlined text) to emphasise certain concepts
- study tips
- advance organisers (e.g. ‘Your next task will be to explore the theory mentioned above’),
- exercises
In electronic learning environments more possibilities become available, such as:
- simulations
- hypertext
- direct links (e.g. OLE) to other applications such as the statistical package SPSS© in a statistics course
All these devices are in fact an attempt to replace the teacher at the front of the classroom. Two examples of printed and electronic esds are presented below (figure 1).
figure 1 Page with ESDs from an
OUN course
In an electronic version these esds may look like the one (in Dutch) under ‘voorbeeld’ in figure 2.
figure 2 ESD in an electronic
learning environment
Much of this research was done in the 70s and 80s in investigations that were primarily aimed at reading comprehension. A large quantity of literature and some reviews (Montague & Knirk, 1993) evidence positive effects of making use of esds in study texts. Many esds seem to have to do something with the coherence in a text, making explicit the -hidden- relations in a discourse (e.g., Lorch & Lorch, 1985). Many educationalists recommended the use of esds in self study materials (Rowntree, 1990; Gonzales, 1997).
Much as these esds are used, however, very little is known about how they actually function and what their effects are. In distance education, it is naturally the students themselves who decide whether or not they are going to use esds, and it is uncertain whether they actually do. Much of the existing research is based on relatively isolated esds (Moore, 1995), often not in a specific self-study context (Marland & Patching, 1992; Wade & Trathen, 1989). Also, individual differences between students must be taken into account. A device that works for one student may simply be wrong for another.
So, esds play a vital role in distance education and are generously implemented in course material, for example produced by the Open University of the Netherlands (OUN). Approximately 40% of the course material of the OUN consists of esds. A lot of effort is put in the implementation of these esds, and they almost double the length of course materials. But there is still quite a bit of confusion about their use, functions and effects. Many publications have pointed out the need for further research (e.g., Marland, Patching, Putt & Putt, 1990; Lockwood, 1995, a).
The considerations made above, lead to the main question: what are the functions and effects of esds, and do students use and appreciate them (Martens, 1998, a,b).
Research overview
In order to arrive at results which can be generalised, it would not do to select a single research method within a single educational context. Each research method has its specific advantages and disadvantages. Interviews, for example, may produce a wealth of information, but they require introspection from students about their own studying process. The validity and reliability of someone’s introspection about their own studying process may be debatable. Some methods generate a lot of information but also take a lot of time; others have precisely the reverse problem. One important difficulty is also that students in experimental contexts often do not behave the way they would in real-life contexts: studying to pass a class, with the final examination furnishing the guideline for their studies. In other words, in actual practice effects are sometimes hard to detect because students will compensate for poorer conditions by making a greater effort. There is a need to combine research in various educational contexts, using different methods and course material from a variety of domains. All these researches have to be considered together, which has been done in detail in Martens (1998, a).
In the research projects which focus on esds in printed instructional material, use is made of interviews, recording of eye movement, measuring the effects of various course versions (with or without esds) and written questionnaires (Martens et al, 1996). This combination is intended to give a better understanding of how esds are used, how much they are appreciated by students, and the effects and functions which are ascribed to them. Because the research was carried out in different research contexts (distance-teaching universities and ‘conventional’ universities) and covers different subject domains, an attempt has been made to reach conclusions which are generally applicable. Briefly, these conclusions are that esds have a positive impact when added to course material. Students make frequent use of them, usually at a deep level (focusing on understanding).
It also appears that students are appreciative of esds and that using them improves study outcomes, particularly when it comes to insight questions. Because the average use of such devices is high, however, it is difficult to explore the correlation with individual student characteristics. The high level of use seems also to indicate that esds do not really take account of differences between students, despite the great diversity of students at the OUN, for example. Where differences between students have become evident, for example in their use of esds, these differences sometimes deviate from what the designers had intended. For example, students who have a great deal of prior knowledge make more use of esds than students who have little prior knowledge, contrary to what the designers had imagined. It is also striking that, although course material with esds is an average of 40% longer than non-esd course material, this does not increase the amount of studying time significantly. In figure 3 an example is shown from a registration of the eye movements of a student, studying a text (two pages) from a course with esds. The large rectangles are text parts containing esds or basic content. The object in the top right corner is a diagram. The black dots are fixation points and the lines are the saccades from one fixation to another. This figure shows that it is possible to detect how much use is being made of certain parts in a course. It is an illustration of the high level of use of esds.
figure 3 pattern of eye movements during
studying two pages of a law course
Another research project looks at an educational medium which, in addition to many other advantages, is above all capable of responding to individual differences between students: interactive learning environments. In such environments it is easier, both in technical and in practical terms, to take differences between students into account. For example, it is possible to draft problems which are tailored to specific students. This research (n=502) with experimental interactive learning environments was carried out at a mainstream university, with students studying for a conventional statistics examination. The setting meant that it was not possible to vary the actual contents of the course material (e.g. quantity or type of esd), as this could be to the disadvantage of some of the students. Instead, there was variation in the form in which the material was presented within the self-study conditions. The two forms used were written and electronic material. There was also material in which the layout of the basic text was distinguishable from that of the esds, and conditions in which the subject matter was simply presented as an uninterrupted text. A post-test revealed that there were no differences in terms of performance between the various conditions and between the experimental and the control conditions. The control condition was the ‘mainstream’, traditional lecture, which is supposed to be replaced by self-study for educational-economic reasons amongst others. The students’ own assessment about the consumability of the material was significantly more positive in the self-study conditions. There were also significant interaction effects. Students who made generous use of esds (measured in log files) in the electronic conditions earned significantly higher post-test scores when the esds were indistinguishable and vice versa. One such an interaction effect is depicted in figure 4.
figure 4 post test scores for
discernability * use of esds
These interaction effects are an important indication of how electronic learning environments can serve to tailor esds to student characteristics measured on line (Martens, Valcke & Portier, 1997).
The investigations described above only studied the effects of esds when used in combination. The literature study, however, reveals a type of esd which many authors feel is the most important of all for self-study: tests and problems with feedback. An effect study (n=1482) (Martens & Dochy, 1997) carried out at the OUN looked at the effects of this specific support component. The experiment involved forwarding prior-knowledge tests (one per course) or progress tests (three per course) to regular students enrolled in four different OUN courses. The students were allowed to complete the tests at home and return them. An computerised procedure then sent them feedback on their outcomes, along with studying instructions. A control group was only given a written questionnaire with questions about their studying process. It was discovered that the experimental and control groups did not differ much in terms of performance, pace of study and studying process. In other words, the effects on studying behaviour and student motivation of a single, isolated form of esd is limited. Students made frequent use of the esd (voluntarily) and were highly appreciative of it. The progress tests were given higher marks than the prior-knowledge tests.
Conclusions and discussion
Many researchers gave argued for more research (e.g., Lockwood, 1995, b), while others have doubts about research in this field (e.g., Moore, 1995; Lockwood, 1992; Marland & Patching, 1992) or have concluded that esds do not have the intended effects (Vermunt, 1991; Marland & Patching, 1992). Others on the other hand recommended the use of esds.
Looking at the studies that were described above as a whole, it appears that they are quite well in line and the conclusion is that esds make an important contribution to self-study, and in that sense are an important form of support in this type of education, as they generally have a demonstrably positive effect on study outcomes and are given a positive score by students. Students do use esds frequently, usually at a deep level. On the grounds of these results, recommendations can be made. The most important is to use and instruct students to use esds. Esds should incorporate individual differences between students. For this purpose the Mercator package was designed. Detailed information about Mercator can be found on http://mercatoruser.spc.nl/englishIndex.asp?page=englishMain and http://www.ouh.nl/open/mva/mercatoren.htm
This system was designed for the production and delivery of flexible electronic course material can be very useful (Valcke & Martens, 1997). Student experiences with this approach are also positive (Martens, Valcke, Portier, Weges & Poelmans, 1997). A detailed description of Mercator is beyond the scope of this text. A screen dump is depicted in figure 5.
figure 5; screen display in Mercator 3.0
The results of the investigations described here have been considered when designing the Mercator-system.
In general, the increasing availability of ict in (higher) education has generated a significant move in distance as well as in ‘conventional’ higher education (these two resemble increasingly) towards powerful and realistic esds and towards a more profound integration of teaching and assessment. The demand for guidelines and instructions on how best to tailor flexible course material and esds in particular to student characteristics, will increase.
This is in a line with a general trend in higher education (Valcke & Martens, 1998), There is a clear development in distance education, which is referred to by some authors as ‘third generation distance education’. Key characteristics of this new approach are: demand driven, flexible, multi-media and network-based, competency-oriented, integrated assessment approaches and collaborative learning. This new approach was predicted by authors, such as e.g., Bates (1995) when he described the emergence of 'electronic universities'.
References
Bates, A.W. (1995). Technology, open learning and distance education. London: Routledge.
Gonzales, C.I. (1997). The learner and learning supports. Paper presented on ‘the new learning Environment’. The 18th ICDE conference, The Pennsylvania State University, June, 1997.
Lockwood, F. (1992). Activities in self-instructional texts. London, Philadelphia: Kogan Page.
Lockwood, F. (1995, a). A cost benefit model to describe the perception and use of activities in self-instructional texts. European Journal of Psychology of Education, 10, 145-152.
Lockwood, F. (1995, b). Students’ perception of, and response to, formative and summative assessment material. In: F. Lockwood (Ed.), Open and Distance Learning Today (pp. 197-207). London: Routledge.
Lorch, R.F., & Lorch, E.P. (1985). Topic structure representation and recall. Journal of Educational Psychology, 2, 137-148.
Marland, P., Patching, W., Putt, I., & Putt, R. (1990). Distance learners' interactions with text while studying. Distance Education, 1, 71-91.
Marland, P., Patching, W., Putt, I., & Putt, R. (1992). Thinking while studying: a process tracing study of distance learners. Distance Education, 13, 193-217.
Marland, P.W., & Patching, W.G. (1992). Learning from text: glimpses inside the minds of learners. Townsville: James Cook University of North Queensland.
Martens, R., & Dochy, F. (1997). Assessment and feedback as student support devices. Studies in educational evaluation, 23, 257-273.
Martens, R., Valcke, M., Poelmans, P., & Daal, M. (1996). Functions, use and effects of embedded support devices in printed distance learning materials. Learning and Instruction, 6, 77-93.
Martens, R., Valcke, M., Portier, S., Weges, H., & Poelmans, P. (1997). Research with interactive learning environments in three content domains: descriptive statistics, continuous mathematics & substantive criminal law. Distance Education, 18, 44-58.
Martens, R., Valcke, M.M.A., & Portier, S.J. (1997). Interactive learning environments to support independent learning: the impact of discernability of embedded support devices. Computers & Education, 28, 185-197.
Martens, R.L. (1998, a). The use and effects of embedded support devices in independent learning. Ph.D. Thesis. Utrecht: Uitgeverij Lemma.
Martens, R.L. (1998, b). Embedded support devices to enhance independent learning in printed and electronic learning environments. Paper accepted for presentation at the European Distance Education Network (EDEN) conference, University of Bologna, June 1998.
Montague, W.E., & Knirk, F.G. (1993). What works in adult instruction: the management, design and delivery of instruction. International Journal of Educational Research, 19, 329-331.
Moore, M.G. (1995). American distance education. A short literature review. In: F. Lockwood (Ed.), Open and Distance Learning Today (pp. 32-41). London: Routledge.
Morgan, A. (1995). Student learning and students’ experiences. Research, theory and practice. In: F. Lockwood (Ed.), Open and Distance Learning Today (pp. 55-66). London: Routledge.
Rowntree, D. (1990). Teaching through self-instruction. London: Kogan page.
Valcke, M., & Martens, R. (1997). An interactive learning and course development environment: Context, theoretical and empirical considerations. Distance Education, 18, 7-23.
Valcke, M., & Martens, R. (1998). Tools for third generation distance education. Paper accepted for presentation at the European Distance Education Network (EDEN) conference, University of Bologna, June 1998.
Vermunt, J.D.H.M. (1991). Leerstrategieën van studenten in een zelfinstructie-leeromgeving [Learning strategies of students in a self instruction environment]. Pedagogische Studiën, 68, 315-325.
Wade, S.E., & Thrathen, W. (1989). Effect of self-selected study methods on learning. Journal of Educational Psychology, 81, 40-47.