Self- and peer-assesment within the context of innovation in assessment & evaluation
Martin Valcke, Department of Educational Sciences, University of Ghent, Belgium
© 1999
Abstract
Dimensions in assessment and evaluation
What learning objectives does the assessment
& evaluation focus upon?
Who is responsible for the evaluation?
When does the evaluation take place?
What is the formal value of the
assessment and evaluation procedure?
Assessment & evaluation and views on learning and instruction
New developments in assessment and evaluation and the third generation distributed
learning approach
Third generation distance education at the Dutch Open University
Positioning the contributions in this publication
References
Abstract
This publication focuses on a number of recent developments in the field of assessment and evaluation. Assessment & evaluation is in this context especially dealt with at the micro-level of the learning and instructional process.
In this introductory editorial contribution, we position these new developments in a larger context. A number of dimensions is elaborated to explain the broadening picture of assessment & evaluation. Next, we relate these dimensions to the development of theoretical views on learning and instruction. These theoretical views are a good introduction to the next issue: third generation distance education. Distance education or distributed learning is undergoing a dramatic change since the adoption and integration of information and communication technologies. The second-generation paradigm is being replaced by a third generation approach that clearly implies a rethinking of assessment and evaluation approaches.
We finalise this editorial introduction by positioning the four contributions in this publication in relation to the dimensions put forward.
Dimensions in assessment and evaluation
When looking at the way assessment and evaluation evolve in the context of instruction, we can distinguish a number of dimension to grasp their role and function.
What learning objectives does the assessment & evaluation focus upon?
When we distinguish between declarative, procedural and meta-cognitive knowledge, we can perceive in the evolution of assessment & evaluation approaches a clear change in focus. Initially, there was a strong emphasis on the assessment & evaluation of declarative knowledge (facts, concepts, principles, theories, structures, etc.). Now there is a clear interest in procedural knowledge (skills, heuristics, procedures, etc.). And with the growing attention being paid at regulative processes (cf. Vermunt, 1996), also meta-cognitive knowledge is being assessed and valued. Learners are as such expected to reflect upon their individual (or group) learning process and measures in relation to these processes are part of the assessment and evaluation procedure. Figure 1 depicts this new set of interrelated objective types
Figure 1: Types of objectives focused upon in
assessment & evaluation approaches
Who is responsible for the evaluation?
Figure 2 depicts how the instructor is now no longer considered as the sole actor responsible for the assessment & evaluation process. On the one hand there is a tendency to involve peer groups of learners in the process. Learners 'learn' to assess their own individual and group behaviour and to value this behaviour; described in the literature as 'peer evaluation'. This is linked to - as will be explained later - new ideas about the role learner in stating objectives and directing the learning process. In the same line is the development along which learners take - at the individual level - a responsibility for the assessment and evaluation process. Self-assessment is an approach that is more and more accepted as a part of the overall evaluation cycle. To a certain extent, we can link this observation to the former dimension that focused upon new types of objectives. Involving learners actively in the evaluation cycle is in line with pursuing and assessing/evaluating meta-cognitive objectives.
The availability of computer-based test service systems also enlarges the possibility to support self-assessment. Such systems give professional tools in the hand of the learner to monitor their learning process.
Figure 2: Shifting involvement of stakeholders in the
assessment & evaluation process
At the other hand we also perceive a growing interest in involving other persons that take a responsibility in the assessment & evaluation process. For typical objectives, experts get involved. Next, we can state a growing tendency to design, develop and control the overall assessment and evaluation cycle at the institutional level. The instructor is becoming a member of a team. He/she is supported with assessment specialists. The assessment is 'instrumented' by making use of test service systems, etc. A further - still considered as an extreme alternative - is the situation where an external body/institute takes the lead in the assessment process. Controlling the extent to which e.g. private training institutes adhere to clear standards and norms is but one context where such external positioning of the assessment might be of relevance.
When does the evaluation take place?
Earlier and very traditional views on instruction solely focus upon assessment & evaluation processes at the end of the instructional process. Considering the development of new views on learning, this has changed to a very large extent. In this perspective we can e.g. refer to the didactical model of Dochy (1995) as depicted in figure 3.
Figure 3 shows ho the instructor can adopt a variety of assessment and evaluation approaches to support the didactical process 'before, during and afterwards'.
This test-driven approach is an early evocation of the later development of adaptive learning systems where assessment & evaluation techniques continuously direct further steps in the instructional process.
Figure 3: Assessment and evaluation integrated in the
entire didactical approach.
What is the formal value of the assessment and evaluation procedure?
The former dimension explains also how the focus has been redirected from summative evaluation that is directed towards certification, towards recurrent formative evaluation cycles that especially serve the instructor of the instructive system to (re)orient the instructional process.
But this new development can also be approached from another direction. In the former way of reasoning, there is still a focus upon a kind of final summative assessment. New directions illustrate that such focus is no longer needed. E.g., learners demonstrate a clear performance level in relation to specific (procedural) objectives on subtasks. The performance on these subtasks is considered as the base for the final evaluation. Adding the task-sub-scores gives the instructor the final score.
Depending on the choices instructors make along the dimensions described above, we can reflect upon their – implicit or explicit – views about the nature of the learning process and consequently the instructional approach. We elaborate this in the next section.
Assessment & evaluation and views on learning and instruction
In this contribution there is no room to elaborate in a detailed way with this issue. But in summary we can focus on three major theoretical positions towards learning and consequently the way instruction and evaluation should be addressed: the behavioural approach, the cognitivist approach and the constructivist approach.
The behavioural approach has introduced a strong focus on operational objectives and evaluation. Skinner (1968) states this as follows: "The application of operant conditioning to education is simple and direct. Teaching is the arrangement of contingencies of reinforcement under which students learn. They learn without teaching in their natural environments, but teachers arrange special contingencies which expedite learning, hastening the appearance of behaviour which would otherwise be acquired slowly or making sure of the appearance of behaviour which otherwise never occur." (…) "... the school of experience is no school at all, not because no one learns in it, but because no ones teaches. ...; a person who is taught learns more quickly than one who is not ... ".
Applications of the behavioural vision on the learning process are abundant. The Personalised System of Instruction (PSI) is a first example (also called the Keller Plan). Learning materials are split into small units with clear objectives. Students progress individually. There is - in relation to each unit a type of testing. Students get immediate feedback by a 'proctor'. Students only progress when they master at least 90 % of the objectives (Gage & Berliner, 1984, pp.529-530, Woolfolk, 1990, pp.185-186).
Another typical application of the behavioural approach is mastery learning. Bloom stresses again the importance of clear objectives and the central position of formative assessment.
The cognitivist approach introduces new perspectives. Ausubel (1968) introduces e.g., the importance of building upon prior knowledge. Instructors need to assess and evaluate this to use it as a corner stone for the forthcoming learning process. Another typical cognitivist theorist is Gagné (1965, 1985) who explicitly states a number of instructional events' that are related to assessment and evaluation:
- State clear objectives.
- Activate prior knowledge; measure prior knowledge.
- Give informative feedback to guide the learner step-by-step to the right solution.
- Use performance tests that focus upon application of things learned. Apply progress tests.
The constructivist approach introduces completely new ideas. Summarising the great variety of interpretations of the constructivist principles, we find again clear-cut ideas about assessment & evaluation (Jonassen, 1991; Merrill, 1991; Valcke, 1990; Wilson & Cole, 1991; Murphy, 1998):
- Build upon learning objectives of the learner.
- Stress the importance of problem solving objectives
- Assessment & evaluation is not a separate process. It is part of the experiential cycle and interwoven in the learning process.
- Look for assessment criteria that are in line with the objectives put forward by the learners.
When looking at specific applications of the constructivist principles, we find an additional list of alternative views. Evans (1994) states that prior knowledge - whether certified or not - should be acknowledged in a learning process. He defends the position that such prior knowledge should be certified and especially that this prior knowledge should direct the intake-procedure.
Also problem-based learning is an application of constructivist principles. Moust & Schmidt (1998, p.165) state in relation to assessment & evaluation that the students themselves state the objectives. They also play a major role in determining whether the objectives have been reached. In this context it is important to indicate that the role of the facilitator in the problem based learning process especially focuses upon meta-cognitive scaffolding of the process.
New developments in assessment and evaluation and the third generation distributed learning approach
Assessment & evaluation has always been a special concern for distance education institutes. The paradigm shifts in this field have strongly affected approaches towards assessment and evaluation. These shifts are described as generations of distance education approaches (Nipper, 1989) and in a way also reflect the historical development of views upon learning and instruction as discussed in the former paragraphs.
The first generation of distance education reflected a correspondence model of learning. Printed packages were distributed. Assessment & evaluation reflected a behavioural approach that focused on objectives that stressed the importance of declarative knowledge. The second generation of distance education reflects an industrial model for developing comprehensive packages of learning materials. Again the objectives are pre-defined and next to declarative knowledge some procedural knowledge is pursued. Assessment & evaluation builds upon the potential of new audio-visual media to develop and assess skills. But there is still a pre-dominance of attention being paid to learning material development. The fact that a large number of second-generation distance education institutes use automated test service systems (with MC-questions) also reflects a certain one-sided orientation towards assessment & evaluation.
The now growing interest in realising the third generation of distance education models introduces completely new perspectives (cf. Kirkwood, 1999 and Thorpe, 1999). In this third-generation approach the information and communication technologies help to restructure the entire learning and instructional setting:
- materials are considered as resources;
- materials are flexible and delivered on demand and in line with learner needs
- collaborative learning;
- competency driven;
- interaction and communication is fundamental (intake, task elaboration, assessment & evaluation).
- Self assessment, peer assessment are e.g., are now used in audio-graphics settings, video-conferencing sessions, discussion lists.
Third generation distance education at the Dutch Open University
The Dutch Open University is currently involved in a large scale project to design and to develop an entire electronic working and learning environment in line with the ambitions of third generation approaches. In this ELON-project an attempt is made to tackle the five principles mentioned above in a very operational and instrumental way. Figure 4 depicts in a schematic way how this is envisioned. The triangles in the scheme indicate the possible moments evaluation and assessment can play a specific role. Competence maps are the driving forces to state curricula that build upon tasks. Tasks are as much as possible elaborated in a collaborative way. To deal with tasks, learners get resources. Special interactive testing provisions help students to state their mastery of these resources (if necessary). Experts can play a part in the assessment.
Positioning the contributions in this publication
When we consider the different contributions in this publication and reflect upon the background information elaborated in this editorial introduction, we can position the individual contributions as follows:
- Judith Thomas of the University of Bristol & University of the West of England focuses on the challenges of developing self assessment practices in different disciplines and institutions
- Dominique Sluijsmans & George Moerkerke, both of the Open Universiteit Nederland discuss student involvement in assessment.
- Vasi Van deventer of the University of South-Africa (UNISA) will take a content specialist perspective on peer evaluation in instructional design.
- Rob Martens & Henry Hermans, again of the Open Universiteit Nederland will report on results of a study about Internet based formative prior knowledge assessment.
References
Ausubel, D.P. (1968). Educational Psychology: A cognitive view. New York; Holt Rinehart and Winston.
Dochy, F. (1992). Assessment of prior knowledge as a determinant for future learning. Utrecht: Lemma.
Evans, N.(1994). Experiential for all. London-NewYork: Cassell.
Gage, N.L., & Berliner, D.G., (1984). Educational Psychology. Boston-London: Houghton Mifflin Company-Palo Alto.
Gagné, R. (1965). The conditions of learning. New York: Holt.
Gagné, R. (1985). The conditions of learning and theory of instruction. 4th edition. New York: Holt.
Jonassen, D. (1991b). Evaluating Constructivist Learning. Educational Technology, September issue, 28-33.
Kirkwood, A. (1999). New media mania: Can information and communication technologies enhance the quality of open and distance learning? . Distance Education, 19(2), 228-241.
Merrill, M. D. (1991). Constructivism and instructional design. Educational technology, may, 45-53.
Moust, J. & Schmidt, H. (1998). Probleemgestuurd onderwijs. Groningen: Wolters-Noordhoff.
Murphy, E. (1998). Constructivism: From Philosophy to Practice. http://www.stemnet.nf.ca/~elmurphy/emurphy/cle2b.html
Nipper, S. (1989). Third generation distance learning and computer conferencing. In R. Mason & A. Kaye (eds.), Mindweave: Communications, computers and Distance education. Oxford: Pergamon Press.
Thorpe, M. (1999). Assessment and third generation distance education. Distance Education, 19(2), 265-286.
Reijntjes, J. & Valcke, M. (1998) Implications of electronic developments for distance and face to face learning. The Law Teacher – The International Journal of Legal education, 32(3), 245-259.
Valcke, M. (1990). Effecten van het werken met Logo-microwerelden. Gent: Faculteit van de Psychologische en Pedagogische Wetenschappen.
Wilson, B., & Cole, P. (1991). A review of cognitive teaching models. Educational Technology Research & Development, 39(4), 47-64.
Skinner, B.F. (1968). The Technology of teaching, New Jersey: Englewood Cliffs, Prentice-hall.
Vermunt, J. & van der Sanden, J. (1996). Inleiding op het thema leerstijlen [Introduction to the thematic issue on learning styles]. Tijdschrift voor Onderwijsresearch, 21, 291-293.
Woolfolk, A.E. (1990). Educational Psychology. Boston: Allyn and Bacon.