NEW

Best of EDEN 2018 Special issue
 

There are not any recent contributions.

There are not any recent contributions.

Archives

EURODL Mailinglist

2851 subcribers
 

EURODL Visitors

back

Converging traditional assessment philosophies with open assessment philosophies - introducing a low tech. solution through action research

Authors
Andrea Cassidy & Lawrence Benson
School of Health Studies, University of Bradford, Bradford, West Yorkshire, BD7 1DP, United Kingdom

© 1999

Abstract
Introduction
Methodology
Results and Discussion
- question 1 & 2
- question 3
- question 4
- question 5
- question 6
- question 7
Conclusion
References
Figure 1: Action Research Plan

Abstract

This paper uses an action research cycle to implement and evaluate a new assessment strategy in the BSc in Professional Practice - Open / Distance Learning. After a short introduction on the action research cycle (given as figure 1 at the end of this article), we present the results of an evaluative questionnaire identified as an action step from the action research process.

Introduction

The Division of Health Care Studies (created April 1996) in the University of Bradford offers an Open and Distance learning degree (BSc in Professional Practice - Open / Distance Learning) with a philosophy grounded in adult learning theories (Bruner 1966, Revans 1983, Knowles 1985, Rogers 1982 ).

Traditionally the University incorporates a written invigilated examination for each component of a degree to ensure that the student’s knowledge is tested appropriately, an assessment incongruent with the adult learning philosophy of the open learning degree.

The original assessment scheme for the degree consisted of written assignments thus not adhering to the University’s requirements nor the Division’s philosophy, as it did not fully assess the application of theory to practice through reflection (Schon 1983). Health care professionals are being encouraged to use reflection to evaluate practice in the United Kingdom (UKCC and the IHSM). It was therefore decided to implement an assessment strategy that would meet the needs of both the university and the division. We consequently commenced an action research study to help us implement and monitor a new assessment strategy.

Methodology

Elliot’s (1991) Action Research cycle was used to structure the enquiry. The stages in action research can be described as a cyclical process involving the diagnosis of a problem, the development of an action plan, the implementation of the plan and the evaluation of the plans effectiveness.

Data was collected using pen and paper methods and, live methods (McNiff 1988). The pen and paper method involved notes from meetings, written feedback from tutors and questionnaires sent to students who had experienced the assessment interview. The live methods included simulating the experience for tutors and the actual interviews themselves.

The sample for the questionnaire was purposive (Frankfort-Nachmias and Nachmias 1997) and included all students who had undertaken the newley implemented assessment interview of which there were 19. The questionnaire was designed using open and closed questions.

Results and Discussion

The action research process identified a number of issues regarding the implementation of the new assessment strategy and each issue formed a new action step, which was then implemented. These issues are best described using the action research framework and this is diagramaticaly demonstrated in Figure 1. One of the major decisions to take place during the process was to evaluate the student experience with a view to further improving the assessment strategy. It was felt that the best way to tackle this was to design and send out a questionnaire. The following results are taken from the questionnaire sent to those students.

There were 13 respondents from the 19 questionnaire sent indicating a response rate of 68. Of the 13 replies 5 had experienced face to face interviews and 8 had experienced telephone interviews. The results were analysed using the student’s experiences as a whole and a comparison of the two options answers were then categorised for tabulation purposes. However the authors were interested in the qualitative answers.

Question 1 & 2

The first question was simply to determine which interview option the student had chosen, face to face or telephone, the second question however was intended to obtain a more qualitative reply and was concerned with the students thoughts about having this type of assessment. 7 of the respondents indicated levels of anxiety, this was more pronounced in the telephone assessment.

• "I was very nervous about having an assessment interview"
• "Apprehensive although reassured because it is using reflective practice"

6 respondents however believed it to be a good way to undertake the assessment for a number of reasons

• "Some people can talk better than write therefore talking and writing gives everyone a fair chance"
• "Had preference for this type of assessment as compared to a written examination"

and 2 students felt they had not been prepared adequately

• "there were no specific guidelines for the assessment so one could have read all the books available and still been unprepared"
• "Very anxious not knowing what questions would be asked, would there be enough time? How would I answer? Would the answer be appropriate…………."

The conclusions from this question were that students needed a great deal of reassurance prior to the interview and needed better preparation, perhaps the module leader to go through the written guidelines with the students and give example questions.

Question 3

Question 3 was concerned with the preparation for the assessments, one of the assessments aims was to encourage students to read their materials and apply the concepts to practice. There was a wide range of different preparation strategies reported by students (Fig 2) the most common ways of preparing were reading the open learning materials and taking notes before for the interview.

Figure 2: Answers for question 3

Students who had chosen the telephone or the face to face assessments reported that the oral assessment had stimulated them to read the open learning materials, they all utilised other sources as well, including their reflective notes. The conclusion from these answers was the assessment was meeting its aim:

• "By reading the material and highlighting the main points and making notes on how each point related to my practice"
• "Read course material, made notes on daily practice"

Question 4

Question 4 was designed to give the student the opportunity to describe the experience first hand so the question asked was "How did you find the interview on the day?" The comments made contained a mixture of emotions, the students were describing the anxiety they felt alongside the relief that the experience was more positive than they had expected and were therefore looking forward to the next time.

• "Relaxing not as bad as I expected encouraged by the non verbal communication"
• "Initially nerve racking, but as the interview progressed I became more settled and at the conclusion my feeling was of a relaxed nature. Generally I enjoyed the interview and felt really good afterwards in the knowledge that I answered the questions confidently. I was surprised that I remembered so much, and also how easy it was to relate theory to my practice. I put this down to my thorough revision of the whole module as I did not need to refer to my notes. Brilliant! Not as worried about the next one"

2 respondents however described a negative experience:

• "Awful, I felt as if I waffled my way through it, however it has prepared me for the next one"
• "A disaster, + stressful occasion. Felt stupid trying to answer questions so long winded. Forgot the question by the end."

The responses to this question has led us to plan a follow up study to ascertain whether the anxiety experienced at this interview was reduced at the next interview and also to repeat this study with a new set of students following the changes made to the preparation.

Question 5

We wanted to discover the student’s opinions of the benefits and weaknesses of the assessment process therefore in question 5 the students were asked to comment upon the benefits and weaknesses of the assessment process (Fig 3).

Figure 3: Answers for question 5a

There was a wide range of benefits reported by both groups of students.

• The most commented upon were that the oral assessment interview:-
• Explored the relationship between theory and practice;
• Provided a stimulus for using learning materials;
• Offered a way with which the student between verbal and written expression could strike a balance across the degree and its modular components.
• Other reported benefits included that the assessment interview:-
• Allowed an exercise of communication skills;
• Gave an opportunity for the student to clarify issues and their reflections;
• Acted as a medium for the synthesis of ideas from the student’s practice and their studies on the module.

There was also a wide range of commentary regarding the weaknesses of this assessment approach (Fig 4)

Figure 4: Answers for question 5b

The most commonly reported included having no time to think due to nervousness, the assessment design and finding the oral assessment off putting if you were not of an out going personality. It was interesting to note however that none of the face to face assessment students made a comment about the assessment design and one student assessed by telephone suggested that the telephone itself could potentially be a weakness.

The conclusions we drew from this two-part question was described eloquently by one student: "The benefits outweigh the weaknesses".

Question 6

Question 6 asked the student directly if they had been encouraged to read the open learning materials by the assessment interview (Fig 5) 11 students stated that this was the case and only 2 stated that it had not and these statements were suggesting that the materials would be read anyway.

Figure 5: question 6

"No – it encouraged reading around ones own chosen topic area"

"Not really, you had to read the materials to be able to instigate the written assessment"

One respondent wrote

"It encouraged me to read the materials in considerable depth and try to develop a clear understanding of complex areas.It also encouraged me to use the materials as a basis for study ie, taking each section and selecting areas which I was weak on. Then carrying out my own research & study in these areas to meet my learning needs"

This reconfirmed one of our main aims which was to make the reading of the materials more meaningful.

Question 7

Similarly question 7 asked students directly if the assessment interview encouraged them to reflect on the application of theory ( Fig 6) 12 said that it did compared to 1 that said it did not

Figure 6: question 7

"NO there was no time to reflect, no amount of studying prepares you for the interview. My mind went blank, couldn’t concentrate on questions, Too nervous."

Of the 12 who believed it had encouraged them to relate theory to practice, comments were made confirming our belief that application of theory to practice can be demonstrated in this way and has actually taken place, going someway to reducing the theory practice divide:

• "Yes. It gave the opportunity to add depth and feeling, adding your values. Kept opinions into perspective. But also made you think and evaluate ourselves & enhance the care each individual client receives. Also realising our own faults and failures and finding ways of compromise"
• "Yes encouraged you to rapidly analyse the topic in terms of applicabilityetc to your own area. One question asked said ‘how your system aides strategic planning?’ as the assessment was verbal I had the opportunity to say ‘well actually it doesn’t BUT I can tell you why that is"

The final question simply asked for any other comments of which there were a number, varying from affirmation of it’s benefits to some tips for improvement such as:

• "a warming up, 5 min exercise before the interview starts"
• "Interviews need to be more structured & more info about them given to students"

Conclusion

In conclusion we believe we have designed and successfully implemented an assessment strategy that enables all students of varying learning needs to achieve the same results. The assessment demonstrates the application of theory to practice and ensures that the student is practising reflectively. The end result must be a positive one not just for the students learning and grades but also for the clients and organisations who will be on the receiving end of this able and knowledgeable practitioner.

"The benefits are to give writers and talkers an equal chance, It is a communication skill that nurses and other health care professionals should have. Speaking clearly and using language appropriate to patients/clients and other people is a necessary part of nursing care"

Action research is a suitable method for carrying out this type of change as it enables close evaluation and refinement to be made constantly ensuring the change is effective and successful.

We once again find ourselves at a reconnaissance and we will be re-defining the general plan soon it is always difficult to end a cyclical process and so this paper does not end it awaits questions, ideas and further discussion.

References

Bruner, J. (1966) Toward a theory of Instruction, New York: W.W. Norton.

Elliott, J. (1991) Action research for educational change Buckingham: Open University Press

Frankfort-Nachmias, C. and Nachmias, D. (1997) Research methods in the social sciences 5^th Ed London: Arnold

Knowles, M. (1985) Andragogy in Action, London: Jossey-Bass

Mc Niff, J. (1993) Teaching as learning: an action research approach London: Routledge

Revans, R. (1983) The ABC of Action Learning, Bromley, Chartwell-Bratt

Rogers, C.R. (1982) On becoming a person, London: Constable.

Schon, D.A. (1987) Educating the reflective practitioner, San Fransico: Jossey-Bass

United Kingdom Central Council for Nursing, Midwifery and Health Visiting (1992) Code of Professional Conduct, London, UKCC.

Institute of Health Services Management (1997) Continuing Professional Development - Support Book - Pilot Scheme, London: IHSM

Figure 1: Action research plan