From application to graduation and beyond:
Exploring user engagement with e-portfolios and the e-advantage
Christopher Murray[c.murray@leeds.ac.uk]
Andrew Smith [ax.smith@leeds.ac.uk]
University of Leeds School of Medicine,
Claredon Way, Leeds LS2, UK
Abstract
This paper is an expanded version of a paper presented at the EDEN conference, Vienna, June 2006. It expands the model of learner engagement in the e-portfolio process and defines types of engagement experienced by the user. The factors which impact on the type of engagement enjoyed by the user are numerous and the paper will argue that feedback provision and relevance are the two most important issues which affect learner engagement and engagement type. Technology is not a barrier to the user but to the educator. Ownership is a secondary factor and in this study is shown to have little impact on the engagement process. The paper will also look at the e-advantage as reported by the users and try to examine why learners choose to use e-portfolios.
Keywords
E-portfolios, engagement, the e-advantage, implementation
Themes
- Introduction
- Literature Review
- The Context
- Methodology
- Analysis
- Technology
- Relevance
- Feedback
- Ownership
- The e-Advantage
- Conclusion
Introduction
The UK government has, through a series of white papers, placed e-learning at the centre of developing learning. In the white paper, 'Harnessing Technology' (DfES 2005) there is a call for education institutions to supply personal web space to learners to enable them to build electronic portfolios of their achievements to facilitate the process of lifelong learning. This development builds on the earlier statement that "Progress Files help make the outcomes, or results, of learning … more explicit, identify the achievements of learning, and support the concept that learning is a lifetime activity" (Universities UK 2003).
The aim of this paper is to build upon previous publications (Murray et al. 2006; Murray 2006; Tosh et al 2005; e-Pistle project report 2006) and explore further the engagement of users with e-portfolios within different educational and institutional contexts (Further Education, Higher Education and work), student's attitudes to e-portfolios; the benefits of using an e-tool for skills development; and the impact of the role of the feedback providers on the learners engagement with e-portfolio tools. If we are to promote lifelong learning and if we are to encourage users in the art of e-portfolio building and the process of reflective learning, their involvement in the process is paramount.
We will attempt to build upon and expand the previous model of engagement (Murray 2006), using data from a larger cohort of students and add into the cycle the effect of ownership on the type of engagement experienced by learners as well as note the e-advantages from the user's perspectives.
Literature Review
The pedagogical benefits of portfolio and e-portfolio based learning have been well documented (Woodward 2000; Woodward and Nanlohy 2004).The move from 'surface' to 'deep' learning (Gibbs 1992) and the ability to reflect upon learning (Kolb 1984; Schon 1983,1987; Boud et al 1985,1993) are all explored facets of learner-centred (e)-portfolio building (Harland 2005; Mathers et al 1999). Attention is also paid to their use in assessment (Barrett & Carney 2005). There have been studies on engagement with e-portfolios, some have placed the e-portfolio at the centre of the cycle (Barrett 2005) others have noted that e-portfolios increase engagement in learning (Armitage, 1998) but very few studies have placed the learner at the centre and asked the learners why they used the e-portfolios presented to them (Tosh et al, 2005) or investigated how they use e-portfolios (Murray, 2006). Without usage benefits cannot be demonstrated. In promoting e-portfolios use the phrase 'learner-centred' is abundantly used but it is not often used as a focus for investigation.
The Context
All of the e-portfolios were created within the University of Leeds VLE system, Bodington. Two of the e-portfolios were voluntary, one, the PRHO e-portfolio, was compulsory.
As part of the Enhancing Learner Progression (ELP) project (ELP Project 2005) The University of Leeds developed an e-portfolio to prepare students for their application to medical school. The purpose of the e-portfolio was to assist the widening participation activities of the school of medicine. Recent surveys have shown that there is still a need to encourage students from lower socio-economic backgrounds to apply to university (The Guardian 2006). The number of applicants from manual backgrounds has largely remained static. The e-portfolio represented a bridge between the last years of further education and university. Students worked through a series of careers education exercises designed to enhance their knowledge of medical and health-related degree courses and careers. Students were able to receive feedback on the work they completed from a careers adviser and undergraduate student mentors, helping them to prepare for the production of a personal statement for university admissions and admissions interviews.
The Pre-Registration House Officer (PRHO) e-portfolio was formulated in conjunction with the Yorkshire Deanery and a local hospital. An on-line version of the 'Curriculum for the Foundation Years in Postgraduate Education and Training' (NHS 2005) was designed and developed with the University of Leeds and the hospital's Director of Training. This curriculum was a new initiative and formed part of the modernising agenda within the NHS. PRHOs now need to complete two years of training before progressing on to the next level of their specialised medical education. The e-portfolio was, in theory, a space where they would be able to view their uploaded practical assessments, reflect on events and learning, formulate Personal Development Plans (PDP) and receive feedback from their assigned educational supervisor.
Another project 'Engaging Teachers in Students' Personal Development Planning and Recording of Progress' enabled the design and implementation of an e-PDP to be used within the undergraduate medicine course. There has been a significant change in the ways doctors are trained and there is a significant focus on reflection and critical analysis within under- and post-graduate medical courses. Personal Development Planning (PDP) by students meets resistance from students and faculty. The primary aim of this project was to facilitate participation of teachers into the process by working with them to incorporate relevant elements of their course materials, and thus enhance their abilities to encourage students to participate in recording achievements and setting goals. The different groups using progress file and e-portfolios are listed below:
Table 1. Groups Involved in Medical e-portfolio Projects
Learners |
Feedback required |
AIM |
FE Students |
Yes Careers Advisers and undergraduate mentors |
Preparation for HE |
Medical Undergraduates |
Yes Tutors |
PDP |
PRHOs |
Yes Educational Supervisors |
Assessments/PDP |
The three distinct e-portfolios represent a journey of early medical training and skills development, from application to registration, and across 3 separate educational sectors. The e-portfolios also incorporate the 3 functions of portfolio usage and styles (Hartnell, Young & Morriss 1999; Greenberg 2004): Developmental, Assessment and Marketing.
Theoretically each of the e-portfolios enable users to match and evidence their skills and competences within professional guidelines, reflect upon their learning and prepare them for applications to courses, postgraduate training and jobs. Theoretically creating a sound basis for lifelong learning and the educational and practical challenges that lay ahead.
Methodology
The projects were launched in June 2005 and evaluation has taken place at key points of development.
Two groups of FE students have been introduced to the e-portfolio: Cohort 1 in July 2005 (n=49) and cohort 2 between February and May 2006 (n=92). To date Cohort 1 have participated in formative and summative online evaluation surveys (n=7) and focus groups. Interviews have also taken place with the staff in these colleges who support the students (n=3). Cohort 2 are still using the e-portfolio and to date a formative evaluation questionnaire (n=55) and informal interviews have taken place.
No formative evaluation was undertaken with the undergraduate medical students using the e-PDP tool but a summative survey has been completed (n=216).
The PRHOs (n=8) and educational supervisors have taken part in a formative evaluation and summative responses are currently being collected. E-mail and informal comments have also been collected from this group.
The qualitative results are supplemented by quantative data collected from statistics of e-portfolio usage and patterns.
Response rate to the formative evaluation was 48%. So far the summative evaluation has yielded a response rate of 19%.
Analysis
From an earlier publication a model of engagement type was developed (Murray, 2006). See table 2 below
Table 2. Engagement of Users by 'Type'
Type of Engagement |
Characteristics |
Reader |
Makes no entries in the e-portfolio. May access once or on multiple occasions just to read the content. |
Tentative |
Makes 1 or 2 entries then stops. |
Selective |
Makes entries but only in sections they feel they can benefit from. May proceed to output stage |
Continuous |
Makes continuous entries and proceeds to output stage. |
The factors which impact on these types of engagement were identified as: technology, relevance, feedback provision, design of e-portfolio content, organisation attitude to technology and gender.
In this paper these factors will be analysed with the addition of a different and larger cohort of students, the e-PDP group. We will not be able to look into gender differences as no gender data is available for the e-PDP group but in addition to the factors cited above we will look at the effect of ownership on engagement and note the e-Advantages, from the user's perspectives, on the use of the e-portfolio tools.
Technology
When designing an e-learning tool we should not assume that all users possess the required level of IT skills. The technology should not overshadow the learning outcomes of e-portfolio use (Woodward and Nanlohy 2004). We should also remember that the entire process is 'about people….not technology.' (Dublin 2004, p294). If this is not taken into account then we are in danger of losing the faith and attention of those we are trying to engage. The use of technology should not be a barrier to the use of the e-portfolio and survey results have shown that the ease of use of the technology is not a barrier (Murray 2006). This view is further strengthened with the addition of the results from the survey of the e-PDP students. Eighty-five percent of the students engaged in the project found the technology easy and intuitive to use. In all of the cases the introduction of the e-portfolio was supplemented with user-guides available in paper and web-based versions as well as contact details for the project officers. Assistance in usage was only a phone call or a click away. Figure 1 illustrates the ease of use across all of the 3 cohorts.
Figure 1. Response To Ease of Use of E-portfolio Tool
Training on the use of the e-tools is an important aspect of a successful launch and 85% of the FE users and 38% of the PRHO users found the training aspect of the launch to have been adequate.
The trainee doctor group had received very little 'hands on' training on the use of the e-tool and their educational supervisors, none. This impacted on the initial phase of the project where only three out of the group (33) were using the portfolio regularly. A further training session resulted in more of the group accessing the area and by the end of the project 87% of the PRHOs have used the e-portfolio.
The undergraduate medicine e-portfolio was launched to all students through a lecture based session and all were aware of the tool and all found the training to be adequate.
Despite experiencing some problems navigating through the areas of the FE tool all of the seven respondents, from cohort 1, completed the e-learning exercise and the large numbers of PRHOs completing the e-portfolio suggests that if the final outcome of the process is of benefit to users, or if assessment is involved, users will persevere and overcome the barriers the technology presents to them.
The use and design of the e-portfolio tool can also impact on the level of the usage. Although they found the tool easy to use three of the FE group had trouble finding the information they wanted. The majority of PRHOs (n=8) reported some problems with usage and, the majority of users (6) state that they have had trouble finding the section/information that they wanted. Only one respondent found the e-portfolio easy to use and only two of the responding group had used an e-portfolio before. The largest number of problems resulted from the use of the e-portfolio itself, i.e. uploading files, making entries and granting permissions of access.
The supervisors and mentors have also faced difficulties in the use of the portfolio. By the end of the project 61% (n=33) of the educational supervisors providing feedback to the PRHOs accessed the system. The majority of this usage was concentrated in the last months of the pilot and around key assessment points. The supervisors have, on average, posted 2 feedback comments each. The attitude of some of the educational supervisors have included,
"A 'paper version' may have been a better solution."
"This is not designed for busy clinicians!"
Where the educators have been involved in the design of the e-tool, FE and medical students, percentages of usage amongst the student body has been higher, none of the FE (Cohort 2) or medical students have had trouble finding information.
The results of the FE and PRHO case studies starkly revealed issues of technology. Of the four colleges who signed up to the initial pilot, by September only two remained. There were difficulties with caching in two of these institutions leading to other students being able to view others work. This has also been in an issue with the PRHO sector. This had a large impact on the confidence placed in the system particularly when there is reflection and formal assessment involved throughout the project duration.
Students within the colleges were subsequently not encouraged to use the e-portfolio tool by tutors, careers professionals etc and no student usage was noted. Where the training and e-portfolio implementation had been successful students were able to make entries with confidence and staff within the two remaining institutions provided on going encouragement.
Within the context of lifelong learning it appears that users are not against the concept of e-learning or e-portfolios but it is the practicalities of usage and the attitude of those supporting them, that can hinder initial engagement and the move to 'reader' of the e-portfolio contents. IT training needs to be thoroughly integrated in curricula at all stages of education and, for some, learning on the use of e-learning tools needs to be thoroughly scaffolded in the initial phases of use, particularly to those supplying feedback. To move to the reader or tentative phase the technology needs to be operational and easy to use for all involved.
Relevance
The relevance of the e-learning process is key to the engagement process. The use of the e-tool needs to not only appeal to the learners but also to those supplying feedback or supporting the learners in the process. Two of the e-portfolios, the FE and undergraduate medicine progress file, were designed with input from tutors. The PRHO e-portfolio was copied from the paper version of the 'Foundation Years for PRHOs Curriculum' and no educational supervisors were involved in the design process.
When developing the e-PDP progress file tutors were engaged in the process of designing the new e-portfolio from the onset. Individual meetings were arranged with tutors to get their thoughts on how it should work and how their course materials could be included but the tutors were not required to provide feedback to the students through the e-portfolio tool. The students would present their evidence to tutors at appraisal meetings using their e-portfolio logs. Sixty one percent of the students used the e-portfolio at their appraisal meeting.
The FE Students found the e-learning exercise to be relevant when they were taken through exercises relating directly to the courses they wanted to apply for. The users noted that the 'Skills' section was the most useful aspect of using the e-portfolio. Relating their own skills to the skills needed to be successful in their chosen careers. The second cohort of FE users were asked what they felt the e-portfolio could help them with, the majority, 95%, felt it would help them to put together an application to university, 55%, the minority, felt it would introduce them to a new way of learning.
As the PRHOs needed to use the e-portfolio for assessment purposes its contents were relevant and the PRHOs rated the PDP section of the e-portfolio as the most useful section with 38%, the largest majority, feeling that the use of the e-portfolio had helped them to track their competencies. The majority of the PRHO users, 55%, used the e-portfolio selectively and only completed the sections they needed for assessment purposes.
When the medical students were asked which part of the progress file they found the most useful the following responses were noted:
Table 3. Response Of Undergraduate e-PDP users to the questions 'How Useful Did You Find 'My Progress File to….?'
Question |
Useful/Very useful |
To think about your strengths? |
52% |
To think about your achievements to date at medical school? |
60% |
To think about the skills you need to continue and work on? |
61% |
To think about new skills you need to develop? |
59% |
The majority of students did find the use of the e-portfolio useful but significant numbers of them did not. Relevance is the most important aspect in engaging students in using the e-tool. Some of the comments from the e-PDP students were noted through evaluation,
"Haven't used it much. Don't find it very relevant"
"I don't use the facility and I don't think my progress has suffered"
There is also the view expressed by some during the evaluation that engaging in PDP is not relevant due to its non-academic nature as expressed below,
"I have not used it as I fell my time is better spent academically studying."
This is a thought mirrored by one of the FE students who stated that the e-portfolio was not useful because,
"It's not for educational purposes/ qualifications."
Some students have found the contents of the tool useful with one FE student commenting,
"It makes you more motivated and you look forward to applying to the course you want, so you feel more confident!"
The contents of the e-portfolio have assisted the student in her career planning but the e-portfolio is only a tool to convey these materials. The practicalities of delivering such a programme within the curriculum would have been time intensive and the e-portfolio has provided accessibility, but, only to those who choose to use it.
It is not the use of the e-portfolio that emerges as the problem here rather what this use is for and the application of PDP as a concept. The contents of the e-portfolio have more impact on usage than the technology. Amongst all groups the learners who do not see the e-portfolio purpose as relevant do not move beyond the reader stage.
The users of these e-portfolios all have different goals and the e-portfolio contents need to match and assist the users in meeting their own specific targets.
Feedback
The advantages of the assessment and feedback properties of the process have been widely noted (Hartnell-Young & Morriss 1999). The feedback process validates the learning of the user but without regular and constructive feedback the use of the e-portfolio declines.
The users of the FE e-portfolios listed mentor feedback as one of the most important aspects of using the e-portfolio. When asked directly what was the most useful aspect of using the e-portfolio the following results were noted,
"Mentor feedback on personal statement. All advice and comments were useful as they have written one before and been successful"
"Personal statement and feedback"
The students valued the support they were given by the mentors and the help the e-portfolio provided in putting together their personal statements. The students also stated that having access to mentor support increased their confidence and awareness of the issues and procedures during the application process. One of the students also stated,
"What's the use in doing the e-portfolio if there is nobody to look at what you put in it?"
All of the FE users have received feedback in their e-portfolios and this has generated the largest percentage of continuous e-portfolio users. Fifty-five percent of those who have posted to the e-portfolio (n=29) have to-date completed each exercise.
Forty-one percent of the PRHO users have used the e-portfolio continuously and all of these learners have received feedback, in some form in their e-portfolio, they have also accessed and completed the non-compulsory (i.e. not needed for assessment purposes) sections of the e-portfolio. See table below.
Table 4. Completion of Non-Assessed Sections of E-Portfolio by Feedback Provision
Non-Assessed Section |
% of Users With Feedback |
% of Users Without Feedback |
Mid-point Review |
94% |
25% |
End of Placement Self Evaluation |
52% |
38% |
Careers Management |
6% |
29% |
The PRHO users are receiving feedback at summative points suggesting that they receive feedback on a more informal and regular basis but that this feedback is not recorded in the e-portfolio. A recent paper (Kilminster & Zukas 2005) suggests that feedback within the supervisory relationship can often be negative and this may account for the non-recording of comments within the e-portfolio tool.
As noted where there is supervisor presence, continuous and wider e-portfolio usage is experienced. PRHOs work with their educational supervisors, therefore using the e-tool remotely is not common place.
Where occurrences of feedback are high continuous use increases and more learners move from the tentative to the continuous stages of use, but how this feedback is provided relates to the context in which the e-tool is deployed.
Ownership
When questioned about the ownership of their respective portfolios, none of the PRHO e-portfolio users felt that they 'owned' their portfolios.
Has this hindered the type of entry the users place in their portfolios? Users do have the option of keeping reflective entries private but to be able to fully use the e-portfolio as a reflective tool supervisors do need to be allowed some access to the entries. This enables them to discuss thoughts and so facilitate learning. One clinical supervisor recently stated that there should be no private sections for reflection as he would like to know exactly what his trainee was thinking. 'This is the only way they can learn!'
When the FE users were asked the same question four stated that they felt that the university owned the portfolio with only two students feeling a sense of ownership over its contents. This portfolio and learning process was informally assessed with advice provision the desired outcome. There was no grading or professional competence to measure.
Snadden and Thomas (Snadden & Thomas 2003) noticed they same effect in their survey of paper-based portfolios. Trainees were less likely to reveal any reflective thoughts within a tool designed for assessment purposes. We need to examine ways to overcome this issue if we wish to encourage a sense of ownership over learning.
Of all of the users who have been asked about their feelings on ownership (n=30) only 2 have stated that they feel they own their e-portfolio log. One user noted,
"It is my personal learning space"
Others stated the university owned the e-portfolio as,
"It's on your computers"
However is ownership really required to engage in the learning benefits of the e-portfolio process? The majority of those who have engaged past the reader stage have noted some form of knowledge acquisition or reflective benefit from using the e-portfolio yet the minority feel that they have ownership over this space. We need to ask how important is ownership to the learner? Does it affect their learning process? Learners are still asked to learn what 'educators' think they should learn. Are the learners asked what they would like in the tool? Is it the mere physical differences between paper and technology that act as a barrier to ownership or are there other factors present? These questions need to be examined in more depth.
The e-Advantage?
It has been put forward that the learning process of adults utilises internalised reflection. (J. Piaget in Kolb 1984). This is an important aspect in lifelong learning and equally important in the writing of CV's and application forms. The developmental career theory (Super, 1979) expounds the view that adult career development occurs through life and is an on-going process. By experiencing jobs and careers and being able to reflect and evaluate our performance and satisfaction in each of these situations, as adults, we are able to crystallise and stabilise our career paths.
Reflection is also vitality important within the health and medical fields and its role in medical training is explored further in 'Tomorrows Doctors' (NHS 2003).
People, however, need to learn how to approach and make the best out of the 'reflection' process. Does the use of e-portfolios enable users to reflect and move through the learning and career development cycles mentioned above?
Sixteen of the PRHOs have been making entries in the 'Reflective Section' of the e-portfolio (all had been receiving some feedback in the e-portfolio) and all of the PRHOs have accessed the PDP (Personal Development Planning) section but only 1 PRHO claims that the use of the e-portfolio has helped them to improve their skills of reflection. They stated that the e-portfolio,
"Makes me think about things I have done that I wouldn't normally think about."
The e-PDP students were asked to provide examples of how using the e-portfolio had helped them 54% (n=104) of these comments were related to reflection and self assessment. Including,
"Helping to get my thoughts from any sessions into words. I have used it quite intensively and this has helped me in essays where I had to think back to a particular patient visit or work session"
Reflection within the college portfolio is not explicit but the work contained within the e-portfolio culminated with a personal statement for entrance to university this involves looking back on the exercises already completed. Has the use of the e-portfolio enabled these students to start thinking about the process in order to prepare them for the future?
When asked what they understood about the word 'reflection' the following points were noted,
"Looking back, addressing any issues, e.g. weakness and highlighting the positive and strengths. By reflection one should be able to appreciate what they are doing well and what needs improving and one should do this by looking back at an event or task and by setting targets for the future."
"Reflection is to look back and think about past actions which you feel have an importance, you consider your actions, reactions and how you could have improved upon that situation."
An understanding of the process of reflection provides a good basis for the learning and work challenges that lay ahead.
Some of the other positive comments to be derived from groups centred on the practical advantages of the e-portfolio,
"Useful to have an online non-losable log of progress"
"Good to have all your evidence in one place so you don't lose it and it looks far more presentable."
"Very helpful introduces [you] to new ways of thinking."
Twenty percent of comments from the e-PDP students related to the organisational and skills recording capabilities of the e-portfolio, 19% to the information it contained i.e. preparation for appraisal and 7% related to the non-use of the e-tool. As only 48% of all the respondents answered this question a comprehensive measure of all views cannot be analysed. Did this 52% find no benefit from using the tool?
It would appear from the results that a larger percentage of the medical students are finding the e-portfolio useful as a tool for reflection. The PRHOs are within a work-based environment and this may have led to the differences in the results. It also appears that different people see different advantages in using an e-portfolio. To some reflection is the major outcome of use to others the ability to receive feedback, organise work, grades and skills is a key factor but these activities do facilitate the process and make reflection easier. When learners make this link the full benefits of e-portfolio engagement is noted.
Conclusion
There are clear trends emerging within this evaluation. Feedback or an audience is pivotal to the continued use of e-portfolios. Students are noticing the reflective qualities of using e-portfolios within a learning environment once they move in to the posting stage. The issue is getting them to the initial reader phase and then maintaining their interest. When assessment is involved the users are more likely to pass through this stage but usage becomes more selective. There is a sense that learners of assessed e-portfolios are merely 'jumping through hoops'.
There is a need to explore what kind of engagement is necessary? Is the level of engagement determined by the users perceived needs and preferred method of study? For example a learner who simply reads the e-portfolio may still derive some learning benefit from usage. What type of engagement do we need? We need to explore more than just the e-portfolio experience of student learning but instead need to take a more holistic view of the individual user and the context they inhabit. Is the e-portfolio process compatible with their learning style and how do they view PDP and the reflective process in terms of their personal achievements? Learner-centred learning provides students with choices, is the non-use of the e-portfolio a valid and acceptable choice for the learner? As noted in a previous paper on the process of portfolio building, "portfolios are only acceptable to some learners" (Pearson and Heywood 2004)
The key factor in getting students to engage with PDP and e-portfolios is the engagement of the tutors and educational supporters. It has been shown that where tutors are behind the process, students are far more likely to take it seriously and become continuous users: 'learner feedback regarding engagement with PDP processes has repeatedly indicated that learners value dialogue with tutors, and are more likely to engage in PDP processes if embedded alongside tutor support'. In order to engage tutors, they must be involved with the process from day one. Their voices must be heard in every aspect from design through to implementation. If they have some level of ownership they are far more likely to champion the cause than if they are simply asked to as a matter of departmental policy. They are closer to the students/PRHOs and so have a better idea of what will work and what is needed. They know from their own past experience, certainly with respect to their own modules and experiences, which areas students struggle with most, or which areas students need to develop most. For an e-portfolio to work and engage learners it needs to be relevant, both to students and tutors.
References
[1] DfES (2005) e-Strategy: Harnessing Technology Transforming Learning and Children's Services
[2] Universities UK (2003) Progress Files for higher education: Guidelines for HE Progress Files, Quality Assurance Agency. http://www.qaa.ac.uk/academicinfrastructure/progressFiles/default.asp (Accessed 25th Sept 2006)
[3] Murray C & Smith A (2006) From Application to Graduation and Beyond: User Engagement in the e-portfolio Process. EDEN Conference proceedings pp 211-216
[4] Murray C (2006) Is Anybody Out There? Exploring User Engagement in the e-portfolio Process, conference paper. E-portfolio 2006, Oxford.
[5] Tosh D, Light T.P, Fleming K, Haywood J (2005) Engagement with Electronic Portfolios: Challenges from the Student Perspective. Canadian Journal of Learning and Technology, vol 31 (3).
[6] University of Wolverhampton. (2006) Factors affecting use/non-use of e-portfolios by learners. Report of e-pistle project. http://asp2.wlv.ac.uk/epistle/5_report.pdf (Accessed 29th Sept 2006)
[7] Woodward, H (2000) Portfolios: Narratives for Learning. Journal of In-Science Education, Vol. 26 (2) pp 329-349
[8] Woodward, H & Nanholy, P (2004) Digital Portfolios: Fact Or Fiction?, Assessment & Evaluation In Higher Education, Vol. 29 (2), pp 227-238
[9] Gibbs, G (1992) Improving The Quality Of Student Learning. Technical and Educational Services.
[10] Kolb, D. (1984) Experiential Learning. Prentice Hall, New Jersey
[11] Schon, D (1983, 1987) The Reflective Practitioner. Temple Smith, London
[12] Boud ,D, Keogh, R, Walker, D Eds (1985) Reflection: Turning Experience Into Learning, London, Kogan Page
[13] Harland, T (2005), 'Developing a Portfolio to Promote Authentic Enquiry in Teacher Education.' Teaching in Higher Education, vol 10, no 3: Pg 327-337
[14] Mathers, N.J, Challis, M.C, Howe, A.C & Field, N.J (1999).'Portfolios in Continuing Medical Education-Effective and Efficient? Medical Education vol.33, pg 521-530
[15] ELP Project at http://www.jisc.ac.uk/index.Cfm?Name=Bradfordelp&Src=Alpha (Accessed 21st Sept 2006)
[16] Smith, A (2006), 'Government Failing to Widen University Access' The Guardian, http://education.guardian.co.uk/universityaccess/story/0,,1824303,00.html (Accessed 29th Sept 2006)
[17] Barrett, H (2005) Professional Development For Implementing Electronic Portfolios at http://electronicportfolios.org/portfolios.html#pubs (Accessed 28th Sept 2006)
[18] Barrett H & Carney J (2005) Conflicting Paradigms And Competing Purposes In Electronic Portfolio Development at http://electronicportfolios.org/portfolios.html#pubs (Accessed 28th Sept 2006)
[19] Armitage, C (1998);'The Benefits of Pause for Thought', The Australian, pg 16 in Woodward, H & Nanlohy, P (2004) 'Digital Portfolios: Fact or Fashion?' Assessment & Evaluation in Higher Education Vol. 29, No 2: 227-238.
[20] Hartnell-Young, E & Morriss, M (1999) Digital Professional Portfolios For Change, Skylight, Illinois
[21] Greenberg, G (2004) 'The Digital Convergence: Extending The Portfolio Model', Educause Review, Vol. 39 No.4: 28-37
[22] Dublin L(2004), The Nine Myths About E-Learning Implementation, Industrial And Commercial Training Vol. 36, No 7 Pg 291-294
[23] Super, D. (1984). Career and Life Development. In D. Brown & L. Brooks (Eds.), Career Choice and Development. San Francisco: Jossey-Bass
[24] General Medical Council (2003), Tomorrows Doctors, http://www.gmc-uk.org/Education/Undergraduate/Tomdoc.Pdf (Accessed 20th Sept 2006)
[25] Snadden, D And Thomas, M (2003), Portfolio Building In General Practice Vocational Training - Does It Work? , Medical Education Vol. 30, pp 148-152
[26] Ward, R & Richardson, H (2005) Developing An Implementing A Methodology For Reviewing E-Portfolio Products. Pg. 34. Report To JISC At http://www.jisc.ac.uk/uploaded_documents/epfr.doc (Accessed 29th Sept 2006)