Promoting the Concept of Competency Maps, Interprofessional Assessments and e-Portfolios to Enhance Student Employability
Catherine A Coates, [c.a.coates@leedsmet.ac.uk]
Jill D Taylor, [j.d.taylor@leedsmet.ac.uk]
Leeds Metropolitan University, United Kingdom
Julie C Laxton, [j.c.laxton@leeds.ac.uk]
University of Leeds, United Kingdom
Chris A Dearnley, [c.a.dearnley@bradford.ac.uk]
University of Bradford, United Kingdom
Ian Hall [rob@myknowledgemap.com]
MyKnowladgeMap Ltd., United Kingdom
Background to the ALPS CETL approach
Assessment and Learning in Practice Settings (ALPS) is a collaborative Centre for Excellence in Teaching and Learning (CETL) comprising five Higher Education Institutions (HEI) with proven reputations for excellence in learning and teaching in Health and Social Care (H&SC): the University of Bradford, the University of Huddersfield, the University of Leeds (lead site); Leeds Metropolitan University, and York St John University. There are 16 professions across the partnership from Audiology to Social Work, and a wide range of partners including NHS Yorkshire and the Humber and commercial partners who are working towards a framework of interprofessional assessment of common competences in the H&SC professions.
The focus of this paper is the development of the common competency maps for communication, teamwork, and ethical practice along with a set of standardised tools to assess these across the sixteen professional groups.
The aim of the ALPS CETL is to ensure that students graduating from courses in H&SC are fully equipped to perform confidently and competently at the start of their professional careers.
Fundamental to the care of service users within modern Health and Social Care are key skills commonly utilised by the range of professionals involved in ALPS. Key skills and learning outcomes vary across the 16 pre-registration H&SC courses but central to the practice of all of the professional groups represented by ALPS is a high level of professional competence in communication, teamwork and ethical practice. In order to make explicit this pretext it was decided that mapping these common skills would enable students to navigate their way through the professional competencies allowing them to gain confidence and competence in practice settings. ALPS worked with a commercial partner, MyKnowledgeMap Ltd. (MKM), to facilitate this process which resulted in interactive and creative competency maps from which multiprofessional assessment tools were derived for students to validate their skills in their practice placements. ALPS has developed a shared services platform that enables these common assessment tools to be delivered onto mobile devices used by the students in their practice placements.
Central to the ALPS process was the development of an e-portfolio tool to which the student could publish their completed tools and any relevant supporting documents and gain feedback from their tutor back at their University, further perpetuating the learning process and enabling the tutor to evaluate the students progress.
This paper discusses how these processes championed by ALPS can be transferred and shared across professions and describes the challenges, benefits and future potential of this approach aimed at enhancing the students ability to learn and produce effective assessments in practice settings.
Development of the Competency Maps
At the beginning of the mapping process each pre-registration course was asked to identify where communication, teamwork and ethical practice featured in programme structures, how these key skills were assessed and the relevant professional, statutory and regulatory body requirements guiding the development of these skills.
This information gave us a breadth of understanding around common language and the variations in terminology and assessment methods from which to build a mutually acceptable concept across professional groups.
A core working group was established to develop the maps constituted from a range of ALPS professions across the five HEIs and practice learning facilitators who had a multiprofessional practice perspective. The core working group of about 8-10 people was supplemented by a wider email consultation group representing all of the professions who were able to comment on each stage of this iterative development process.
The maps emerged with the guidance of MKM by grouping statements describing firstly communication skills into common themes and established a hierarchy of broad cluster statements. Each of these was then described by a dimension statement and then further subdivided into element descriptors for which performance criteria were written.
Figure 1 The ALPS Communication Skills Map
This process was then repeated for team working and ethical practice skills to develop 3 maps with many linking performance criteria which would ultimately form the basis of the common interprofessional assessment tools. The communication skills map is illustrated in Figure 2 and can be accessed, along with the other maps for teamworking and ethical practice from the ALPS CETL website. The common assessment tools that were derived from the maps can be rendered onto mobile devices allowing the students to complete mobile assessments in the practice setting and gain 360 degree feedback from practice assessors, service users and peers as well as promoting self evaluation and reflection to enhance their practice.
Additionally students could prepare themselves for practice and employment using the map to gain an understanding of the professional expectations within the work environment. The completed maps allow course tutors to develop assessment learning and teaching strategies which are linked to professional practice. At Leeds Metroplitan University these common competency maps have been embedded into the interprofessional learning (IPL) strategy for students to use in a multiprofessional group as an aid to developing practice competencies.
Establishing an acceptable and effective process and structure during the mapping of communication was crucial to the success of this collaborative process (Coates et al, 2008; Holt et al, 2010). Initial discussions of the working group were lengthy and reflected discussion on topics such as common and differing language and terminology across professions, and whether the maps should reflect academic levels or a hierarchy of skills and protocols for the dissemination of the maps to colleagues. Gradually such debate resulted in a comfortable trust across and within professions and facilitated an acceptable and effective process and structure from which the same principles could be used to develop similar competency maps for other professional competencies, effectively embedding the ALPS approach across other Faculties and professions. For example the Enterprise CETL at Leeds Metropolitan University wanted to facilitate the development of an enterprise skills map to enable staff in the Faculty of Business and Law to develop modular content and assessment tools which can be linked to existing enterprise frameworks and also allow for consistency in terms of assessing enterprise attributes across the wider University.
The success of the competency mapping process depended on effective consultation across and within the professions during development and on completion. Each HEI ran consultation workshops for academic staff, students and service users to encourage feedback on the language, structure and usability of the maps. ALPS also consulted with the Professional, Regulatory and Statutory Bodies regarding the concept and content of the maps to gain their approval. As a result of this process a number of changes were made and the graphical representation of the maps was changed to a wheel format rather than the original linear presentation making the maps easier to navigate and more user friendly. This spherical presentation also eliminated hierarchy within clusters which had been a concern at the consultation events.
Development of the common assessment tools
The maps have been used to generate an assessment toolkit to increase the range of formative and summative assessments which can be used within and across the sixteen professions. Traditionally, H&C professions have tended to require a member of their own profession to assess a student’s competency. ALPS challenged this principle by suggesting the common competencies, can be assessed by another profession, as long as the workplace assessor has been appropriately briefed.
The ALPS assessment tools were simulated using students, service user representatives, tutors and practitioners at a series of workshops to test the acceptability of the language and usability of the mobile devices prior to being piloted with students.
So far five assessment tools have been developed and accepted for use in practice assessment scenarios:
- Gaining Consent,
- Providing information to a colleague,
- Knowing when to Consult or refer,
- Demonstrating respect for service user during an Interaction and
- Working Interprofessionally.
The ALPS assessment scenarios can be used at different stages of an individual student’s career and also by different professions at different levels of their academic development. For example, the Dental Hygiene & Therapy students used the “Gaining Consent” for peer feedback in their second year paediatric placement, in order to enhance their feedback skills and reflective skills (Norcini, 2003). The Audiology students used the same tool for their level three students in general clinics whilst in practice.
The approach to improving competence of students is to build on the Boud (2000) theory of sustainable assessment. Students are encouraged to take feedback from a variety of sources, reflect on that feedback and deduce further action to improve performance. This student activity is predominantly reflection in action (Schon, 1995) and enhances the richness and quality of the students’ reflections thus developing life-long learning skills. The ALPS common assessment tools were developed using agreed best practice from the different professions involved in the ALPS programme. For example; Social Work students already gain feedback from service users at some time during their practice placement experience. This concept is being considered in the Nursing profession (Speers, 2008). The Dietetic students are used to reflective assessment tools which encourage feedback from a variety of sources as well as self evaluation and action planning to improve their practice. The ALPS common assessment tools reflect these principles with the potential to deliver formative feedback from a range of potential participants in the assessment scenario. These include the practice educator (from their own or a different profession), peers (from their own or a different profession), service users (and carers) and self. Feed forward is a key outcome of the process and there is provision in the tool for students to reflect on the feedback they have been given and to develop an action plan which can be signed off by a practice educator.
Developing the mobile delivery of the common assessment tools
The decision to “Go Mobile” was made to allow the provision of “any time, any place” access to the common assessment tools, learning materials and tutor support to enable students to gain maximum benefit from the learning opportunities on offer to them whilst on placement. ALPS commissioned from its commercial partners, ecommnet MKM and T-Mobile the procurement of a shared mobile services platform (MSP), mobile devices and connectivity along with software tools. Development of the MSP was informed by the ALPS mobile learning pilots (Dearnley et al 2009) and the outcomes of the Mobile Enabled Disabled Students (MEDS) project (Dearnley et al, 2010). ALPS is a large scale implementation with a 1000+ user and is supported by a shared ALPS Helpdesk based in Learner Support Services at the University of Bradford, which provides support to the five HEIs for device roll out and gives advice to users across the whole of the ALPS programme. Cohorts of students from the partner sites were supplied with HTC Vario smartphone PDAs and were given unlimited free data connectivity on the T-Mobile network.
Figure 2 The ALPS Assessment Cycle
The ALPS assessment Cycle is depicted in Figure 2. The ALPS MSP allows the common assessment tools to be securely sent out to the students’ ALPS devices and any completed assessment tools can be uploaded securely from their devices to a password protected web based e-Portfolio. Students are able to collect timely 360 degree feedback which can be captured in a variety of formats including images, audio, video, as well as text, supported by the predicted text and spell check function. This has meant that the devices have been found particularly useful by students with dyslexia (Dearnley et al, 2010). The uploaded assessments can be accessed by academic tutors (back at base in the university), via the e-portfolio, to allow them to monitor students’ progress and provide an additional source of feedback.
Confidentiality was secured by requiring students to sign a “Contract of Use” drawn up, with legal support, jointly with the five ALPS partner universities. The contract had a number of governance purposes including; responsibility and fair usage of the device particularly advice on appropriate use of the camera function, ownership of the device and where to go for help. Students carried ID cards to confirm their device was being used for assessment and learning activities and all managers were briefed about the work of ALPS. Posters were sent to NHS Trusts to display in placement areas to inform staff and service users about the ALPS Programme.
The current Government policy promoting the implementation of interprofessional education and training for H&SC students (Craddock et al, 2006) has necessarily had to encompass elements of IPL. The introduction of mobile interprofessional assessments was a huge learning curve for staff and students on many levels. ALPS not only had to provide technical training on how to use the mobile devices and all the associated assessment and e-portfolio tools but also there was a significant change in practice for some professions with the introduction of 360 degree interprofessional assessment and reflective learning practices.
Many students from the majority of professions involved were not initially confident in peer assessment practice. Other students were unconvinced by the idea of gathering feedback from service users using the mobile device; some social work students for example saw it as a potential barrier to communication. Most of the professions involved in ALPS were familiar with the concept of self reflective practice and so many could see a link with the self assessments and their placement diary. All students seemed to accept the idea of practice assessor assessment, probably as this does not signify a particular change to current practice. However whilst the majority of students could relate to the self and practice assessor assessments, they did not automatically see them as adding value to their placement. A large part of training consequently concentrated on linking the assessments to their current placement practice and illustrating the benefits of completing ALPS assessments.
The potential of the ALPS approach to assessment and learning in practice settings
The mapping of key skills is a powerful way of enabling students and staff to navigate their way through professional competencies in a user friendly, interactive format which can be used to develop a deep understanding of the expectations of the practice environment and allow for consistency when translated to assessment tools in terms of evaluating knowledge, skills and attributes across and within professions.
The process initially established within ALPS to expose the key skills and attributes inherent in communication, teamwork and ethical practice has now been extrapolated to develop a new enterprise skills map and a patient safety map to be used by students and staff to enhance learning and employability. The ALPS competency mapping process was equally relevant to make explicit enterprise skills in the curriculum at Leeds Metropolitan University. The Institute for Enterprise CETL supports and promotes enterprise education within the University and with key external national partners. Its small team has strengthened and developed enterprise education in the last 5 years through development of new courses and modules, collaborative projects and student led events. We anticipate that representing the enterprise dimensions in a competency map will allow students to grasp the diversity of the competency and help them to identify their strengths and areas for improvement. Additionally as part of the NHS Yorkshire and the Humber funded ALPS CETL extension project, Leeds Metropolitan University have secured financial support to set up a Competency Mapping Network whose primary objective is to produce a Patient Safety Competency Map. The network which is led by Catherine Coates and Julie Laxton has attracted many local experts in the field of patient safety and aims to complete this new ALPS map for a consultation event in May 2011.
The ALPS processes were designed to provide reliable, standardised, interprofessional assessments in practice settings. This enables opportunistic assessment scenarios to be fully realised and to include the full range of players in the assessment process, providing effective feed forward so enabling students to improve their practice competence. The cohort management aspects of the ALPS e-portfolio tool allow tutors to comment on the students’ assessments and provides the opportunity for the tutor to have an overview of the student’s developing capabilities whilst back at base in the university. This will allow them to make informed choices regarding which students need early visits that has the potential to reduce the number of placement visits required, thereby reducing costs.
The whole ALPS journey has allowed tutors, practice educators and students to better understand one another’s practice models and competency frameworks and the potential for further development of common assessment tools is evident.
As with the introduction of any technology enhanced learning process there has been an initial reticence to embrace this ‘mobile’ assessment process fully, however the benefits of the approach are widely recognised. Most of the ALPS professions still manage paper driven assessment processes but the ALPS electronic and mobiles assessment tools present the possibility of changing entrenched practices and cultures. The technological innovations recommended within the NHS modernisation agenda will be necessary to support this paradigm shift.
References
- Boud, D. (2000). Sustainable Assessment: rethinking assessment for the learning society. Studies in Continuing Education, 22 (pp 151-167).
- Coates, C. A.; Laxton. J.; Frith, G. (2008). Collaborative Working, Production of Inter-Professional Common Competency Maps. SOLSTICE Conference, Edge Hill University. 5 June 2008.
- Dearnley, C..A.; Taylor, J. D.; Hennessay, S.; Parks, M.; Coates, C. A.; Haigh, J.; Fairhall, J. R.; Riley K.; Dransfield, M. (2009). Using Mobile Technologies for Assessment and Learning in Practice Settings: Outcomes of Five Case Studies. International Journal on E-Learning 8(2), (pp 193-208).
- Dearnley, C.A.; Coulby, C.; Rhodes, C.; Taylor, J. D.; Coates, C.A. (2009). Service users and carers and mobile devices; Interprofessional practice assessment: joyful mix or a step too far too soon? Innovations in Education & Teaching International.
- Dearnley, C.A.; Walker, S.A.; Fairhall, J.R. (2010). Accessible Mobile Learning: Exploring the Concept of Mobile Learning for All; in Bromage A., (ED) Inter professional E-Learning and Collaborative Work: Practices and Technologies. IGI-Global.
- Craddock, D.; O’Halloran, C.; Borthwick, A.; McPherson, K. (2006). Interprofessional education in health and social care: fashion or informed practice? Learning in Health and Social Care, 5(4), (pp 220-242).
- Holt, J.; Coates, C. A.; Eastburn, S.; Laxton, J.; Mistry, H.; Young, C. (2010). Identifiying commom competencies in health and social care: an example of a multi-institutional and interprofessional working. Nurse Education Today Vol. 30, Issue 3, (pp 264-270)
- Norcini, J. J. (2003). Peer assessment of competence Medical Education. Medical Education 37, (pp 539-543).
- Schon, D.A. (1995). The reflective practitioner: How professionals think in action. Arena: Aldershot.
- Speers, J. (2008). Service user involvement in the assessment of a practice competency in mental health nursing-the stakeholders’ views and recommendation. Nurse Education in Practice 8, (pp 112-119).
- http://www.alps-cetl.ac.uk/maps.html
- http://www.leedsmet.ac.uk/enterprise/resources/