Activity design in online professional development for university staff
Janet Macdonald [j.r.macdonald@open.ac.uk], Anne Campbell
Open University in Scotland [http://www.open.ac.uk],
10 Drumsheugh Gardens,
Edinburgh EH3 7QJ, UK
Abstract
Professional development for distance tutors is traditionally delivered through texts or websites, supplemented by face to face workshops. However, one of the major challenges is to ensure that they engage with the materials and carry away something which is of value to their practice. At the Open University (UK) our 8000 tutors work from home, and have few opportunities to meet fellow tutors on the course that they tutor. We have been exploring the use of online activity design to support tutors in learning by undertaking activities in online communities. This provides opportunities for professional development and discussion with their peers, while at the same time offering some flexibility in participation.
This paper describes a case study of an award winning initiative at the Open University (UK) which now delivers professional development in online communities of distance tutors, operating at scale: some 2000 tutors have been trained to date, and our courses have a 75% completion rate. Drawing on data from 370 recent participants, it discusses some of the lessons we have learnt on the reasons for the widespread success of this initiative and the factors influencing effective activity design on the course.
Keywords
Activity design, distance tutors, online learning, professional development; engagement
List of topics
- Introduction
- The Open University
- Tutor Moderators course
- Methods
- Findings
- Participant perceptions on the course
- Participation levels and message base
- Conclusions
- Acknowledgements
- References
Introduction
Traditionally the professional development of distance tutors is delivered through texts or websites, supplemented by face to face workshops. However the challenge is to ensure that tutors engage with the materials, and carry away something which is of value to their practice, particularly when they are working away from a conventional classroom, with a host of competing priorities. We have been exploring the options for encouraging tutors to undertake professional development in online communities which can provide opportunities for discussion and peer support, while at the same time offering some flexibility in participation. According to Wenger (1998), communities of practice are groups of people who share a concern or a passion for something they do and who interact regularly to learn how to do it better. Although used originally in the context of professional and social practice in 'real-world' settings, the concept of the community of practice is now increasingly used in relation to virtual environments (see for example Kimble et al 2008), and has obvious application for professional development initiatives. In the field of academic development, although not necessarily online, Boud (1999) has underlined the significance of approaches which are of obvious relevance to staff when they are situated or embedded within the context of staff working practices, and peer learning has clear benefits in this regard.
Alongside a growth in interest in online communities has been a rise in activity design, an approach which replaces course content with a series of learning activities. The underpinning constructivist pedagogy starts with the premise that people learn better by doing, and by sharing their understanding with their peers. The way in which they do this is related to a variety of factors, including their individual context: the experiences they have had in the past, their motivation and the purposes to which they intend to put the new knowledge. Biggs' (2003) influential model of constructive alignment suggests that an effective way of getting them to engage is to devise activities in which they have to take an active part and ensuring that this activity is aligned with assessment strategy. This thinking has acquired a new relevance in online environments, and it underpins the principles of learning design (Britain 2004). At the Open University (UK) activity design has been in use for some years on a few undergraduate courses (see for eg Macdonald & Twining 2002) and is now widely in use: a recent initiative to share activity design amongst staff and across faculties as part of the Open University (UK) Learning Design project has resulted in a database of 44 case studies. We describe here its adoption to support the learning of staff in a major online professional development initiative.
One of the practical challenges to effective activity design which makes use of interaction in an online community is in achieving a balance between developing a community with "buzz", where participants have a sense that they belong to an active group where they are likely to read recent messages from fellow participants; and avoiding overload with an excessive number of messages to read. On achieving engagement there has been considerable debate in the literature. For example, Rovai (2000) describes eight factors which impact on a sense of community in an online group including the student – instructor ratio, transactional distance, social presence, lurking, social equality, collaborative learning and group facilitation. Caspi et al (2003) found that group size was a major determinant regarding levels of interaction in large non-mandatory discussion groups, and the nature of interactions changed with group size, with larger groups having greater potential for learner-learner interaction. While Macdonald & Twining (2002) demonstrated that small online groups have little chance of viability unless activity is framed within the assessment strategy, Holliman & Scanlon (2006) report some success in sustaining the viability of small groups using near-synchronous activity, where students must complete activities in a forum within a fixed time frame. They use the term "near-synchronous" to describe activity which has some of the flexibility of an asynchronous email exchange or forum, but for which that flexibility is restricted to contributions over a few days. By restricting flexibility in this way there is increased likelihood that participants will encounter recent postings from fellow participants.
There are implications for estimating or anticipating workload in activity design which involves online interaction. While there are fairly well established procedures for estimating reading speeds for content whether paper or web based, the workload for activities which depend on interaction with a community is problematic to estimate, because it depends on the levels of engagement of other individuals in the group. Macdonald (2008) has illustrated the minimum size of message base (the total number of messages), calculated from the number of participants in the group, the number of activities and the duration of the activity, with obvious consequences for workload (see Figure 1). So for a group in which one activity has to be completed over a period of four weeks the workload per week will be different to a group in which the activity has to be completed over four days. And a large group of 50 will produce more messages per activity than a small group of 5, with consequences for everyone's workload. This illustration shows how the period of near-synchronous activity might need to be balanced against the number of participants and the nature and number of activities. Of course a calculation of minimum message base assumes that participants will only do the minimum required to complete the course.
Figure 1. Illustrations of minimum message base
This paper describes a case study of an award winning initiative which set out to provide professional development using activity design in online communities of distance tutors operating at scale, providing a service to thousands of staff. It discusses the environment for distance tutors who work at the Open University (UK) and then goes on to describe the course structure. It then discusses participant perceptions on the reasons for the success of the course and finally the factors influencing participant workload.
The Open University
The Open University (UK) employs 8,000 part-time tutors who act as the human interface between the university and its students. Tutors work from home, and rarely have the opportunity to meet fellow tutors on the course that they tutor. All tutors must have access to the internet for administrative purposes and for supporting learners. They are increasingly required to make use of the new Virtual Learning Environment (VLE) for both student support and tuition and the electronic assignment submission system.
In terms of professional development, opportunities for conversations on course teaching and the exchange of good practice commonly take place in an initial face-to-face briefing for tutors at the outset of a new course, with a debriefing after the first year on some courses. In addition tutors are encouraged to attend professional development events which provide course or faculty-specific workshops alongside more generic sessions. Online communities are already enthusiastically embraced by tutors on some courses, where a staff forum caters for ongoing discussion as the course proceeds, and they can be particularly significant during assignment marking, when tutors may use the online group to discuss their understanding of the question, or of marking criteria. In this case, peer support is highly valued as it is closely tied to the specific context of the course. It is not surprising therefore that core duties associated with tutoring students drive the habitual use of these groups, and make a discussion of other aspects of teaching more likely to occur (Macdonald & Hewling, 2008).
The challenge is how to create the conditions for an online community which could discuss more generic aspects of online facilitation and which tutors will find sufficiently attractive and engaging to persevere with.
Tutor Moderators course
Our most successful innovation in online peer communities for professional development is the course Tutor Moderators, which provides a three-week introduction to the facilitation of online groups. Developed at the Open University in Scotland in 2004, it has become standard provision for tutors on Level 1 courses (that is undergraduate entry level) and is delivered at scale to staff throughout the University, indeed a total of 1500 staff had undertaken the course by April 2009. Reflecting a diversity in use of platforms and a concern from Faculties to provide induction for their staff of direct relevance to their needs, the course has been run on two platforms, on First Class and also our Moodle VLE. With the adoption of Elluminate within our VLE, we are currently reversioning the course so that it will include the facilitation of online groups using both asynchronous and synchronous tools. The course has proved to be a model of online professional development which works, and which promotes engagement through activities and reflection with a peer community.
Since the course operates at scale, with cohorts running concurrently we have had to adopt ways of automating our administrative systems. We operate a "fill-up-and-go" web based booking system which offers staff a choice of course dates, automatically collates their details and fills each course with a maximum of 25 participants, thereby modelling the size of a typical Open University tutor group.
The course has minimal content and relies heavily on experiential and collaborative learning within a community. In a choice of three out of five simple activities per week, participants are required to undertake a task and leave a message reflecting on their actions in the course forum. Activities range from reflecting on the experience of being a newcomer in an online group, to experimenting with functionality of a tool, or discussing approaches to online facilitation and coping with shy or dominant students. Participants learn by trying out the tools, reading input from their peers or observing the actions of the course moderator.
The course has an automated self-certification system where participants check off a record of activities as they are completed; on finishing the course the checklist generates an automated certificate of completion, and compiles our database of completion records.
In the week before course start, participants are sent a reminder which includes advice from previous participants, with an invitation to engage in planning personal objectives for the course and also to consider time management.
The student as "identifiable individual" is a central premise of tuition and support at the OU: it means that in spite of the scale of the university, every student is known to their tutor, and if they do not submit an assignment, or appear in an online forum as expected, their absence is noticed and followed up. We use this principle on Tutor Moderators course to optimise completion rates, if participants have signed up for the course and do not join within a couple of days, then we get in touch with them. If at the end of the course a participant has not completed, then we write again, allowing them an extra two week's grace to complete, with a reminder to complete the activity checklist. Course moderators are required to give strong pastoral input in week 1 by responding initially to each individual, but to stand back somewhat in weeks 2 & 3.
The next part of this paper assesses the extent to which this design works in practice. We set out to investigate two areas, with the following research questions:
-
What are participant perceptions on the factors contributing to the success of the course?
-
What is the size of message base (number of messages posted) each week for this particular activity design, and how does this relate to estimates?
Methods
A total of 600 participants in 34 cohorts completed the course between June 2008 and March 2009 (see Table 1). The cohorts were sponsored by a variety of Faculties, in response to the urgent need to train tutors for new courses coming on stream. Courses were also sponsored by regional or national centres of the Open University, which are responsible for organising continuous professional development for staff in their geographical area.
Table 1. Percentage completions by Unit Jun 08 – Mar 09
Completions Jun 08-Mar 09 |
Percentage |
|
Regions/Nations |
161 |
27 |
Faculties |
439 |
73 |
Totals |
600 |
100 |
In order to address the two research questions we have drawn on two sets of data. For participant perceptions of factors contributing to the success of the course we made use of quantitative and qualitative data from the exit questionnaire. This questionnaire was completed by 373 participants (a 62% response) in the period Jun 08 – Mar 09. Quantitative data on competence and confidence was automatically collated in the database which stores this exit data. The qualitative data was taken from responses to the questions: "In what ways has the course helped you as a moderator?", and "What improvements might be made to the course?". Through iterative reading of the responses we were able to establish a number of categories which describe the main factors influencing satisfaction, and those needing further consideration. These categories are described here and illustrated with quotes from participants.
For the second part of this study, which set out to establish the size of message base, that is, the number of messages which are posted each week, and the implications for workload, we undertook a quantitative study of levels of message base for each week, averaged over 18 cohorts for courses which were run on the same platform. Of the 34 cohorts which ran between June 2008 and March 2009, 18 cohorts were run on our Moodle VLE. These 18 cohorts involved 354 participants who contributed 7834 messages together with 12 moderators who contributed a further 2384 messages. We compared this message base with an estimate of the minimum size of message base, and discuss some of the reasons for the discrepancy.
Findings
Participant perceptions of the course
Quantitative data from the exit questionnaire has given us a graphic illustration of participants' perceptions with respect to a growing confidence and competence (see figures 2 and 3)
Figure 2. Perceptions of confidence before and after the course
Figure 3. Perceptions of competence before and after the course
While personal perceptions of competence are inevitably fairly subjective, the figures certainly illustrate a change in participants as a result of the course and we note the high proportion of complete novices (51%) and of those who said they were either terrified or anxious of moderating an online group (62%) before they did the course. It seems probable that with an increase in confidence alone they will be better equipped to experiment for themselves, and be more prepared to try out new functionality or new approaches to supporting their online group.
A number of factors have contributed to the success of this online course. Perhaps the first issue is the motivation of participants, who are eager and often anxious to learn about the support of online groups because of the new duties associated with the increasing use of online media for supporting students, and the steady adoption of VLE tools for teaching, as new courses come on stream. The course has an average completion rate of 75%, which suggests that participants are highly motivated to complete it.
Indeed we note that 60% of completions were associated with the Faculties of Arts, Health and Social Care and Social Science, which have just recently begun to adopt online technologies for supporting their students: they probably belong to what Rogers (1995) refers to as the "late majority" in terms of adoption of technology for working practices, and do not have a long tradition of belonging to online communities. This is reflected in the low levels of confidence and competence of tutors before completing the course.
In the same exit questionnaire, respondents were asked in what ways the course had helped them: this section draws on the qualitative data we have collated in response to this question. Around half of the responses refer to the design of the course, in terms of the value which participants attach to the experience of learning by doing, in the opportunity to engage in reflective practice, and in belonging to an online community:
There's nothing like learning by doing. This was my first experience of an online forum so doing it was great
the most important learning came from being a participant.
It has made me aware that I can be quite long-winded - aim to cut down - overlong messages can be intimidating! It's been reassuring too, in that for instance, my use of the telephone to try and tempt reluctant students to go online/try contributing, is something others seem to do too.
Because we recruit cohorts of participants to work together, the groups can develop an identity and sense of community over the three weeks. They have the opportunity to share ideas and best practice with their peers, not only learning from those who are experienced, but also sharing the realisation that not all problems have easy answers.
I felt part of a community and have been able to benefit enormously from the comments of all colleagues, many of whom speak from experience
Good discussion about problems I was having. Lots of very good ideas and discussion from tutors facing the same problems and issues that I am facing
I've also learned a lot from other colleagues about what doesn't work
Clearly the affective issues associated with online facilitation were of pressing interest. Many referred to the confidence building which comes from being part of a safe environment, and discovering that by joining an online community they were no longer alone in their concerns and anxieties. For Faculty sponsored cohorts, there was added currency and relevance in belonging to a group with common interests and approaches to teaching.
It was also a great way of feeling connected with other tutors, who were experiencing the same anxious moments as myself
It has been great to feel less isolated. I no longer feel my students are weird
I also realise I'm part of a large group of moderators many of whom share my excited apprehension about the task in hand
Many also commented on the opportunity to reflect on their actions and feelings associated with being a newcomer in an online community, and of recognising how their students might feel in the same circumstances. There were implications for ways in which they might help their own students more effectively.
You gain from the collective experience of colleagues and you learn new skills. You know how it feels when your tutor replies with enthusiasm to your posting and when a forum member responds to your ideas positively
Also, having experienced the forum as a student has been really helpful : I should be able to empathise with the participants and I am aware of how difficult it is to take the initial plunge and join in.
I've really learned a lot about the role of the forum moderator as someone who encourages, actively participates (Bert, your involvement has been a fantastic example of what I should be doing!)
Increased confidence, skills, knowing where to go for information. its been a helpful supportive resourceful on line community.
The near-synchronous approach, created by contributions which must be made within each week gives participants limited flexibility over a restricted period, and has helped to develop a sense of presence. The focus which is lent to participation within a limited timeframe means that participants have to set aside the time to concentrate on professional development which otherwise can be overtaken by more pressing tasks.
… the activities make you think but the course still lends itself to being fitted into the kind of time you can devote to it, whether that is short bursts or a long run
When asked what improvements might be made to the course, while the majority found the limited flexibility fitted well with other commitments, a significant minority felt the course might be close to their limit in terms of workload, particularly in week 1.
I've been rushing it a bit I think.
I sometimes had the impression of everyone talking at once at a very noisy party! The number and enthusiasm of the participants meant that a great deal of reading was generated
In summary, participants value the course because it not only gives them the opportunity to try out new tools and discuss their use, but also because it allays fears and gives them a chance to understand what it feels like to be an online student. While the near-synchronous design has been successful in creating a sense of presence, there is inevitably a balance to be struck between engaging staff in near-synchronous activity, and in overloading them with an excessive message base.
In the next section we set out to establish to what extent one can estimate message base when planning workload, and what factors should be taken into account.
Workload, participation levels and message base
Although course activities require participants to undertake a variety of tasks involving various tools, they are asked in each case to reflect on their experience in the course forum, therefore the size of message base is a realistic reflection of participation levels: and this inevitably has an impact on workload, because the more messages there are, the more time is spent in reading them. The larger the size of group, the larger the workload, if participants are expected to read each others' messages as well as post their own. And inevitably the number of activities in which participants are required to engage will also impact on the message base.
Using Macdonald's (2008) illustration (described in Figure 1) we worked out the anticipated minimum size of the message base, that is, the minimum number of messages which might be posted in each week. For each cohort the average number of participants actually starting Week 1 is 21; and by Week 3 it is 16. For both weeks we estimated the message base on the assumption that each participant makes 6 contributions over 5 activities for week 1 and 7 contributions over 5 activities for week 3. We then estimated variable levels of moderator input: at 30% of participant input for week 1, and 15% for week 3, in line with their brief to provide a decreasing level of support after week 1.
Table 2. Estimated minimum message base for weeks 1 and 3
Week |
No. Participants |
Min messages contributed |
Participants message base |
Moderator input |
Total est. message base |
1 |
21 |
6 |
126 |
38 (30%) |
164 |
3 |
16 |
7 |
112 |
17 (15%) |
129 |
We then set out to compare these estimates with actual activity, The distribution of the message base averaged over 18 cohorts is illustrated in Figure 4.
Figure 4. Average message base from 18 cohorts of Tutor Moderators
The figures show that in each week the message base was considerably in excess of the estimated minima. We can also see that perceptions of overload in week 1 are borne out by evidence of a considerable message base in that week (286 messages), which is almost double the size of message base in week 3 (148 messages), although the number of activities requiring a posting to the forum was the same in each case. Having recognised the influence of group size which was largest in week one, the following paragraphs describe other factors which may have contributed to the size of message base.
The forums for each week contain a large number of message threads, and those related to course activities are marked with the number of the relevant activity. It has therefore been possible to identify responses to course activities from additional conversations, of which there were many. For example, the data from weeks 2 and 3 showed that additional threads or conversations constituted 20% of total messages in week 2; and 11% of total messages in week 3. The additional conversations covered issues related to the course material, as well as incidental discussion on other topics related to work with the OU, illustrating the value which staff attach to this opportunity to talk to each other, but which considerably adds to the workload of both participants and moderator if these messages must be read. The data for week 1 is ambiguous because in one activity participants were required to experiment by initiating additional threads: so it was impossible to distinguish these activity related threads from incidental conversations. We can only surmise that the excessive message base in week 1 was probably the result of initial enthusiasm, compounded by the invitation to start new discussion threads as part of one of the activities. Finally, we note that it is possible that the functionality of Moodle forums could be encouraging this "divergent" behaviour with incidental conversations, since it is very simple for any participant to initiate new threads, and new users might in fact be doing so when they are not familiar with forum protocol to respond to existing threads.
Support from the course moderators also added to the total size of message base, and they contributed more in week 1 than in subsequent weeks: 34% of the total message base in week 1 in contrast to 19% in week 3, which was slightly higher than our estimates of 30% and 15%. We ran a correlation of total number of messages per individual moderator over the three weeks with the total number of participant messages, but found no association between the level of moderator messages in the three weeks and the level of participant messages. However it seems likely that their encouragement in the first week had an impact on the atmosphere of the group and the extent to which participants felt confident enough to contribute and respond to each other, and that is consistent with previous work (see for example Salmon, 2004)
In summary, while it is possible to estimate minimum levels of participation and size of message base, our observations in this study have illustrated the extent to which these levels can be exceeded in practice. In particular, we have illustrated how the motivation and enthusiasm of part-time distance staff who may otherwise have little opportunity to discuss their work, can have a dramatic influence on participation levels, particularly where the discussion is situated within their working context, with peers in a similar situation and with experience of direct relevance to the job.
Conclusions
We believe this case study illustrates a remarkable success story of online professional development provision at scale for part-time tutors at a distance. We have demonstrated the value of a near-synchronous strategy which enhances a sense of presence while providing sufficient flexibility to accommodate part-time working. The activity-based approach combined with the opportunity to discuss online teaching is valued by tutors as it provides a safe environment in which to try out new techniques and tools and to reflect on what is a pressing concern for many tutors at our institution. The activity checklist and certification scheme have provided a framework for engagement and probably contributed to high completion rates.
At the same time our observations on the size of the message base underline the tensions in designing activity which is of an appropriate workload for the target market, and in balancing that tension against the need to provide a degree of limited flexibility and focus. Somewhat unusually we have demonstrated the influence of participant enthusiasm, and the interest and relevance of particular themes and activities, in driving high levels of participation and engagement.
The findings do support many studies of student participation, where engagement in small online groups has been minimal and rarely successful unless aligned with assessment strategy (see for example Macdonald & Twining 2002). In the area of professional development similar problems have been reported with participation and engagement (Kimble & Hildreth 2008). Our intention in requiring participants to contribute to a minimum of three out of five activities in each of three weeks, underpinned by the activity checklist and self certification system was that this would effectively work as an informal assessment strategy, aligned with the activity design. While some work has highlighted the danger of encouraging a strategic approach by tying activity to assessment there is no evidence in this study to indicate that participants were driven to strategic participation by the prospect of a certificate, although clearly the 75% completion rate indicates that most followed the checklist and completed the required activities. We therefore feel justified in the adoption of this particular aspect of the design.
This study has illustrated how online participation and engagement for professional development need not be problematic if the activities are sufficiently relevant and core to the job so it is particularly gratifying to be able to report on a success story.
References
- Biggs, J. (2003) Teaching for quality learning at University. Buckingham: SRHE/OUP, 2nd edition.
- Britain, S. (2004) A review of learning design: concept, specifications and tools. A report for the JISC Elearning Pedagogy Programme http://www.jisc.ac.uk/publications/reports/2004/learningdesignfinalreport.aspx (accessed 1st Feb 2010)
- Boud, D. (1999) Situating academic development in professional work: using peer learning. International Journal of Academic Development 4 (1) 3-10
- Caspi, A, Gorsky , P & Chajut, E. (2003) The influence of group size on non-mandatory asynchronous instructional discussion group. Internet and Higher Education 6 227-240
- Holliman, R. & Scanlon, E. (2006) Investigating cooperation and collaboration in near synchronous computer mediated conferences. Computers and Education 46 322-335
- Kimble, C. Hildreth, P. and Bourdon, I. (2008) Communities of Practice: creating learning environments for educators, Charlotte, North Carolina: Information Age Publishing
- Macdonald, J. & Twining, P (2002) Assessing activity based learning for a networked course. British Journal of Educational Technology 33 (5) 603-618
- Macdonald, J. (2008) Blended learning & online tutoring. Planning learner support and activity design. Aldershot: Gower, 2nd edition.
- Macdonald, J & Hewling A (2008) Exploring the potential of online communities of practice for distance tutors. In: Communities of Practice: creating learning environments for educators, by Chris Kimble Paul Hildreth and Isabelle Bourdon (eds) Charlotte, North Carolina: Information Age Publishing
- Rogers, E.M. (1995) Diffusion of innovation. New York: Free Press, 4th edition.
- Rovai, A (2000) Building and sustaining community in asynchronous learning networks. Internet and Higher Education 3 285-297
- Salmon, G. (2004) E Moderating: the key to teaching and learning online. London: Routledge 2nd edition.
- Wenger, E. (1998) Communities of Practice: Learning, Meaning, and Identity. Cambridge: Cambridge University Press.
Acknowledgements
My heartfelt thanks go to Anne Campbell, Catherine Scanlon and Clive Buckland for their sterling work in collecting and analysing the data on which this paper draws. I would also like to acknowledge the contribution of the Tutor Moderators team, who have worked so hard over the last few years to make the course so successful. They include: Anne Campbell, Fiona Barnes, John Berriman, Peter Blackledge, Charlotte Brownhill, Clive Buckland, Pat Crawford, Gill Duncan, Paul Easterbrook, Sarah Falcus, Gill Farnen, Andrea Gibson, Jean Gordon, Julie Hirst, Hong Yu, Dave Horan, Peter Hull, Sybella Kirkbride, Vicki Lawson-Brown, Pat Lanham, Rae Simpson, Keith Wakeman