Can Video Software Support Assessment and Feedback for Teachers Engaged in Distance, Postgraduate Programmes?
Scott Farrow [farrows@liverpool.ac.uk], University of Liverpool, Liverpool L69 3BX [http://www.liverpool.ac.uk], Anne McLoughlin [Mclougha@edgehill.ac.uk], Edge Hill University, St Helens Road, Ormskirk, Lancashire, L39 4QP [http://www.edgehill.ac.uk], United Kingdom
Abstract
This study aims to explore whether teachers on a postgraduate, distance learning programme can be supported by video technology. The research looks at assessment and feedback around the students’ lesson observations, an aspect of practice undertaken across a wide range of courses within higher education (HE).
Semi-structured interviews were used with students and tutors for data collection and thematic analysis for analysis. Themes from the analysis include “Learning and Teaching from a Distance, Observation Feedback Electronically, Technology Anxiety, Observation Anxiety and Critical Reflection”.
The study outlines the strengths and weaknesses of using video software, as opposed to face-to-face observation of practice, but highlights key pedagogical benefits that come with the use of electronic submitted video observations.
Keywords: Distance learning, higher education, assessment, adult learners, education technology, video observation
Introduction
Distance learning is increasingly popular, particularly with students who are in employment. In addition many have family commitments and having to physically attend a university at the same time, every week can cause great difficulties for them (Vryonides & Vitsilakis, 2008). MacDonald (2008) believes that the majority of students accessing distance learning in their study, are mature learners who study off campus, work independently and have access to the university network. However, they may need to attend face to face sessions.
Ramos (2011) and Burgess (2008) argue the difficulties part time students have in academic study compared to their full-time counterparts; they often underperform in assessments, require more extensions, generally have poorer attendance and express more anxiety regarding the expectations of them. McGinnes (2011) considers the issues an institution may have trying to address the needs of mature students, such as delivering evening sessions. However, these bring their own difficulties as the face-to-face time is reduced as evening sessions have to be shorter, so content has to be condensed to fit in to the time frame, which reduces the learning opportunities and potentially increases students’ stress levels.
Taking a distance learning approach is a possible solution to help non-traditional students cope with the demands of work, a home life and their studies (McGinnes, 2011; Ramos, 2011; Burgess, 2008; MacDonald, 2008). Selwyn (2010) suggests that e-learners may have access to a wider range of opportunities and greater freedom of choice but goes on to highlight that this only occurs when students have access to the appropriate technologies and the ability to use them.
A popular concept is that traditional students are now considered digital natives and mature students and members of staff are digital immigrants (Prensky, 2001). Prensky discusses the natural abilities of the digital natives and the difficulties the digital immigrants encounter. However, Salajan et al. (2010), Margaryan et al. (2011) and Guo et al. (2008) found in their studies that the digital natives were very competent in a limited number of technologies, and had little advantage over the digital immigrants in using technology to enhance their learning. Wolsey and Grisham (2011) use another term; digital tourist as someone who experiments and investigates the wide variety of technologies available and look to “incorporate the best of the experience into their thinking and instructional approaches” (Wolsey & Grisham, 2011). This seems the best option for students, to limit fear of failure, by just looking to explore what will support them through their learning.
The need for tutor engagement is highlighted in numerous studies (Salmon, 2011; Dixson, 2012; MacPherson & Nunes, 2004). MacPherson and Nunes (2004) debate specific requirements for teaching in an online environment. They emphasise the need for greater flexibility; as tutors cannot confine their work to specific times, they need to adapt their teaching style, be involved in all conversations and be enthusiastic in promoting active, meaningful activities to engage students in the learning process. While this may be asking a great deal, it is what students want (Salmon, 2011; Paechter et al., 2010).
Classroom Observation
Observation of classroom practice is a fundamental concept of many teaching programmes. O’Leary (2014) puts forward the view that it originated in the mid nineteenth century as Her Majesty’s Inspectorate began to observe teachers to assess if they were value for money and if they demonstrated practice which would benefit others. Lahiff (2015) emphasised that learning through practical experiences promotes an in depth understanding and O’Leary (2014) affirmed this. He highlights how the observer can nurture key pedagogical skills to promote the development of effective pedagogical practices. In order for this to be effective, it is important that a supportive relationship is developed between the observer and observed (Maxwell, 2010; Ollin, 2009; Cockburn, 2005). This allows an open and honest dialogue between them which promotes constructive feedback and discussion (Orr & Simmons, 2010). This is essential as Cockburn (2005; p.373) raises the issue that there may be a feeling of an “infringement of professional rights” when professionals are faced with observations. O’Leary (2012) expresses that issues such as these arise when teachers feel that observations were imposed on them rather than a positive, supportive process which assists their continuous professional development needs. The professional dialogue following an observation should be used as a constructive means to develop a reflective approach which enables both the teacher and observer to interrogate practice, discuss difficulties and possible adaptations to address these, as well as celebrating good practice (O’Leary, 2012; Cockburn, 2005).
The reflective process can be used to enhance professional development experiences as Baecher and Kung (2014) advocate a variety of approaches being employed to add depth to the analysis of one’s own practice and design new methods to enhance this. The use of video to support this process was investigated by van Es and Sherin (2010) and found to be effective in changing the participants’ beliefs and practices.
Video use for observation
As Grossman and McDonald (2008) highlight, classroom observation is generally discussed from memory and notes made by one person with one perspective. This can limit the conversation, however employing video observations allows more complex aspects of practice to be analysed (Wang & Hartley, 2003). In research exploring the use of video observations, Rosaen et al. (2008) suggested that the aim of observations of practice was to support students to move beyond discussing experiences to learning from them. Their study found that the use of video encouraged the students to make more specific observations about their teaching than they would from memory. It also enabled them to move away from focussing on themselves and consider the wider perspectives including the classroom and children. Tripp and Rich (2012) also discussed the limitations of memory as their study found that analysis was enhanced when video was available. They highlight that there is little empirical study as to why using video has a greater impact but their findings revealed that the participants recalled images from the video as they were teaching and therefore helped them change their practice.
As more students access distance learning opportunities, there is a need for Higher Education (HE) institutions to embrace technology as a means of moving away from traditional approaches and providing innovative alternatives, which ensure that all students have equitable experiences and that barriers of distance do not endanger this (Heafner, 2012).
Assessment and feedback process
To give some context to the research and the research question, it is important to understand the process the students and associate tutors undertake using this method of assessment and feedback.
The students that this study refers to are all adult, distance learners on either a Post Graduate Certificate in Dyslexia or Dyscalculia where numerous observations of one-hour, one-to-one lessons are required as formative and summative assessment.
Prior to the implementation of using Panopto (2018) as a tool for video observation assessment, tutors were travelling across the country to scheduled lessons and assessment sessions between students and their learners, taking up valuable time and monetary resource. In recent years, this method advanced to utilising video for assessing the practice, which the students would post to their tutors on DVD disc or USB device. Due to various issues, including data protection as well as impact on teaching and learning and unnecessarily long feedback time, an innovation was needed to enhance the process further. Over the past two years we have used Panopto, primarily a lecture capture tool, within our Virtual Learning Environment (VLE) for students to be able to upload their recorded lessons to a secure, easily accessible area for tutors to assess and provide feedback.
The students can upload any video file they record to Panopto, which is stored within a secure assessment folder on the VLE, which can only be seen by themselves and tutors of the course. From any location with an internet connection, the tutor can then review the videos and provide feedback on these lesson observations directly through the VLE. Students receive their feedback in order to develop subsequent lesson observations which are again, submitted through Panopto on the VLE.
The research context
The aim of this study is to determine whether the use of video software (Panopto) to capture self-recorded lesson observations within a HE institute can support, or improve, the assessment and feedback process. It is hoped that this practice, if proven positive, could be replicated in other institutes and courses which have a requirement for observation of practice.
Ferrence (2000) argues that powerful justification for pedagogical research is that teachers become more effective when they are encouraged to assess their own work and then consider ways of working differently. It can be used to encourage change, encourage reflective practice and be a test-bed for new ideas and practices (Creswell, 2012).
Methodology
To investigate the aims of the research, qualitative thematic analysis has been used. Braun and Clarke (2006) provide current guidelines on thematic analysis and their six-step guide was followed in this study. Unlike other qualitative options, thematic analysis is not restricted to a specific theoretical view (Braun & Clarke, 2006; Clarke & Braun, 2013). This gives it advantages over other options, specifically for research within learning and teaching. According to Boyatzis (1998) it also allows for creating meaningful accounts from the analysis of qualitative data from multiple participants, making it a good choice for this study.
Participants
Pre-requisites were that participants needed to have undertaken at least one module on the Dyslexia or Dyscalculia post-graduate distance-learning programme at Edge Hill University, as these were the courses using the video software for lesson observation. The study looked to determine the successfulness of the process for both students and tutors, therefore associate tutors were interviewed for their perspectives too.
Three students and two associate tutors were deemed to be a sufficient sample due to the course cohort sizes and the timescale of the research project. (Barker, Pistrang, & Elliott, 2002). Associate tutors were selected rather than tutors due to them not being involved in the design of the course or the selection of educational technologies used; therefore, it was felt they would be more likely to be impartial with their responses.
Students were selected from across both cohorts of the course, meaning that some had used the software once and some twice. Various age ranges, genders and levels of confidence in terms of digital capabilities and lesson observation experience were selected.
Interviews
The study used a semi-structured interview technique as it offers flexibility for unanticipated ideas to emerge, whilst still keeping a form of structure to the interview (Robson, 2002). Appendices 5 and 6 show examples of the pre-prepared questions that the interviews were built around.
The interviews took place using Blackboard Collaborate, an online learning collaboration space. As the students are distance learners, this was the best option so that participants did not have to travel and potentially incur costs to undertake the research. The platform also allows for secure session audio recording, the recordings will abide by data protection policy and this is discussed further in the later ethical consideration section of the study. In order to capture the data for analysis the five interviews were then transcribed.
Analysis
The first step of analysis was to become familiar with the data. By undertaking the interviews ourselves, as well as listening to them back multiple times whilst transcribing, this enabled familiarisation (Isaac, 2015). Once everything had been transcribed, we further read through the interview transcripts and immediately started to notice a few initial patterns of what the students were discussing and also that the associate tutors had picked up on these elements too, such as anxieties around technology and observation as a whole.
The next step was to generate some initial codes and organise the data in a more meaningful way. Rather than to take an inductive approach (Thomas, 2006), a theoretical thematic analysis allowed for addressing the specific research question that was set out. (Boyatzis, 1998). Bree and Gallagher (2016) explain how Microsoft Excel can be utilised to code and help identify different themes. This process was used within this study due to the size of the data which didn’t justify using more expensive qualitative analysis software, such as Nvivo for example, which are effective with larger data sets (Spencer et al., 2003).
Step three was searching for the themes within the data. Initial analysis identified the following as opening themes: the technologies impact on students; receiving feedback electronically; learning from a distance; teaching and assessing from a distance; digital capabilities; technology anxiety; observation anxiety and using the technology as a critical reflection tool. Due to the small data set, it was noted that some of the initial themes identified overlapped slightly. For example, digital capabilities and technology anxiety were two identified themes that had slight overlaps. Further analysis in step four allowed for a review of the themes, overlapping themes were merged, further themes and sub themes were checked to ensure nothing had been overlooked and the themes were checked back against the original research question to solidify the themes appropriateness. For example, teaching and assessing from a distance data covered the same aspects as learning from a distance.
The final step in analysis was defining the themes. Once these were determined, comments from the data, were ordered in a way that would make them presentable when writing up the results.
Results
Five main themes were identified from the data, relating to the use of video software for assessment and feedback. These were “Learning and Teaching from a Distance, Observation Feedback Electronically, Technology Anxiety, Observation Anxiety and Critical Reflection”. All of these themes were frequent within the interview transcripts, across both students and associate tutor comments.
Learning and teaching from a distance
An overwhelming amount of the data within this theme was focussed around accessibility and student opportunity. One of the associate tutors voiced their opinion that no form of technology would ever live up to a face-to-face observation.
“There is nothing quite the same as been in the same environments and actually observing a session in the present… So I don’t think it would ever live up to that kind of observation.”
However, went on to say that students wouldn’t have constraints in tutor availability when scheduling appointments to observe their practice. Confirming what MacPherson and Nunes (2004), Salmon (2001) and Paechter et al. (2010) suggested theory of greater need for tutor flexibility.
“Having said that without using tools like that the students wouldn’t necessarily have those observations because of tutor availability.”
The same associate tutor went on to comment on the standard of tutor potentially being higher because of the technology allowing them to use practitioners currently working in the field of study.
“Well if [the course] was face-to-face observations, I wouldn’t be able to undertake work for Edge Hill… So for me it’s essential really… and it means the advantages for the University are that they can use associate tutors who are undertaking the role day in day out and actually working as specialist teachers.”
The other tutor interviewed commented on access and opportunity from a student’s perspective, stating it opens up opportunity to students living and studying further afield (Selwyn, 2010).
“It enables us to get to students throughout the whole country. I’ve even taught students in Australia, I know another lady was in Bahrain.”
As well as more opportunities for non-standard applicants (McGinnes, 2011; Ramos, 2011; Burgess, 2008; MacDonald, 2008) who currently cannot access a classroom environment.
“It just opens up opportunities for getting students in who perhaps couldn’t get to a classroom because of their time commitments or because they live in a different country.”
An example of this was the following student, who wouldn’t have had the opportunity to study on the course if it was a requirement for face-to-face observation.
“I wouldn’t be able to do the course without it… as the people I work with are vulnerable adults, so the observer may come to harm, so they won’t let another person in the room… I wouldn’t have been able to do the course had they been adamant on a face to face observation.”
As briefly mentioned in the introduction, a previous method for video observation was for students to post their work to tutors, also limiting accessibility to those who can afford additional costs to their course fees, this was also the case for students at other HE institutes.
“I’ve had some colleagues [on similar courses] that have had to post stuff to tutors, so they start incurring postage fees.”
Observation feedback electronically
The discussion within this theme touched on three sub-themes, which were feedback quality, turnaround time and ease of process.
With regards to the quality of feedback, none of the students interviewed suggested that their feedback was any different being delivered electronically through the VLE than if it was face-to-face.
“I don’t feel like my feedback has been any different because the person hasn’t been in the room if that make sense.”
The main negative aspect picked up in all of the interviews from a student’s perspective was the length of time between the observation and receiving feedback on it.
“If the person was here you could get the feedback instantly and you can take it on board straight away.”
“Waiting for the feedback wasn’t so terrific because I’m used to having feedback straight away, on face to face. So I can ask questions instantly, that was taken away.”
This element of feedback was also touched on by tutors.
“You can’t give them real-time feedback, but I think that is the only thing I can see that there is a difference in it.”
However, the same student also stated that overall; they prefer the electronic assessment and feedback process.
“I’d prefer to do it the way that I’m doing it… uploading stuff and getting feedback, I much prefer that.”
Perhaps had this been compared to any other feedback they receive on the course, which is typically four weeks turnaround, the students may have looked differently upon this. As it was, the students were making direct comparisons to the face-to-face observations. In addition, these students have not had the experience of previous year’s cohorts where the video observations were posted and took almost three times longer to provide the feedback.
When discussing the ease of the process, from submitting through to feedback, both student and associate tutor commented positively.
“It’s been such a positive experience for me and seeing how easy it is to upload something and to get the feedback promptly.”
“It’s moving forward and as a progressive University we have to move forward with technology.”
Technology Anxiety
By far the most prominent theme within the data from both tutor and student, was technology anxiety. Three sub-themes emerged within, which were digital capabilities, the argument of technology versus face-to-face and finally, pressure caused by ‘bad’ technology.
It was noticed immediately within the first sub-theme that there was a divide between the two ends of the spectrum of digital capabilities (Prensky, 2001).
“It’s like they [students] have a fear of pressing a button and they don’t try it because they’re fearful of doing something wrong.”
However, deeper reflection on the data showed that those who were self-confessed as less confident in their technological abilities were actually willing to give these new technologies a try, described in Wolsey and Grisham’s (2011) study as ‘digital tourists’.
“I feel we’re in a culture where technology is much more prevalent than it was before and I feel like we have to have a go or we’ll fall behind.”
“I’d say its quick, efficient, easy to use and the more that we’re becoming used to in a workplace of using electronic apps, communication, I think it’s almost becoming second nature.”
They also suggested it could improve learners’ digital capabilities.
“I feel like I’ve learned more about technology than I probably knew 12 months ago so that’s definitely helped me.”
Interestingly the comments that were highlighting a potential divide between the digital natives and immigrants (Prensky, 2001) were those who were said to be technologically savvy when discussing others with less experience.
“Just that little snippet of reassurance to put a little bit more confidence in the electronic world, I think. Certainly for those that aren’t that competent or trustworthy with it.”
This included tutors.
“The students who are technically advanced; they just embrace it and get on with it. The students who have a fear of tech, they’re the ones who it really has an impact on, it gives them an added stress to their learning.”
Other tutor comments separating the digital natives and the digital immigrants (Prensky, 2001) by age. A study by Czaja et al. (2006) indicated that age could have an impact on digital capabilities, but other factors such as age of adoption of technology and perceived need for technology should be taken into account.
“Nowadays I think the younger teachers I know don’t have any problems with it.”
Technology versus face-to-face in relation to practice observations was the second sub-theme highlighted. Intriguingly again, there was a divide here between the comments received from the students and the tutors.
Tutors valued face-to-face observations more than video, suggesting there is a difference in how both teacher and students perform in front of a camera.
“Sometimes the children or the adults don’t perform the same on a video… because there’s a camera there.”
When asked if the same would be true of having an observer in the room, technology anxiety was added to the debate.
“I do because it is the added pressure of making sure your tech is working as well… That anxiety associated with it was pretty strong for me… The anxiety comes down to whether it’s recorded properly or whether the hardware’s going to work.”
Could this be because of the tutors’ experience of study? When asked, neither participating tutor had undertaken video observation from a student perspective themselves. Cox (2014) argues that instructors prefer more traditional methods of delivery, because of the wait they were taught themselves.
“Sometimes I think the anxiety to be recorded will be a problem for the students and for their learners as well. I certainly know when I did my videos for my course, it was unnecessary anxiety to be recorded as well.”
Students on the other hand, much preferred the observation to be done through video technology as they thought the camera was less intrusive and more of a natural environment than having an observer in the room with them.
“You just put the camera on the side and get on with it… I think if there was someone else in the room at the time it would make them feel different as well.”
“It’s kind of in your own environment, so you feel more comfortable.”
The third sub-theme was additional pressure caused by what was described as bad technology. Tutor interviews raised this in the first instance but did highlight that it was usually their own technology or connection speeds that caused issues.
“I do know from the students’ point of view from the technology side of it, it does let them down. Their own technology more than the Panopto technology… and the internet speed, really of uploading.”
This was confirmed by one student in particular who works from a remote location.
“Our internet isn’t brilliant, it drops out, we don’t have the speed, we live on a farm in the middle of nowhere. If you’re trying to upload anything of any volume if you like, it struggles, that might be the issue.”
The need for students to succeed and produce quality work was meaning some were putting more pressure on themselves to produce a quality video, even though they were being assessed on the content of their lesson rather than quality of the product. Ramos (2011) and Burgess (2008) suggest part-time students have a need to succeed attitude, which creates more anxiety regarding their expectations.
“I religiously checked all of the equipment, so many times, I spent hours thinking “this is ridiculous”, only because of the pressure I was putting on myself… I wanted it to be right, and in one go.”
However, this acknowledged digital native (Prensky, 2001), felt that repeated process helped to build her confidence and skills.
“Once I’ve got the right equipment its fine and once I’ve got the hang of it. The last one that I did just worked like a charm and was no bother at all.”
Leading to a consideration that an initial assessment much earlier in the module may be a useful way for those not familiar with the tools, to get a feel for the technology and build their confidence before their formative and summative observations.
Observation Anxiety
As well as the anxiety of using technology, anxiety of observations in any form was a theme that occurred frequently throughout the data.
Two of the three students involved in the study declared that they did not like to be observed regardless of method.
“I don’t like being observed and I don’t like anything about them. I don’t like hearing what people have got to say… I’m not terribly good about hearing not good news.”
“I probably don’t like been observed in general I don’t think, it makes me very nervous.”
When digging a little deeper in to this past experience and the feedback that came with this was a reason for the anxious feeling around practice observation. The supportive nature that Olin (2009), Cockburn (2005) and O’Leary (2014) suggest to develop effective pedagogical practices, have not been supported in previous observation situations.
“The experience was not a good one so that adds to the anxiety you feel. When I got the 2nd set of feedback, that was also positive as well… I started to relax a bit.”
The anxiousness seemed to come from not knowing what the feedback was going to be, or an expectation based on previous experience that the feedback was going to be negative. Positive feedback or delivering feedback in a different way could alleviate some of the pressure students are putting on themselves (O’Leary, 2013; Cockburn, 2005).
When then asked if the technology makes a difference, all of the comments stated that the technology either makes a positive impact, or no difference at all on their observation anxiety.
“I think there’s always going to be a level of anxiety and nervousness because you know that there’s a set of eyes on you whether it’s human eyes or an electronic device recording you.”
“I’d prefer this way [using the technology] but I think that’s because even though you’re recording yourself you forget the camera is there, but if the tutor was in the class with me I’d be much more nervous therefore id make much more mistakes without realising it.”
“I would say it takes away from the pressure of somebody watching you. Somebody physically watching you is the biggest thing for me.”
One commented on the technology allowing them to teach or assess their students in the environment they would be used to, making them more relaxed.
“This way, you’re in the environment you’re going to be doing those session in with the individual and forget about the camera, so you just naturally be who you are without being on edge thinking someone else is watching.”
Critical Reflection
Within this theme, it was expected only one topic would become clear, which was students using the tool to replay their practice and critically reflect on this. However, through analysis a second sub-theme arose, which was that tutors also use the ability to re-play the videos as a chance to reflect on their feedback and give students deeper, more impactful comments.
“I watch it and re-watch parts again to really see the areas they are missing… I can watch it three times and really pin point exactly where they’re going wrong… They’re getting deeper feedback because I can play it back.”
“I like particularly you can slow the video down and speed the video up as well.”
Encouraging teachers to embrace a reflective approach enables them to investigate various dimensions of teaching (Belvis et al., 2012). Also, encouraging conversations allow more complex aspects of the practice to be analysed and reflected upon (Wang & Hartley, 2003).
“This day and age with faster connections, whether it’s at work or home, the amount of electronic devices that people have, once you’ve videoed it you can review it yourself.”
“It gives you a chance to be able to see yourself in terms of what you are doing and how you’re doing that. It gives you a better understanding on how you can develop yourself.”
“It gives me something to look back on when I’m looking to do the next sessions. I can understand myself better by having those videos available to me.”
“I don’t just record the one session I need to upload, I’ve ended up recording more than one… I wouldn’t have used that method if it was face to face.”
Discussion
The study aimed to determine whether the use of video software to capture self-recorded lesson observations within a HE institute can support or improve the assessment and feedback process. The views expressed are a concise sample and were taken in the early adoption stages of the software within this HE institute, however the views expressed give valuable insight to some of the early thoughts, as well as students’ and tutors’ experiences, with adapting to using this technology.
Students discussed the process as “simple and quick” unless there was difficulty with digital competency. Interestingly, those that self-diagnosed themselves as inexperienced using technology for education didn’t negatively describe the process themselves. Comments regarding digital inability or a fear of technology came from those who were describing themselves as digitally aware and capable, when they were describing others on the course. Furthermore, this language was also present when the tutors were describing a section of their students too. Those that were less able, digitally, suggested that repeating the process helps build confidence and in turn increases their digital capabilities, which has a knock-on effect on things such as employability (Digibyte, 2017; ECORYS UK, 2016; Houlihan, 2018).
Further reassurance in the technology is required by some and better internet connections to decrease the upload time during submission is needed for all students to be sold on this process. However, deeper feedback is developed using this process, but is not as timely as face-to-face.
One of the most impactful sub-themes taken from the results was seeing the opportunity and increased accessibility the technology within this course is creating. Secondary to this, students making use of the technology available to them for critical reflection and self-development is an unintentional but positive benefit.
Accessibility is also increased using technology for tutors. However, the face-to-face observations were still thought to be more impactful than technology alternatives in their opinion. This possibly, could be a sign of the studying these tutors did previously as the students’ data show that they prefer the technology option over traditional face-to-face. (Cox, 2014).
Summary
The three methods of assessment and feedback have evolved over recent years, from face-to-face observation with no technology input, through to the current method which is entirely reliant on technology, in the form of Panopto, video software.
Face-to-face observation offered instant feedback to students, but with no record of the lesson and no chance for critical reflection could take place. Video observations that were physically posted was a big progression on the course, had less onus on tutors and opened accessibility and opportunities to students by not requiring them to have tutors in a classroom observing them. However, feedback time was drastically increased and students also incurred costs due to postage. The current method of using Panopto through the VLE offers all of the benefits from the previous solution but with the option for students to critically reflect on their lessons and develop their practice. And although feedback turnaround time has increased since face-to-face observations, it is drastically less than it was with the postal option.
The research study has not only helped to confirm that the technology solution we are providing to students is having an impact on their experience of assessment and feedback, it has opened up conversations with students and tutors as to how the process can be further improved. It is a positive outcome that the technology being embedded is opening opportunities for students that would not have had a chance if face-to-face observation was a requirement to study in this field. It was also encouraging to witness the tool being utilised for more than initially predicted through self-reflection.
Hopefully the process used to observe post-graduate, adult learners in their workplace can be shared and provides useful insight to other HE institutes looking for similar solutions in fields where observation of practice is required.
Reference
- Baecher, L., & Kung, S. (2014). Collaborative video inquiry as teacher educator professional development. Issues in Teacher Education, 22(2), 93–115.
- Barker, C., Pistrang, N., & Elliott, R. (2002). Research methods in clinical psychology (2nd ed.). Chicester: Wiley.
- Belvis, E., Pineda, P., Armengol, C., & Moreno, V. (2013). Evaluation of reflective practice in teacher education. European Journal of Teacher Education, 36(3), 279-292.
- Boyatzis, R. E. (1998). Transforming qualitative information: thematic analysis and code development. Sage.
- Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77-101. Retrieved from http://eprints.uwe.ac.uk/11735
- Bree, R. T., & Gallagher, G. (2006). Using Microsoft Excel to code and thematically analyse qualitative data: a simple, cost-effective approach. The All Ireland Journal of Teaching and Learning in Higher Education, 8(2), (pp. 281-195). Retrieved from http://ojs.aishe.org/index.php/aishe-j/article/view/281
- British Educational Research Association (2011). Ethical guidelines for educational research. London: BERA.
- Burgess, J. (2008). Is a Blended Learning Approach Suitable for Mature, Part-Time Finance Students? Electronic Journal of e-Learning, 6(2), 131-138. Retrieved from http://files.eric.ed.gov/fulltext/EJ1098715.pdf
- Czaja, S. J., Charness, N., Fisk, A. D., Hertzog, C., Nair, S. N., Rogers, W. A., & Sharit, J. (2006). Factors Predicting the Use of Technology: Findings from the Center for Research and Education on Aging and Technology Enhancement (CREATE). Psychol Aging, 21(2), 333-352.
- Clarke, V., & Braun, V. (2013). Teaching thematic analysis: Overcoming challenges and developing strategies for effective learning. The Psychologist, 26(2), 120-123.
- Cockburn, J. (2005). Perspectives and politics of classroom observation. Research in Post-Compulsory Education, 10(3), 373-388.
- Cox, S. E. (2014). Perceptions and Influences Behind Teaching Practices: Do Teachers Teach as They Were Taught? Bingham Young University. Retrieved from https://scholarsarchive.byu.edu/etd/5301/
- Creswell, J. W. (2012). Qualitative inquiry & research design: Choosing among five approaches (4th ed.). Thousand Oaks, CA: Sage.
- Digibyte (2017). New report shows digital skills are required in all types of jobs. European Commission. Available online at https://ec.europa.eu/digital-single-market/en/news/new-report-shows-digital-skills-are-required-all-types-jobs
- Dixson, M. (2012). Creating effective student engagement in online courses: What do students find engaging? Journal of the Scholarship of Teaching and Learning, 10(2), 1–13. Retrieved from https://scholarworks.iu.edu/journals/index.php/josotl/article/view/1744
- ECORYS UK. (2016). Digital Skills for the UK Economy.
Department for Business Innovation & Skills. Retrieved from https://assets.publishing.service.gov.uk/government/uploads/system/uploads/
attachment_data/file/492889/DCMSDigitalSkillsReportJan2016.pdf - Edge Hill University (2014a). Information Strategy. Retrieved from https://www.edgehill.ac.uk/documents/files/information-strategy.pdf
- Edge Hill University (2014b). Code of Practice for the Conduct of Research. Retrieved from https://www.edgehill.ac.uk/documents/files/code-of-practice-for-research.pdf
- Edge Hill University (2016). Research Strategy. Retrieved from https://www.edgehill.ac.uk/documents/files/research-strategy.pdf
- Edge Hill University (2017). Research & Ethics Policy. Retrieved from https://www.edgehill.ac.uk/documents/files/ethics-research-ethics-policy.pdf
- van Es, E. A., & Sherin, M. G. (2010). The influence of video clubs on teachers’ thinking and practice. Journal of Mathematics Teacher Education, 13(2), 155-176.
- EUGDPR (2018). The EU General Data Protection Regulation. Retrieved from https://www.eugdpr.org
- Ferrence, E. (2000). Action Research. Themes in Education.
- Grossman, P.; McDonald, M. (2008). Back to the future: Directions for research in teaching and teacher education. American Educational Research Journal, 45(1), 184-205.
- Guo, R. X., Dobson, T., & Petrina, S. (2008). Digital Natives, Digital Immigrants: An Analysis of Age and ICT Competency in Teacher Education. Journal of Educational Computing Research, 38(3), 235-254.
- Heafner, G. L. (2012). Windows into Teaching and Learning [WiTL]: Exploring Online Clinicals for a Distance Education Social Studies Methods Course. Proceedings of SITE 2012 – Society for Information Technology & Teacher Education International Conference, 4594-4615. Austin, Texas, USA: Association for the Advancement of Computing in Education (AACE).
- Houlihan, M. (2018, February 6). Mind the gap – improving the UK’s digital skills in an evolving job market. Cisco UK & Ireland Blog [Blog post]. Retrieved from https://gblogs.cisco.com/uki/mind-the-gap-improving-the-uks-digital-skills-in-an-evolving-job-market/
- Isaac (2015, May 21). How to transcribe an interview for dissertation – Part 1. Weloty Academic Transcription Services [Blog post]. Retrieved from https://weloty.com/how-to-transcribe-an-interview-for-dissertation-part-1/
- Lahiff, A. (2015). Maximizing vocational teachers' learning: The feedback discussion in the observation of teaching for initial teacher training in further education. London Review of Education, 13(1), 3.
- MacDonald, J. (2008). Blended Learning and Online Tutoring (2nd ed.). Hampshire: Gower.
- MacPherson, M., & Nunes, M. B. (2004). The role of tutors as an integral part of online learning support. European Journal of Open, Distance and E-Learning, 2004(I).
- Margaryan, A., Littlejohn, A., & Vojt, G. (2011). Are digital natives a myth or reality? University students’ use of digital technologies. Computers & Education, 56(2), 429-440.
- Maxwell, B. (2010). In-service Initial Teacher Education in the Learning and Skills Sector in England: Integrating Course and Workplace Learning. Vocations and Learning, 3(3), 185-202.
- McGinnes, S. (2011). Overcoming time, place and pace: using technology-enhanced teaching to support mature evening degree students. Paper presented at the NAIRTL 4th Annual Conference.
- O’Leary, M. (2012). Exploring the role of lesson observation in the English education system: a review of methods, models and meanings. Professional Development in Education, 38(5), 791-810.
- O’Leary, M. (2014). Classroom observation: a guide to the effective observation of teaching and learning. Routledge, London.
- Ollin, R. (2009). The grading of teaching observations: implications for teacher educators in Higher Education partnerships. University of Huddersfield Repository. Retrieved from http://eprints.hud.ac.uk/id/eprint/8470/1/OllinGrading.pdf
- Orr, K., & Simmons, R. (2011). Restrictive practice: The work-based learning experience of trainee teachers in English further education colleges. Journal of Workplace Learning, 23(4), 243-257.
- Paechter, M., Maier, B., & Macher, D. (2010). Students’ expectations of, and experiences in e-learning: Their relation to learning achievements and course satisfaction. Computers & Education, 54(1), 222-229.
- Panopto (2018). The Video Platform for Universities and Businesses. Retrieved from https://www.panopto.com
- Prensky, M. (2001). Digital Natives, Digital Immigrants. On the Horizon, 9(5). MCB University Press. Retrieved from https://www.marcprensky.com/writing/Prensky%20-%20Digital%20Natives,%20Digital%20Immigrants%20-%20Part1.pdf
- Ramos, J. A. (2011). A Comparison of Perceived Stress Levels and Coping Styles of Non-traditional Graduate Students in Distance Learning versus On-campus Programs. Contemporary Educational Technology, 2(4), 282-293.
- Robson C. (2002). Real World Research (2nd ed). Blackwell.
- Rosaen, C. L., Lundeberg, M., Cooper, M., Fritzen, A., & Terpstra, M. (2008). How does investigation of video records change how teachers reflect on their experiences? Journal of Teacher Education, 59(4), 347-360.
- Salajan. F. D., Schonwetter. D. J., & Cleghorn, B. M. (2010). Student and faculty inter-generational digital divide: Fact or fiction? Computers & Education, 9(1), 11-22.
- Salmon, G. (2011). E-Moderating: The Key to Online Teaching and Learning. Tech Trends, 55(2).
- Selwyn, N. (2010). Schools and Schooling in the Digital Age: A Critical Analysis. Oxon: Routledge.
- Spencer, L., Ritchie, J., & O’Connor, W. (2003). Analysis: practices, principles and processes. In J. Ritchie & J. Lewis (Eds.), Qualitative research practice: A guide for social science students and researchers (pp. 199–218). London: Sage.
- Thomas, D. R. (2006). A General Inductive Approach for Analyzing Qualitative Evaluation Data. American Journal of Evaluation, 27(2), 237-246.
- Tripp, T. R., & Rich, P. J. (2012). The influence of video analysis on the process of teacher change. Teaching and Teacher Education, 28(5), 728-739.
- Vryonides, M., & Vitsilakis, C. (2008). Widening participation in postgraduate studies in Greece: mature working women attending an e-learning programme. Journal of Education Policy, 23(3), 199-208.
- Wang, J., & Hartley, K. (2003). Video Technology as Support for Teacher Education Reform. Journal of Technology and Teacher Education. Society for Information Technology & Teacher Education, 11(1). Waynesville, NC USA.
- Wolsey, T. D., & Grisham, D. L. (2011). A Nation of Digital
Immigrants: Four Principles. California Reader, 44(2), 1-9.