NEW

Best of EDEN 2018 Special issue
 

There are not any recent contributions.

There are not any recent contributions.

Archives

EURODL Mailinglist

2851 subcribers
 

EURODL Visitors

back

Effects of Multimedia Feedback on Pre-Service Teachers’ Perceptions, Self-Assessment, and Academic Achievement

Gökçen Aydin, Mithat Çiçek, Mustafa Güleç, Middle East Technical University, Turkey

Introduction and Review of Literature

The importance of feedback for learning is recognized by a number of theoretical perspectives rooted in behaviourism and pursued by many others such as goal setting theory (Locke & Latham, 1990), social cognition theory (Bandura, 1991) and conditions of learning (Gagne, Briggs, & Wager, 1992). Feedback serves as an essential component of teaching and learning process in these theories because it provides important information to learners about their performance on a specific task or goal (Hollenbeck, Karam, DeRue, & Lam, 2011). 

This crucial component of the learning is mostly classified in terms of its source (Brett & Atwater, 2001; Greller & Herold, 1975; Vancouver & Morrison, 1995, as cited in Lam et al., 2011), timing (Druskat & Wolff, 1999, as cited in Lam et al., 2011; Kulik & Kulik, 1988), type (Earley, Northcraft, Lee, & Lituchy,1990; Hammond, Summers, & Deane, 1973; Jacoby, Mazursky, Troutman, & Kuss, 1984, as cited in Lam et al., 2011) frequency (Anderson, Kulhavy, & Andre, 1971; Hundal, 1969, as cited in Lam et al., 2011) and its methods for response (Dopke, 2010). These methods are known as text-based (Tang, 2000; Quible, 1997 as cited in Dopke, 2010), audio (Sipple, 2007) and multimedia (Ice et al., 2007).

Text based feedback is the most common one for the majority of instructors because it seems the most pragmatic method for providing feedback (Dopke, 2010). Butler and Nisan (1986) reported that the group which received task related written comments showed significantly more interest on the task than the other group which received only the grades (as cited in Styrk, 2007). Kumar and Stracke (2007) stated that expressed opinions of instructor in written feedback were perceived very helpful for doctoral students to build their confidence (as cited in Can, 2009). Providing written feedback seems to be same with the feedback that is provided electronically. However, it offers lots of benefits. Gould (2012) argues that two most common benefits of electronic text based feedback are its legibility and efficiency. In a study, Denton (2001) found that emailing feedback to learners contributed to the improvements in students’ report writing skills over a 2-week period (as cited in Gould, 2012).

Audio feedback is mainly known as recorded comments that are prepared by the instructor on a task. This method has some advantages compared to the written-based. According to Dopke (2010), instructor can provide a great deal of information than can be provided in writing with the help of audio based feedback. Some researchers (Ice et al., 2010; Norcliffe & Middleton, 2007; Oomen-Early et al., 2008; Rotheram, 2008) reported favourable results on instructor workload and learner perceptions, performance, and satisfaction when providing assessment feedback in recorded audio format (as cited in Gould, 2012). Providing audio feedback has also proven to be more time efficient than text, under certain circumstances (Davies, 2010; Nortcliffe & Middleton, 2007; Rotheram, 2008, as cited in Gould, 2012).

Combining text, audio and video into one format can increase the effectiveness of the feedback, because lots of benefits of these methods can be combined in multimedia feedback experience. Several studies indicated that students preferred a combination of these methods in the process of receiving feedback (Ice et al., 2010; Oomen-Early et al., 2008; Simonsson et al., 2009, as cited in Gould, 2012). Tsutsui and Kato (2001) designed a multimedia feedback tool that was developed by the University of Washington’s Technical Japanese Program in their study. Results showed that this tool was approved as effective for oral skills training. Additionally, in Gould’s (2012) study on multimedia feedback, the findings revealed that students reported positive effects on their cognitive, affective, and psychomotor learning through the learning process along with the multimedia feedback. 

Majority of existing research about effectiveness of feedback in terms of response method is based on text-based feedback (Butler & Nisan, 1986, as cited in Styrk, 2007; Denton, 2001 as cited in Gould, 2012; Quible, 1997; Kumar & Stracke, 2007) and audio feedback (Ice et al., 2010; Oomen-Early et al., 2008; Davies, 2010; Nortcliffe & Middleton, 2007; Rotheram, 2008, as cited in Gould, 2012; Sipple, 2007). However, based on technological developments, using multimedia technologies in educational environment is increasingly becoming popular among researchers. Some of these researchers argue that students prefer multimedia method, a combination of audio and text-based methods, in the process of receiving feedback (Ice et al., 2010; Oomen-Early, Bold, Wiginton, Gallien, & Anderson, 2008; Simonsson, Kupezynski, Ice, & Pankale, 2009, as cited in Gould, 2012). In the light of those statements, the investigation of pre-service teachers’ experiences on this new response method of feedback might be beneficial in order to determine its effect in educational environments.

The purpose of this study is twofold: One of them is to investigate the effect of providing feedback in digital multimedia format on pre-service teachers’ perceptions toward multimedia feedback and self-assessment about their skills on a performance-based task. The second purpose is to investigate whether or not providing feedback in digital multimedia format affects pre-service teachers’ academic achievement on a performance-based task. 

In compliance with these purposes, the following research questions are addressed:

• Does providing feedback in digital multimedia format affect pre-service teachers’ perceptions toward multimedia feedback?
• Does providing feedback in digital multimedia format affect pre-service teachers’ self-assessment on their skills for a performance-based task?
• Does providing feedback in digital multimedia format increase pre-service teachers’ achievement on a performance-based task?

Methodology

Two different research designs were used in the light of research questions of current study.

Study 1

A matching only pre-test-post-test control group design among quasi-experimental designs was applied in this study. This type of design is more suitable when the random assignment for study groups is not applicable since some of the other variables may not be equal in a study (Frankel, Wallen & Hyun, 2012).

The sample of the study composed of 19 sophomore students from Computer Education and Instructional Technology department at Middle East Technical University and they voluntarily participated in this study. The data were collected from laboratory sessions of Design and Use of Instructional Material Course being taught in this field. The participants were assigned into two groups named as experimental and control groups. This selection process was administrated by applying to the pre-test scores, prior feedback experience, GPA and gender of the participants.

Gould’s (2012) pre- and post-course survey was used in this study to obtain the information of participants. This instrument was a modification of an instrument that was used and validated by Ice et al. (2007). It is composed of four main parts such as demographics, experience, course expectations, and multimedia feedback. The multimedia feedback section of the survey consists of nine questions aiming to obtain information about participants’ perceptions on multimedia feedback. The second instrument in this study is a self-assessment form about the competency levels on use of software program which is Microsoft Expression Web. The researchers developed the instrument and content validity was provided by an expert from the field. The data analysis of Study 1 was conducted by using SPSS 23 software. As a non-parametric test, Mann-Whitney U test was utilized along with descriptive statistics.

Study 2

One of the sequential types of mix methods design, explanatory sequential design was utilized in this study. The quantitative data collection and analysis process is followed by qualitative data collection and analysis phase in this design. Two types of data are analysed separately. The results of qualitative analysis are used to extend the results of quantitative analysis (Fraenkel, Wallen & Hyun, 2012). In this direction, the qualitative analysis results were used to support the quantitative analysis results.

Only the experimental group students (n = 9) from the previous study participated in this study. For the quantitative phase, convenient sampling method was used for the selection of the participants. A purposive sampling technique was applied for the qualitative phase of this study. Four participants from the experimental group were invited for interviews after taking the post-test.

In quantitative phase, achievement scores of the students were obtained via the rubric developed by the researchers to evaluate the web sites designed by students. This rubric consists of twenty criterions that have different percentages out of 100-point. For the validation of the instrument, expert view was taken into consideration. On the other side, semi-structured interviews were administrated with four students for qualitative phase of the study. The content validity of interview questions was provided by expert views. A schedule was also followed during the interviews. The quantitative data of Study 2 was analysed by applying Wilcoxon test in SPSS 23 software. For qualitative data, codes were defined and reported with related quotations.

Procedure

After pre-defined lab sessions of Design and Use of Instructional Material Course, there were assignments for students. Students uploaded the related file prepared via Microsoft Expression Web design editor to online course page. The researchers examined these assignments and feedback files which were created. Students could download their feedback file from the same course page. Feedback files were different for experimental group and control group. For the experimental group, this file was created with Adobe Captivate software that enabled to record onscreen activities easily. In these activities, text and audio based feedback was provided via videos as a part of multimedia feedback. On the other side, feedback files created with Microsoft Office Word software in text were delivered to students in control group.

Results

Descriptive Statistics and Past-Feedback Experience of Participants

Of 19 participants, the number of females was 12 and the rest (n = 7) were males. Their ages ranged from 20 to 23. The experimental group was formed by 10 students. While 5 of them reported that they had already taken feedback in text format in three or more courses, the number of students whom had taken such feedback in two courses was 2. The rest (n = 3) stated that they had taken this type of feedback for only one course. Besides, 6 students had never taken feedback in audio format, whereas 3 of them had taken this feedback type in one course and 1 student had already taken this type feedback in two courses. Finally, for multimedia feedback, when 7 students out of 10 had never taken this type of feedback, the remaining 3 students had taken such feedback in one course.

On the other side, 9 students were assigned to control group. While 3 of them had previously taken feedback in text format in 3 or more courses, 4 students had taken this type of feedback in two courses and 2 of them had taken in two courses. For audio type of feedback, 4 students reported that they had never received such feedback before. When 2 of them had taken this feedback type in one course, the rest 3 students had taken in three or more courses. Lastly, the majority of the control group reported that they had never taken this multimedia feedback before (n = 6). One of them had received in one course and the remaining (n = 2) had received multimedia feedback in two courses previously.

Results for Study 1

The aim of Study 1 was to investigate the difference between pre and post test results of experimental and control group in terms of skills and perception towards multimedia feedback. As the number of participants was below 30 (n < 30) (see Table 1), among non-parametric tests, Mann-Whitney Test was utilized. The results showed that there was no statistically significant difference between pre-and post-test results of experimental and control group except the perception post-test results as indicated in Table 2. The significant difference was found between post-test scores of experimental and control group and experimental group (M = 13.25) had larger mean score compared to control group (M = 6.39). That is, experimental group had an increase in their level of perception towards multimedia feedback, z = -2.71, p < .05.

Table 1:  Means of Pre-Post Test Results for Groups

Table 2:  Results of Mann-Whitney U Test

Results for Study 2

The aim of Study 2 was to find whether there was a significant difference between pre and post test result of experimental group in terms of academic achievement. According to descriptive statistics, the mean of scores before multimedia feedback was 71.50 and the mean scores after multimedia feedback was 90.30 (see Table 3 below). The results of non-parametric test indicated that there was a significant difference between pre and post test scores of students who participated to experimental group, z = -2.66, p < .05 as stated in Table 4. Providing multimedia feedback increased students’ academic achievement.

Table 3:  Descriptive Statistics for Multimedia Feedback Scores

Table 4:  Results of Wilcoxon Test

In order to support these findings, four students from experimental group were interviewed based on their academic achievement. The important aspects mentioned by students included understanding and correcting mistakes easily, remembering the concepts or steps to be followed, opening space for new learning, discovering even small mistakes, increasing step by step learning, giving the opportunity to work on mistakes and flexible learning as a result of getting multimedia feedback. As there was the opportunity to watch the feedback again, one participant resembled this type of feedback to face-to-face feedback. The students also underlined some motivational factors leading to higher academic achievement: The more interaction between instructor and student encouraged students to work on the necessary changes about feedback; reduction of procrastination; new ways of providing feedback was supportive for students in terms of motivation because they know how to find answers to their questions whenever they need. Interestingly, one participant stated an increase in her level of self-confidence as a result of much care from instructor. Getting multimedia feedback was not only useful for checking mistakes about the task, but also useful for willingness to discover mistakes. Some of the direct quotations were provided below:

“I looked at the feedback there, and did the task again according to feedback. I watched from the beginning by checking my mistakes and when I thought that it was completed, I uploaded it.”

“When I got written or oral feedback, I can forget if I do not take notes. When you provided multimedia feedback, I could look at it whenever I want.”

“This shows that the instructor spends time for student. In a way, instructor cares for student, and this supports studying much. Somehow, it gives student self-confidence.” 

“I did what you said and showed one by one there and so, my score increased.”

Conclusions and Discussion

The aim of this study was to examine the effect of providing feedback in digital multimedia format on pre-service teachers’ perceptions toward multimedia feedback, their self-assessment about their skills, and academic achievement on a performance-based subject. Pre, post-test experiments and semi-structured interviews were applied in order to response the research questions under two different research designs.

According to pre, post-test results for perception and skill scores in Study 1, significant difference was found only post-perception scores of study groups. Experimental group had larger mean scores than control group in terms of their perceptions toward multimedia feedback. One can interpret that students in experimental group perceived that taking multimedia feedback for a performance-based task is useful for their performance. They prefer to take multimedia feedback. In the light of this result, Tsutsui and Kato (2001) found multimedia feedback more efficient and rewarding for students while comparing to other feedback types. Furthermore, Ice at al. (2010) reported that students preferred taking a combination of different feedback types which refers to multimedia feedback. On the other side, no significant difference between study groups was found in terms of their self-assessment scores for their skills while performing during the study. That is, providing multimedia feedback did not significantly affect students’ self-assessment scores while using a web based material for a performance-based task.

Finally, academic achievement of experimental group was examined in Study 2. The pre, post-test results concluded that statistically significant difference was found. Students’ achievement scores during that performance based task significantly increased with the help of multimedia feedback they took from the instructor. The interview results also supported the experiments’ results. Four students stated that taking multimedia feedback helped them to increase their course scores, complete related task, have chance to follow task steps when needed, increase the interaction between instructor and student, and increase their self-confidence. They all had positive attitudes toward taking multimedia feedback during a performance-based task. The results of Gould’s (2012) study were also parallel to current study. He reported that providing multimedia feedback increased the satisfaction, motivation, interaction level and learning of students.

Since the number of participants in this study was not efficient enough for making reasonable generalizations, more research studies formed from larger samples are suggested to be conducted. Furthermore, different performance-based tasks can also be applied in order to explore the effect of multimedia feedback for various parameters in educational environments.

References

1. Anderson, R. C., Kulhavy, R. W., & Andre, T. (1971). Feedback procedures in programmed instruction. Journal of Educational Psychology, 62, 148-156.
2. Bandura, A. (1991). Social cognitive theory of self-regulation. Organizational Behavior and Human Decision Processes, 50, 248-287.
3. Brett, J.F., & Atwater, L. E. (2001). 360 degrees feedback: Accuracy, reactions, and perceptions of usefulness. Journal of Applied Psychology, 86, 930-942.
4. Can, G. (2009). A Model for Doctoral Students’ Perceptions and Attitudes toward Written Feedback for Academic Writing. Retrieved from http://digitalcommons.usu.edu/etd/227
5. Davies, S. (2010). Effective assessment in a digital age: A guide to technology-enhanced assessment and feedback. JISC.
6. Denton, P. (2001). Generating coursework feedback for large groups of students using MS Excel & MS Word. University Chemistry Education, 5, 1-8.
7. Dopke, L. (2010). An Examination of Effectiveness of Different Types of Feedback Across Controlled Written Assignment Scenarios. (Unpublished doctoral dissertation). Grand Valley State University.
8. Druskat, V. U., & Wolff, S. B. (1999). Effects and timing of developmental peer appraisals in self-managing work groups. Journal of Applied Psychology, 84, 58-74.
9. Earley, P. C., Northcraft, G. B., Lee, C., & Lituchy, T.R. (1990). Impact of process and outcome feedback on the relation of goal setting to task performance. Academy of Management Journal, 33, 87-105.
10. Fraenkel, J. R., Wallen, N. E., & Hyun, H. H. (2012). How to design and evaluate research in education (8^th ed.). New York: McGraw-Hill.
11. Gagne, R., Briggs, L., & Wager, W. (1992). Principles of Instructional Design (4^th ed.). Fort Worth, TX: HBJ College Publishers.
12. Gould, B. E. (2012). Using Multimedia Feedback to Enhance Cognitive, Affective, and Psychomotor Learning. (Doctoral dissertation). Royal Roads University. Canada.
13. Greller, M. M., & Herold, D. M. (1975). Sources of feedback: Preliminary investigation. Organizational Behavior and Human Performance, 13, 244-256.
14. Hammond, K. R., Summers, D. A., & Deane, D. H. (1973). Negative effects of outcome-feedback in multiple-cue probability learning. Organizational Behavior and Human Performance, 9, 30-34.
15. Hundal, P. S. (1969). Knowledge of performance as an incentive in repetitive industrial work. Journal of Applied Psychology, 53, 224-226.
16. Ice, P., Curtis, R., Phillips, P., & Wells, J. (2007). Using asynchronous audio feedback to enhance teaching presence and students’ sense of community. Journal of Asynchronous Learning Networks, 11(2), 3-25.
17. Jacoby, J., Mazursky, D., Troutman, T., & Kuss, A. (1984). When feedback is ignored: Disutility of outcome feedback. Journal of Applied Psychology, 69, 531-545.
18. Kulik, J. A., & Kulik, C-L. C. (1988). Timing of Feedback and Verbal Learning. Review of Educational Research, 58(1), 79-97.
19. Kumar, V., & Stracke, E. (2007). An analysis of written feedback on a PhD thesis. Teaching in Higher Education, 12(4), 461-470.
20. Lam, C. F., DeRue, D. S., Karam, E. P., & Hollenbeck, J. R. (2011). The impact of feedback frequency on learning and task performance: Challenging the ‘‘more is better’’ assumption. Organizational Behavior and Human Decision Processes, 116(2), 217–228.
21. Locke, E.A., & Latham, G.P. (1990). A theory of goal setting and task performance. Englewood Cliffs, NJ: Prentice-Hall.
22. Nortcliffe, A., & Middleton, A. (2007, September). Audio feedback for the ipod generation. Paper presented at the meeting of International Conference on Engineering Education, Coimbra, Portugal.
23. Oomen-Early, J., Bold, M., Wiginton, K. L., Gallien, T. L., & Anderson, N. (2008). Using asynchronous audio communication (acc) in the online classroom: A comparative study. MERLOT Journal of Online Learning and Teaching, 4(3), 267-276. Retrieved from http://jolt.merlot.org/vol4no3/oomen-early_0908.pdf
24. Quible, Z. K. (1997). The Efficacy of Several Writing Feedback Systems. Business Communication Quarterly, 60(2), 109-123.
25. Rotheram, B. (2008). Sounds good: Quicker, better assessment using audio feedback. Retrieved from https://sites.google.com/site/soundsgooduk/downloads/Sounds_GoodEvaluation_10.pdf?attredirects=0&d=1
26. Schunk, D. H. (2012). Learning theories an educational perspective (6^th ed.). Boston, MA: Pearson
27. Simonsson, M., Kupczynski, L., Ice, P., & Pankake, A. (2009, April). The impact of asynchronous audio feedback in the dissertation advising process. Paper presented at the American Educational Research Association Annual Meeting, San Diego, CA.
28. Sipple, S. (2007). Ideas in Practice: Developmental Writers’ Attitudes toward Audio and Written Feedback. Journal of Developmental Education, 30(3), 22-31.
29. Styrk, H. L. (2007). The Effects of Differing Types of Task Related Feedback on Students’ Artistic Skill. (Doctoral dissertation). Retrieved from ProQuest Information and Learning Company. 3271302.
30. Tsutsui, M., & Kato, M. (2001). Designing a Multimedia Feedback Tool for Developing Oral Skills. Elm Bank Publications, 81-88.
31. Vancouver, J. B., & Morrison, E. W. (1995). Feedback inquiry: The effect of source attributes and individual differences. Organizational Behavior and Human Decision Processes, 62, 276-285.