Use of MOOCs in Traditional Classroom: Blended Learning Approach
Antonia Bralić, Blaženka Divjak, Faculty of Organization and Informatics, University of Zagreb, Croatia
Best Research Paper Award Winner
Abstract
MOOCs were initially designed as stand-alone products but blended learning programs have been developed to utilize the benefits of MOOCs. This paper presents case study research of incorporating a MOOC into traditionally delivered course with specific goals of giving students opportunity to experience learning in virtual environment and helping part-time students in achieving particular learning outcomes. Benefits and obstacles of such an effort were researched in a case study where part of the traditionally delivered learning was replaced with a MOOC at a compulsory graduate level course. Students were asked to choose between completing a MOOC and taking project-based activity to work towards the final grade. While going through the MOOC, it was required from students to keep a learning diary to showcase the understanding of the content but also to provide insight for qualitative analysis based on a set of questions that the students were required to answer. The case study research was based on qualitative analysis of the learning diaries and on quantitative review of achieved results. Qualitative analysis of the answers to open ended questions showed that students’ overall feedback has been positive. Students were generally satisfied with this form of learning, despite some difficulties, such as language barrier or sufficient prior knowledge or required workload. Most appreciated characteristics of MOOCs were self-paced learning and the option to assess knowledge on regular basis, especially among part-time students.
Introduction
Growing popularity of Massive Open Online Courses (MOOCs) is indisputable. Morris (2014) for example states that “MOOCs are available to students to supplement their learning and personalized learning environments, and use of learning analytics are set to transform education”. Until today, different options of integrating MOOCs in formal education have been explored, implemented, and evaluated. Two main types of embedding MOOCs in traditional taught classes to achieve learning outcomes have been noted: (a) recognizing results achieved in MOOCs before enrolling in a learning program (prior learning) and (b) supplementing or replacing segments of academic courses with content from MOOCs. Variety and availability of MOOCs have been a strong encouragement for both cases. This paper focuses on the latter. Goal is to enhance learning processes and ultimately, increase retention of acquired knowledge and engage learners. This method of teaching has advantages and challenges, as discovered in previous research. A right blend should take best of offline and online learning to create a system that enable students to achieve learning goals in the best possible way.
Many blended learning programs in academic setting today are created around traditional courses that are now enriched with online content and capabilities. Blended learning program can be built with any online content, not necessarily MOOC content. There are multiple benefits of incorporating MOOCs in traditionally taught courses. Griffiths, Mulhern, Spies, and Chingos (2015) study generated six of them: “replaying lectures, augmenting or replacing secondary materials, filling gaps in expertise, exposing students to other styles of teaching and class discussion, reinforcing key skills, and teaching students how to teach online”. Series of authors claim that blended learning approach can enhance teaching and learning, for example (Morris, 2014; Gilbert & Flores-Zambada, 2011). Several attributes characteristic for blended learning agenda at some institutions were detected by (Sharpe, Benfield, Roberts, & Francis, 2006): “flexibility of provision, supporting diversity, enhancing the campus experience, operating in a global context and efficiency”, which can also be translated to benefits of programs like this. Often mentioned downside of MOOCs is low completion rate. Koller, Ng, Do, and Chen (2013) coin the term retention funnel and state that high dropout rates in MOOCs can be alarming for traditional educators. Embedding MOOCs in traditional face to face (f2f) classes by assigning credit to its partial or full completion could partly solve drop out problem. One student that participated in the covered case study in this paper stated that she “used MOOCs earlier to supplement the classroom learning but due to university related commitments she was not able to complete MOOCs. That is why incorporating a MOOC in classroom taught courses so its completion counts towards the final grade is a great motivator”. Computer literacy and technology acceptance are general challenges of online learning that need to be kept in mind. Israel (2015) emphasizes that integrating a course that is not designed to be a part of a blended learning program, holds its challenges, such as ensuring student engagement. Having benefits and challenges of MOOCs in mind, it has become clear that a proper blend of online and classroom activities has to be created to achieve success among a targeted student audience.
Series of studies and experiments have been conducted to trial and evaluate the use of MOOCs in traditional taught courses. Griffiths et al. (2015) have conducted a study to “examine the use of MOOCs in fourteen campus-based courses”. No statistical difference in pass rate or final score was registered in those studies, but feedback covering rating, interest, amount learned, and difficulty was better for traditional taught classes. Further on, blended learning with MOOC content was piloted at San José State University (SJSU), leveraging an edX course in class. Flipped classroom model including projects and quizzes was implemented. This program achieved “a high success rate with 90% of the students passing the final exam, as compared with 55% in the traditional class of the previous year” (Ghadiri, Qayoumi, Junn, Hsu, & Sujitparapitaya, 2013) in (Yousef, Chatti, Schroeder, & Wosnitza, 2015). Israel (2015) reviewed models of blending MOOCs in traditional classroom teaching; major findings were that “there is modest positive impact on learning outcomes, no significant evidence of negative effects for any subgroups of students, and lower levels of student satisfaction are recognized in blended MOOCs in classrooms. A blended learning program has also been experimented at Vanderbilt university as well where Coursera Machine learning MOOC was incorporated in a graduate level course on machine learning. MOOC chapters were built in the course, accompanied with additional learning and tasks that were important for the students. Students’ experience was evaluated through a focus group and qualitative analysis. The student feedback was overall positive; students appreciated flexibility, possibility to learn at their own pace, and bite sized videos, but also realized that it takes motivation and self-discipline to stay on track. Students did not participate in forums and discussions but described forums as useful to realize the issues other students might have. Students appreciated the flipped classroom model and the possibility to apply what has been learned outside of class. Students were asked to give a rating of the course and the blended approach in 2012 had higher satisfaction score than the traditional taught course in 2006, on these small samples, which needs to be taken into consideration (4.17 and 3.83 respectively) (Bruff, Fisher, McEwen, & Smith, 2013). This model is similar to the one applied at University of Zagreb, Faculty of Organization and Informatics in Croatia (FOI) that is presented in this paper.
Case study and research questions
The course Discrete Mathematics with Graph Theory (DMGT) is taught in the first year of master level of study programs Information Systems and Software Engineering at FOI. It is taught as a blended learning course and both full-time and part-time students are enrolled in the course. The syllabus consists of two parts: in the first part different topics in discrete mathematics are covered and the second half is dedicated to the graph theory and its applications. The topics have sound foundations in mathematical theory but offer multitude of applications of the covered theory in computer science and business, e.g. problem solving exercises that are performed individually or in teams (Divjak, 2015). Incorporation of a MOOC in the course is learning outcomes-based. The constructive alignment for the two learning outcomes (out of 7) for the course DMGT is presented in Table 1. Term “constructive alignment” is coined by John Biggs (2003). In order to guarantee achievement of intended learning outcomes they must be aligned with teaching and learning method, assessment and student workload.
Table 1: Constructive alignment of two of the learning outcomes on the course DMGT
MOOC has been offered as an alternative activity to project work that was credited towards the final grade in the course. Both activities are aligned to the same learning outcomes as it is shown in Table 1. There is a two-folded goal for the introduction of MOOC in the course. Firstly, to give students more online learning experience and secondly to help part-time students, that are working and are not able to fully participate in campus teaching, in meeting the course’s learning outcomes. We researched the case study in order to answer the following research question:
- Can use of MOOCs help in giving students positive learning experience in virtual environment and help part-time students in achieving particular learning outcomes?
- How to align course learning outcomes and student workload with use of MOOCs in a specific course?
- What are the main challenges for students in using MOOCs?
MOOC learning program is covered and evaluated in this paper for academic years 2014/2015 and 2015/2016. In the first academic year, students were asked to choose a course-related MOOC on Coursera platform and a teacher needed to approve a choice. In the second academic year students were supposed to choose a course among those that were preselected and offered in LMS (Moodle), to provide a more focused approach and increase quality of the program.
Case study analysis: quantitative and qualitative
The sample in this research was made out of graduate students enrolled in DMGT course. In the academic year 2014/2015, there were 107 students enrolled in DMGT class, 28 female and 79 male students, out of which 9 chose to complete a MOOC. In year 2015/2016, 88 students were enrolled, 75 male and 13 female, out of which 22 chose a MOOC. In both years, part-time and full-time students were enrolled in the course but part-time students were particularly encouraged to take a MOOC. Some trends are shown in Table 2.
Table 2: Metrics, 2014/2015 and 2015/2016
Number of users that chose MOOC over project work grew significantly in 2015/2016. This could be related to previous experiences shared by students that attended DMGT course. In year 2014/2015 better average final grade was achieved by students who completed project work but the results changed in 2015/2016 when a better final grade was achieved by those students who chose MOOC over project work. This change can be explained in different ways but since there was too small proportion of all students taking MOOCs to make any reliable conclusion based on quantitative data. Data taken from the second year is more reliable because every fourth student took a MOOC. Based on that, it seems that the achievement of learning outcomes is on very similar level for both groups. Finally, MOOC related tasks have been graded better in 2014/2015. This grade consists of two parts: 90% are based on MOOC’s final grade received by students and 10% on their diary quality and short presentation of MOOC to other students. Further research needs to be done in the upcoming years of teaching this course with the same approach, to be able to detect trends and more reliable results because the main limitation of the quantitative part of this research is a small number of students in MOOCs.
Students were obliged to keep a learning diary when going through the MOOC. Diary was roughly determined by required open-ended questions. Journal feature in Moodle was used to keep learning diaries by students and to deliver feedback by teachers. Questions that were set in front of the students to describe their experience with MOOC were essentially the same in both academic years as it can be seen in table 3. In 2015/2016 the questions were fine tuned to gather more detailed information and the volume of required handed review was increased from 400-800 words to 800-1000 words. Open ended questions offer guidance and ensure loose uniformity in results. Still, this way of feedback gathering allows enough flexibility to express personal experience and opinion. The answers relevant to the research questions are analyzed in the following paragraphs.
Table 3: Questions asked in MOOC learning diary
Connection between MOOC and DMTG course (questions 1-5)
As required, all students have uploaded a screenshot of their certificates of completion and shared the results they achieved. Similarly, a description of weekly activities was done, in more or less detail, by all students. Most variety was encountered in questions covering connections between MOOCs and DMGT as well as courses taken throughout formal education. Students that took different MOOCs were able to connect them with particular learning outcomes of DMGT not necessarily the ones from Table 1: (a) “I would link the Coursera course to a specific learning outcome of DMGT: to use mathematical literature from multiple sources, at least one tool for processing mathematical language, and an e-learning system, having specific characteristic of mathematics in mind.”; (b) “I would connect MOOC learning outcomes with three DMGT learning outcomes”. Some students have found loose connection between MOOC content and classroom taught content. Several have emphasized that MOOC in fact covers practical implications of what is taught in the classroom taught course and how well they supplement: “DMGT and MOOC are complementing each other very well. In DMGT I received theoretical grounds and MOOC helped be to understand the theory following practical examples”. One of the most mentioned elements of this type of learning was the language of MOOCs. Learning in English was rather new for most students and the feedback was various: (a) “I was sceptical because of the language barrier (…) and taking a MOOC was quite a challenge because it required combining English language and important content”; (b) “I liked this way of learning because, in addition to learning itself, I had a chance to practice my English skills and to think about this topic in English”. Not all language related feedback was positive: (a) “As the MOOC was progressing, it took more time and effort to complete everything in the curriculum. Completion was additionally slowed down by English language”; (b) “I spent most of the time translating tasks to Croatian to understand what needs to be done”.
Experience with using MOOCs (question 6)
Generally speaking, the experience of taking a MOOC has been positive, with majority of students reporting that they would encourage continuing this way of teaching DMGT and that they will continue to use MOOCs to supplement their own learning: (a) “As I was going through the course I selected, I have also browsed through the platform and detected several other courses I plan to take at a certain point”; (b) “The entire experience (…) is very positive. This is the first time I have studied something this way, but it is definitely not the last one”. In the first year, choosing a MOOC that fitted DMGT was students’ responsibility and the reactions were various. Some have appreciated this approach: “A significant advantage was that we (students) were not limited by a certain topic, but only by an area that needs to be covered in a MOOC”, while some would have preferred to have a specific MOOC to take: “It would be good to have a specific course as a task, rather than being given the option to choose any course that fits DMGT. Coursera library is very extensive so it took some time to find the appropriate course”. Based on this type of feedback, practice changed in the second year and a list of potential MOOCs was shared in LMS. Self-paced learning was much appreciated; students valued the option to learn when possible and when it suits them. It also helped to have a structure in place to keep them on track: (a) “I was able to plan my time dedicated to learning. The only element to have in mind was the quizzes deadline, where I had three attempts without time limit, which was more than fair”; (b) “Being time-flexible was one of the most important elements of MOOCs”; (c) “Advantage of MOOCs is the possibility to access content anytime, when I was focused and motivated, and interested in that content. Thanks to this, I was able to master the content in a more efficient way – simple and fast”. Students also have had positive experience with more frequent knowledge evaluation: “More frequent knowledge evaluation is far more effective than having two exams per semester”.
Time required to successfully complete the MOOC (question 7)
To fit MOOC in a classroom taught course properly, it is important to value the time spent to complete the MOOC. This aligns with ECTS points awarded for each traditionally taught course in formal education. As mentioned in Table 1, goal was to cover MOOCs with estimated 40-50 hours to completion, to fit approximately 30% of complete DMGT course ECTS load. The actual time spent on completing a MOOC occasionally differs from what is stated on MOOC providers’ websites. Still, most students have shared that the time it took them to complete the MOOC corresponds to what is stated on MOOC homepage. However, common feedback was that the required time can prolong significantly depending on prior knowledge of the subject and consequently speed of completing follow up tasks, as well as on efforts put into studying follow up literature: (a) “In the beginning I was fast with solving problems (…) because I have encountered this content before (1h-2h/week). Later, it took me longer to solve tasks and I needed to go through materials again (4h-5h/week)”; (b) “It took the same amount of hours as stated on Coursera site to complete the MOOC, but to rewind the videos and to fully understand the content, it took twice as much time as suggested”; The role of English as the language of all chosen MOOCs was also significant in actual time required to complete the MOOC: (a) “Some tasks were easy while some required significant effort to master mathematical cryptography terminology in English”; (b) “If a student understands English well, it is possible to follow lectures at a higher playback speed”. Challenging tasks are not merely a time consuming activity; some students report that challenging tasks make MOOC participation more interesting: “Tasks that trigger intensive thinking are the reason why I’m glad I chose MOOC”, but also: “It would be hard to follow MOOC content without basic subject knowledge gained in classroom.”. Interesting feedback in regards to time spent on MOOC was given by a student who stated that “5 hours per week is the optimal amount of time to dedicate to this type of learning, as it’s likely that individuals spend the same amount of time on activities that are not at all connected to university related tasks”.
Discussion and conclusion
After analysing learning diaries and overall student performance on the course let us summarize answers to research questions. Since the small sample is serious limitation of the quantitative part of research conclusions are mostly based on qualitative analysis of students’ diaries.
Can use of MOOCs help in giving students positive learning experience in virtual environment and help part-time students in achieving particular learning outcomes?
According to feedback gathered in the learning diaries, MOOCs have supported learning in virtual environments, providing an experience that was new for majority of students that participated and opening doors to online learning for students. To them, possibility to learn at their own pace was very important. Recognition of the value of forums, discussions, and partnering with other to achieve best results justifies that the MOOC activity is an alternative to the team work that also has a goal to enhance collaboration skills of students. Furthermore, feedback showed that part time students are happy about the opportunity to have an option to manage their learning. This option was mentioned as an improvement opportunity for traditionally taught classes as well. Common student feedback was that this exercise resulted in exploring MOOC platforms and what they have to offer. Authors, after carefully examining answers, strongly believe that students will continue to use MOOC platforms to supplement the classroom teaching, or to expand their knowledge in general. Knowledge assessments have also been accepted well by students; general belief is that regular knowledge assessment increases knowledge retention and reduces stress related to adopting big amount of content.
How to align course learning outcomes and student workload with use of MOOCs in a specific course?
In order to introduce MOOCs into traditional classroom fine tuning with learning outcomes, assessment methods and students’ workload is required. Special attention should be given to student’s workload having in mind students’ prior knowledge and possible language barriers. Therefore, a teacher should check all recommended MOOCs very carefully in advance and estimate student workload. Actual students’ workload for non-native English speaker students is usually higher than listed on an official MOOC declaration. Interestingly, intended learning outcomes (Table 1) were not always recognized by students as covered by MOOC exercise. The possible explanation is that students are not very interested in the pedagogical foundation of the course and the concept of learning outcomes. Students much more easily map concrete content than abstract competences such as problem solving.
What are the main challenges for students in using MOOCs?
Language has been pointed as a barrier for multiple students. Obviously, good command of English language can significantly contribute to the MOOC completion. Still, even though English language was emphasized as a barrier, it was also characterized as a positive challenge an all students successfully finished courses despite of the potential language barrier. Further, students emphasized importance of previous knowledge (mathematics and programming) that enable them to be successful in MOOC despite of the declaration at the beginning of majority of MOOC that no specific prior knowledge has been required. Finally, students appreciate applications and implementations of knowledge as well as more frequent knowledge and skills assessment. Students are also aware that self-motivation and completing tasks in time is required to successfully complete the MOOC, which requires thorough planning of MOOC related activities so they fit in students’ schedule.
To conclude, blending MOOCs in DMGT course resulted in multiple findings and opened further research questions. Even though the model described in this paper is similar to certain models and researches in blended learning, this model is based on learning outcome approach and have student workload and prior knowledge in mind. Furthermore, part-time students perceived offering MOOCs flexibility as an alternative to project team work very positively. Comparing research results to similar research, it was found that it is similar in some segments but differs in others. For example, similar as in research by (Israel, 2015), there was no evidence of negative effects for learners that completed MOOCs. There is not a significant difference in final grade among students who joined project work and those who completed a MOOC. However, unlike in research by (Israel, 2015), all students that took MOOC reported high level of satisfaction. This research is a starting point for further research in blending MOOCs in traditionally taught courses, to detect trends, progress, and generate guidelines for a successful implementation of online content from strategic point of view.
References
- Biggs, J. (2003). Teaching for Quality Learning at University (2nd ed.). Buckingham: Society for Research into Higher Education /Open University Press.
- Bruff, D., O., Fisher, D. F., McEwen, K. E., & Smith, B. E. (2013). Wrapping a MOOC: Student Perceptions of an Experiment in Blended Learning. MERLOT Journal of Online Learning and Teaching, 9(2).
- Divjak, B. (2015). Assessment of complex, non-structured mathematical problems. Presented at the IMA International Conference on Barriers and Enablers to Learning Maths: Enhancing Learning and Teaching for All Learners, M.A. Hersh and M. Kotecha (eds.).
- Ghadiri, K., Qayoumi, M. H., Junn, E., Hsu, P., & Sujitparapitaya, S. (2013). The Transformative Potential of Blended Learning Using MIT edX’s 6.002x Online MOOC Content Combined with Student Team-Based Learning in Class. Retrieved June 8, 2016, from https://www.edx.org/sites/default/files/upload/ed-tech-paper.pdf
- Gilbert, J. A., & Flores-Zambada, R. (2011). Development and Implementation of a “Blended” Teaching Course Environment. MERLOT Journal of Online Learning and Teaching, 7(2). Retrieved from http://jolt.merlot.org/vol7no2/gilbert_0611.htm
- Griffiths, R., Mulhern, C., Spies, R., & Chingos, M. (2015). Adopting MOOCs on Campus: A Collaborative Effort to Test MOOCs on Campuses of the University System of Maryland. Online Learning, 19(2). Retrieved from http://eric.ed.gov/?id=EJ1062937
- Israel, M. J. (2015). Effectiveness of Integrating MOOCs in Traditional Classrooms for Undergraduate Students. International Review of Research in Open and Distributed Learning, 16(5), 102–118.
- Koller, D., Ng, A., Do, C., & Chen, Z. (2013, June 3). Retention and Intention in Massive Open Online Courses: In Depth [Blog post]. EDUCAUSEreview. Retrieved July 17, 2016, from http://er.educause.edu/articles/2013/6/retention-and-intention-in-massive-open-online-courses-in-depth
- Morris, N. P. (2014). How Digital Technologies, Blended Learning and MOOCs Will Impact the Future of Higher Education. Paper presented at the International Conference e-Learning 2014. Multi Conference on Computer Science and Information Systems, Lisbon, Portugal, July 15-19, 2014. International Association for the Development of the Information Society. Retrieved from http://eric.ed.gov/?id=ED557272
- Sharpe, R., Benfield, G., Roberts, G., & Francis, R. (2006). The undergraduate experience of blended e-learning: a review of UK literature and practice. The Higher Education Academy. Retrieved from https://www.heacademy.ac.uk/sites/default/files/sharpe_benfield_roberts_francis_0.pdf
- Yousef, A. M. F., Chatti,
M. A., Schroeder, U., & Wosnitza, M. (2015). A Usability Evaluation of a
Blended MOOC Environment: An Experimental Case Study. International Review
of Research in Open and Distributed Learning, 16(2), 69–93.