NEW

Best of EDEN 2018 Special issue
 

There are not any recent contributions.

There are not any recent contributions.

Archives

EURODL Mailinglist

2851 subcribers
 

EURODL Visitors

back

Lessons Learnt from a Professional Development MOOC: Engaging Culturally and Linguistically Diverse Learners from Low- and Middle-Income Countries

Pascal Launois [launoisp@who.int], UNICEF-UNDP-World Bank-WHO Special Programme for Research and Training in Tropical Diseases (TDR), Avenue Appia 20 1211 Geneva 27, Switzerland, Pascale Allotey [pascale.allotey@unu.edu], United Nations University International Institute for Global Health, Bandar Tun Razak, Cheras, 56000 Kuala Lumpur, Malaysia, Daniel Reidpath [daniel.reidpath@monash.edu], Monash University Malaysia, Bandar Sunway, 47500 Subang Jaya, Selangor, Malaysia, Dermot Maher [maherd@who.int], UNICEF-UNDP-World Bank-WHO Special Programme for Research and Training in Tropical Diseases (TDR), Switzerland, Edith Certain, WHO, Bella Ross [bella.ross@monash.edu.au], Monash University, 900 Dandenong Road Caulfield East VIC 3145, Australia

Abstract

This article reports on the lessons learnt from the pilot of a professional development massive open online course (MOOC) for a culturally and linguistically diverse student cohort located across low- and middle-income countries (LMICs). The focus of the MOOC was implementation research related to infectious diseases in LMICs. It was developed by the Special Programme for Research and Training in Tropical Diseases hosted by the World Health Organisation, in collaboration with the École Polytechnique Fédérale de Lausanne. The pilot took place in late 2017 with a total of 110 participants from 29 countries across 6 continents. Participants were a mix of programme managers, researchers and local health workers. Drawing on online anonymous survey responses gathered on completion of each of the five modules and a focus group conducted with participants from the Americas region, the authors outline participants’ views of the benefits and challenges of the MOOC. Participants reported positive benefits in terms of increased knowledge and professional performance. Challenges related to time requirements, language of instruction, cultural context and online discussions. These findings will be of interest to those developing and facilitating MOOCs in LMICs for professional development purposes, particularly for learners from culturally and linguistically diverse backgrounds.

Abstract in Spanish

En el artículo se explican las enseñanzas adquiridas durante la realización preliminar de un curso en línea masivo y abierto (MOOC, por sus siglas en inglés) para una cohorte de estudiantes de entornos culturales y lingüísticos diversos, en países de ingresos medianos y bajos. El curso, cuyo tema fue la investigación aplicada relacionada con las enfermedades infecciosas en esos países, fue elaborado por el Programa Especial de Investigaciones y Enseñanzas sobre Enfermedades Tropicales albergado por la Organización Mundial de la Salud, en colaboración con la École Polytechnique Fédérale de Lausanne. El curso se impartió a finales de 2017 a 123 personas (directores de programas, investigadores y profesionales sanitarios locales) de 29 países repartidos en los seis continentes. Los autores reseñan las opiniones de los estudiantes sobre las virtudes del curso y también sus objeciones al mismo, a partir de las respuestas a una encuesta anónima en línea realizada al final de cada uno de los cinco módulos de que consta el curso y de un grupo de discusión para los participantes de la Región de las Américas. Los participantes mencionaron, como aspectos positivos, la mejora de los conocimientos y del desempeño profesional, y, como cuestiones que se pueden mejorar, las limitaciones de tiempo, el idioma utilizado, el contexto cultural y los debates en línea. Estos resultados serán de utilidad para todas las personas que preparen y faciliten la elaboración de cursos en línea masivos y abiertos en los países de ingresos medianos y bajos con fines de desarrollo profesional, sobre todo los dirigidos a personas de entornos culturales y lingüísticos diversos.

Abstract in French

Cet article présente les enseignements tirés d’un projet pilote portant sur un MOOC de développement professionnel destiné à une cohorte culturellement et linguistiquement diversifiée d’étudiants issus de pays à revenu faible ou intermédiaire (PRFI). Le MOOC portait sur la recherche sur la mise en œuvre concernant les maladies infectieuses dans les PRFI. Il a été mis au point par le Programme spécial de recherche et de formation concernant les maladies tropicales (TDR) hébergé par l’Organisation mondiale de la Santé, en collaboration avec l’École Polytechnique Fédérale de Lausanne. Le projet pilote s’est déroulé fin 2017 et réunissait au total 123 participants venant de 29 pays situés sur six continents. Ces participants étaient des administrateurs de programme, des chercheurs et des agents de santé locaux. S’appuyant sur les réponses anonymes à une enquête en ligne, recueillies à la fin de chacun des cinq modules, et sur un groupe de discussion mené avec des participants de la Région des Amériques, les auteurs présentent les points de vue des participants sur les bénéfices du MOOC et sur les problèmes qui lui sont associés. Les participants ont fait état d’une amélioration des connaissances et de l’efficacité professionnelle. Les problèmes rapportés étaient liés au temps requis, à la langue d’enseignement, au contexte culturel et aux discussions en ligne. Ces résultats intéresseront ceux qui élaborent et animent des MOOC dans les PRFI à des fins de perfectionnement professionnel, en particulier pour des apprenants d’horizons culturels et linguistiques divers.

Keywords: massive open online courses (MOOCs); professional development; culturally and linguistically diverse (CALD) learners; online learning; continuing education; low- and middle-income countries; developing countries.

Introduction

In late 2017, a massive open online course (MOOC) on implementation research (IR) with a focus on infectious diseases of poverty developed for professionals in low- and middle-income countries (LMICs) was piloted. The MOOC was developed by the Special Programme for Research and Training in Tropical Diseases (TDR) co-sponsored by UNICEF-UNDP-World Bank and the World Health Organisation (WHO) and established at WHO, in collaboration with the École Polytechnique Fédérale de Lausanne (EPFL). Participants in the MOOC pilot who were invited to take part were researchers, programme managers and health workers.

Drawing on the experiences of the participants piloting the MOOC, this article identifies a range of lessons that were learned from the process which are presented here according to whether participants viewed them as benefits or challenges. This preliminary research seeks to add to the limited literature of how linguistically and culturally diverse learners from LMICs engage in a MOOC designed for professional development purposes. Findings will be of value to those developing MOOCs for professional development purposes in LMIC contexts, particularly for cohorts of learners that are culturally or linguistically diverse.

Background

MOOCs have increased in popularity in the decade since they were conceived of with just over 100 million enrolments in 2018 (Shah, 2018). As indicated by the name, MOOCs are open to all, yet participants are typically well-educated and employed, with a majority from high-income countries (Christensen et al., 2014; DeBoer, Ho, Stump, & Breslow, 2014; Escher et al., 2014; Liyanagunawardena, Lundqvist, & Williams, 2015; van de Oudeweetering & Agirdag, 2018).

Research shows that MOOCs are a valuable source of professional development and have been widely used for this purpose worldwide, including in LMICs, for a range of learners (Christensen et al., 2014; Czerniewicz, Deacon, Small, & Walji, 2014; Deacon, Jawitz, Small, & Walji, 2017; Garrido et al., 2016; Hrdličková & Dooley, 2017; Laurillard, 2016; Milligan & Littlejohn, 2017; Mori & Ractliffe, 2016; Murugesan, Nobes, & Wild, 2017; Yuan & Powell, 2013).

Overwhelmingly, MOOCs are offered in English (Brouns et al., 2015; Colas et al., 2016) followed by French (Noukakis, Escher, & Aebischer, 2016) and come from either a western or American perspective (Altbach, 2014). As such, the majority of MOOC users are not as diverse as originally expected (Albelbisi, Yusop, & Salleh, 2018). Although MOOCs are increasingly used for learners from different cultures and language groups (Fitzgerald, Wu, & Witten, 2014; Noukakis et al., 2016), much research of MOOCs is based on U.S. and Western contexts (Bozkurt, Akgün-Özbek, & Zawacki-Richter, 2017; Jansen et al., 2015).

MOOCs have potential to provide education on a large scale to learners in LMICs (McGreal, 2017). Indeed, in recent years, an increasing number of MOOCs have been developed by agencies such as the United Nations and WHO on topics such as health and sustainability. These aim to raise awareness, enhance local capability and provide training and professional development on these topics (Weybrecht, 2018).

Research on MOOCs in Low- and Middle-Income Countries

MOOCs were expected to democratise or increase equality in education, but some studies indicate they may not have delivered on this. Studies reveal that MOOC participation in LMICs is considerably lower than that of high-income nations, particularly regarding learners from Asia, Africa and Southeast Asia (Liyanagunawardena, Adams, & Williams, 2013; Liyanagunawardena, Williams, & Adams, 2013). Other studies find that MOOC participants from LMICs account for approximately one third of Coursera users (MacGregor, 2014) or almost half of all edX users (Palin, 2014). Almost two thirds of users from LMICs are under 30 years of age in contrast to U.S. learners where those under 30 account for just under a quarter of all learners (Palin, 2014).

The educational levels of MOOC participants are higher than their fellow citizens (Christensen et al., 2014) suggesting that rather than opening up education, MOOCs may reinforce traditional education patterns (Yuan & Powell, 2013). A further concern is that MOOC content is primarily offered in English (MacGregor, 2014). In LMICs, inequalities and challenges in online education may arise due to a lack of focus on local social, cultural, economic and political contexts and issues with language, access and computer literacy (D’Antoni & Savage, 2009; Fahmy, Bygholm, & Jæger, 2013; Kanwar, Kodhandaraman, & Umar, 2010; King, Luan, & Lopes, 2018; King, Pegrum, & Forsey, 2018; Liyanagunawardena, Williams, & Adams, 2013). In response to such limitations, the EPFL in collaboration with African universities developed a series of MOOCs to respond to the urgent needs of capacity building of these countries (Noukakis et al., 2016).

There is limited research into MOOCs in LMICs (Bulger, Bright, & Cobo, 2015; Deacon et al., 2017; Hrdličková & Dooley, 2017; King, Luan, & Lopes, 2018; King, Pegrum, & Forsey, 2018; Liyanagunawardena, Williams, & Adams, 2013; Murugesan et al., 2017; Sukhbaatar, Choimaa, & Usagawa, 2018; Zhenghao et al., 2015) or online education in LMICs generally (Georgsen & Zander, 2013; Nti, 2015). More recently, however, has seen an increase in research aiming to address this gap in the literature, including for example the Advancing MOOCs for Development Initiative (International Research & Exchanges Board, 2019).

MOOC participants in LMICs are more likely to report career and educational benefits than their counterparts in high-income countries (Zhenghao et al., 2015). In LMICs, MOOCs are used primarily to gain specific job skills, to prepare for additional education, and to obtain professional certification (Garrido et al., 2016). Employers and government acknowledge the value of MOOCs to increase professional skills and knowledge and, interestingly, MOOCs offered by government institutions are gaining in prominence for professional development purposes as they are more likely to link to government certification (Garrido et al., 2016).

Recent studies of MOOCs in LMICs indicate contrasting findings to those in high-income countries including the U.S. For example, MOOCs in high-income countries generally reveal low completion rates of up to 10% (Hew & Cheung, 2014; Breslow et al., 2013; Zhenghao et al., 2015), though considerably higher completion rates are found in studies of MOOCs for professional development purposes in LMICs of 30% (Garrido et al., 2016) and between 47 and 68% (Hrdličková & Dooley, 2017; Murugesan et al., 2017). These percentages, however, should be viewed with caution as modules are often commonly completed for the sole purpose of gaining required skills. This is particularly the case in Africa, as participants otherwise do not have reason to complete an entire MOOC (Noukakis et al., 2016).

MOOC users in high-income countries tend to be male, relatively young and well-educated (Christensen et al., 2014; Emanuel, 2013; Zhenghao et al., 2015), while the majority (80%) of those in LMICs have lower socio economic status (Garrido et al., 2016). Such contrasting findings suggest the need for further investigation into, and indeed development of, MOOCs in the LMIC context (Garrido et al., 2016; King, Luan, & Lopes, 2018).

Learner Engagement in MOOCs

Although many studies have addressed learner engagement and interaction in MOOCs (Crosslin et al., 2018; Joksimović et al., 2018), there is limited research into learner engagement and interaction within culturally and linguistically diverse learner cohorts. Few studies address the issue of how best to assist learners who are not fluent in the language of instruction and online participation, which is most often English. Those that do have found that learners may be reluctant to join online discussions in a language they are not comfortable using (Garreta-Domingo, Hernández-Leo, Mor, & Sloep, 2015). In multilingual MOOCs, therefore, providing multilingual facilitation may activate participation (Colas et al., 2016).

The MOOC pilot and participants

The IR MOOC was designed as a resource for programme managers, researchers and health workers across South America, Africa and Asia. The aim of the MOOC is to educate participants about what IR is and how it can be used in practice when working with infectious diseases. As such, any professional with an interest in IR would benefit from the MOOC. The MOOC opened in October 2017. On successful completion of the MOOC and all of its assessments, participants would receive a certificate of attendance.

The MOOC was hosted on the edX platform and offered by TDR. Prior to the pilot, an education consultant was assigned to develop facilitator resources, monitor and evaluate the first iteration of the MOOC.

The Structure of the MOOC

The MOOC consisted of five modules and took approximately five weeks to complete. Each module was expected to take a week to complete at the participants’ own pace. There was no synchronous teaching scheduled, which was largely due to the geographical locations of participants spanning several continents and time zones. This made it difficult to coordinate a time that would allow all learners to participate.

The five modules of the MOOC are as follows:

• Module 1: What is Implementation Research?
• Module 2: Needs Assessment for Implementation Research
• Module 3: Designing Implementation Strategies
• Module 4: Implementation Research Outcomes
• Module 5: Implementation Research in Practice

Each module consists of information videos, most of which were in English (with some Portuguese and French) with linked English captions, links for readings, links to a discussion forum focusing on a new topic for each module, and an assessment task for that module. The assessment tasks in modules 1 and 2 were quizzes. The assessment tasks in modules 3, 4 and 5 contained a peer assessment component.

Participant Demographics

There were 110 participants who were invited to take part in the pilot. Participants came from 3 groups:

• Group 1: 77 selected from LMICs
• Group 2: 18 selected by the WHO African Communicable Diseases Cluster
• Group 3: 15 WHO Headquarters

TDR coordinates an initiative named Regional Training Centres supported by TDR with the aim of institutionalising and disseminating short training courses on good health research practice and IR. TDR asked each Regional Training Centre to identify participants for the MOOC. Group 1 consisted of participants from the African (AFR), Americas (AMR), Eastern Mediterranean (EMR), European (EUR), South-East Asian (SEAR) and Western Pacific (WPR) WHO regions. This group was a combination of researchers and programme managers. See Figure 1 for an overview of where group 1 participants were located.

Figure 1. Group 1 participant locations

Group 2 consisted of participants from WHO country programmes who were control managers (mainly HI/TB and malaria) largely from Ethiopia (10) followed by South Africa (3). Figure 2 provides an overview of the countries where group 2 participants were located.

Figure 2. Group 2 participant locations

Group 3 comprised participants employed at WHO headquarters located in Geneva, Switzerland in the following areas: TDR, the Global Malaria Programme (GMP), the Human Reproduction Programme (HRP), the Global Tuberculosis Programme (GTB) and the Global Health Workforce Alliance (Alliance). See Figure 3.

Figure 3. Group 3 participant locations

Of the MOOC participants, 19 (17%) obtained a certificate of completion.

Data collection and analysis

The data used in this article comprises online anonymous survey data gathered from the MOOC participants on completion of each module of the MOOC, including an exit survey administered after assignments had been assessed on completion of the MOOC. Participant demographics were collected from the exit survey. The exit survey comprised 28 questions which included socio-demographic information (including gender), the language of instruction, the levels of education, the reasons for taking the MOOC, how participants rated the entire MOOC and its components (i.e. videos, case studies, etc.), and their perception of their understanding of IR before and after the MOOC. An open question was added to gauge overall participant satisfaction. Of the participants, 35 (31%) responded to the final exit survey. Several of the survey questions made use of a 4-point Likert scale to evaluate participant agreement with statements, with 1 the lowest and 4 the highest on the scale.

A focus group with the representative of the Americas region was conducted in November 2017 with the first, second, third and sixth authors. Participants in this region had met online weekly during the MOOC to discuss their experiences and the representative recounted these discussions during the focus group.

Findings and discussion

The findings presented in this article are divided into two main categories: benefits and challenges.

Benefits

The data revealed three main category of benefits as identified by the MOOC participants: (a) an increased knowledge of the topic, (b) impacts on their professional performance, and (c) potential future impacts. These are discussed in the following sections.

Increased knowledge of implementation research

The MOOC team specifically wanted to know how much participants perceived that their understanding of IR had improved as a result of completing the MOOC. Table 1 reveals that most participants felt that there was a strong improvement (ranging from 45.8 to 69.2%) or some improvement (ranging from 30.8 to 54.2%) of their understanding of IR.

Table 1:   Participants’ perceptions of their improvement in understanding of IR

The exit survey asked participants to reflect on their perceived understanding of IR before commencing the MOOC and after its completion. See Table 2. Overall, most participants felt that they had a weak (35.3%) to moderate (41.2%) understanding of IR before commencing the MOOC. On completion of the MOOC, however, most participants felt that they had a strong understanding of IR (73.5%). No participants felt that they had no or a weak understanding of the topic on completion of the course. These changes reveal that participants perceived that the course had provided them with expertise and knowledge of IR.

Table 2:   Exit survey of participants’ understanding of IR before and after completing the MOOC

Exit survey responses confirmed these positive findings. On conclusion of the MOOC, participants reflected on the learning journey as positive:

“This has been a great learning experience. It has laid the foundation for me to learn more and impact this knowledge onto others.” (Exit survey)

Participants reflected that the use of real cases aided their learning:

“Overall I found this course very useful and helpful to understand IR from a real life setting perspective.” (Exit survey)

The materials provided in the MOOC were identified as contributing to learning outcomes:

“The MOOC overall is a really good experience!! This tool, plus the IR toolkit, is a powerful combination for learning and strengthening this area of research.” (Exit survey)

Moreover, some felt the benefits of the MOOC should be promoted beyond the research community:

“This is a very educative course that needs to be promoted beyond the research community.” (Exit survey)

Overall then, participants indicated that the MOOC had been a positive learning experience that had contributed to their knowledge of IR. In this respect, the MOOC achieved its aims.

Impact on professional performance

On completion of each module, participants were asked whether they perceived that the knowledge they gained from the MOOC would contribute to them performing better in their current or future roles. See Table 3. For all modules, the majority strongly agreed followed by slightly agreed with this proposition indicating that the participants felt clear value in participating in the course.

Table 3:   Participant responses to whether they expected to perform better in current or future jobs as a result of the module

Survey comments indicated that the knowledge gained from the course was useful for professional purposes. One commented that the IR MOOC would be useful for training purposes:

“This is a very good course to me. This course is good for a training of trainer in IR.” (Exit survey)

Other participants commented on the value of the MOOC to their professional understanding of the topic and their ability to use it in practice:

“I found that I learnt most from the assessments, and especially assessing my peers’ work. The latter often helped me to better understand where I may have misunderstood and to drive home certain ideas. Given other work priorities, I did not read as widely as I would have liked. Even so, I feel that I have a strong understanding of the objects of inquiry of IR, and how to set up such phased implementation strategies and evaluations.” (Exit survey)

These positive views of their learning contribute to participants’ motivation for undertaking a MOOC and may be a key component to course completion (Salmon, Pechenkina, Chase, & Ross, 2017; Hakami, White, & Chakaveh, 2017). Such comments reveal that the MOOC achieved its aim of increasing participants’ perceptions of their professional knowledge.

Future impacts

The researchers were interested in investigating the degree to which participants felt the IR MOOC would impact on their future studies on work. Participants were asked whether the course had inspired them to pursue the topic of IR further. The majority of respondents strongly or slightly agreed that this was the case. See Table 4.

Table 4:   Participant responses to whether the course inspired them to pursue the topic further

Overall then, the participants found personal and professional benefits to participating in the MOOC. Whether participants pursue IR in their student or professional lives is the topic of planned future research. The next section will address the challenges perceived by participants.

Challenges

The following three areas were identified as challenges by participants: (a) time requirements, (b) discussion forums and (c) the cultural and linguistic context of the MOOC. Access issues were also flagged by a single participant. Each of these areas are discussed in further detail in the following sections.

Time requirements

As this MOOC is aimed at working professionals, time considerations were one of the key areas of focus for the research team. Participants were asked if they felt that the time allocated for modules was adequate. Responses in Table 5 indicate that participants generally perceived that the time requirements for modules 1 to 4 were adequate. This is an important finding as research has shown that time requirements may prevent students from completing a MOOC (Belanger & Thornton, 2013; Eriksson, Adawi, & Stöhr, 2017). Limited time for working professionals to devote to participating in MOOCs has been documented in studies in other LMICs (King, Luan, & Lopes, 2018).

Table 5:   Agreement rating that the time allocation for the MOOC was adequate

Participant drop-out rates increased for modules 3 and 4, which coincided with peer assignment due dates. This may be an indication that participants did not have time to develop these assignments. For this reason, subsequent iterations of the MOOC have replaced the peer assessments in these modules with quizzes.

Interestingly, however, on completion of the MOOC, many participants stated that time allocations for tasks and assessment deadlines needed to be re-evaluated to better suit the needs of working health professionals. Representative exit survey quotes include: “The deadline to complete the assessment and the course was a bit short. Better to give more time”; “Time is something that needs to be evaluated. If we want public health workers taking the course, they will need more time to complete the modules and the assignments” and “But the time of this pilot is too limited for a full time professional to complete all readings and some assignments before the deadline”. Whether the differences in views apparent between the mid-course and exit survey are a result of participants having time to reflect on the overall MOOC experience on completion remains to be seen.

Importantly, and particularly for future offerings of this MOOC, several participants indicated that they were unable to complete the course due to time constraints:

“Excellent session. But I regret because I did finish and I did not do the assessment because of time. Sorry. I was too much work for me to read the proposed articles and participate to the discussions. Really I appreciate the pedagogy, the cases taken in real life. I hope that another time, I will succeed in finishing Thank you!” (Exit survey)

Some had suggestions for how to address timing and workload issues:

“If the intention of the course is that it can be taken by health officials or people working in health, there should be alternatives in the deadlines for the assessments. If in a particular week the student have a field trip or a meeting of several days, it is almost impossible to meet the deadline and the materials are reviewed very superficially. It is suggested to teach the MOOC with a few weeks of lag with another section of the same course so that if a student falls behind, they can join another section and complete the course.” (Exit survey)

Focus group findings from the Americas region likewise reveal that participants felt the MOOC was too fast-moving and that more time was required to complete each module. As such, they suggested more flexibility in terms of the module and assessment timelines in order to accommodate competing workloads.

These findings confirm the importance of developing realistic workloads and timeframes for completing MOOC requirements. Indeed, for a MOOC aimed at working professionals to be successful, finding the right balance and timing is crucial.

Discussion forums

One area identified by the MOOC facilitators and participants as needing improvement was the discussion forums. During the pilot, the facilitator team acknowledged that it was a challenge to get participants engaged and discussing the issues covered in each module. Facilitating engaged and quality participation in online discussion forums is known to be challenging in MOOCs due to low participation rates, slow responses and the quality of discussion (Breslow et al., 2013; Hew & Cheung, 2014; Kirschner, 2012; Koutropoulos et al., 2012; Watson et al., 2016).

A further issue identified by both the facilitator team and participants was confusion caused by the format and navigation of discussion topics. Unfortunately, anyone who has taken a MOOC or online course recognises these issues which are well-documented in the research (Kirschner, 2012; Watson et al., 2016). These challenges were all encountered in the IR MOOC as reflected in the following participant quote:

“The discussion forums are a very good tool to improve learning and understanding. However it is always difficult to engage participants. Some of the important topics that were posted in the discussion board didn’t end in a good engaging discussion. We should think in strategies to motivate participants in next courses to engage in discussion.” (Exit survey)

One participant suggested increasing facilitator presence to address this issue:

“It would be nice to have a more active facilitator in the discussion forums so that it actually is a discussion instead of students just writing their answers.” (Exit survey)

On completion of the MOOC, participants were asked which of the components of the course contributed most and least to their learning. See Table 6. The components listed by participants as contributing the most to their learning were case studies followed by the readings assessments and videos. The component that participants overwhelmingly listed as contributing the least to their learning was the discussion forums.

Table 6:   Components of the MOOC that contributed most to participants’ learning

A further contributing factor to the lack of activity on the discussion forums may be attributed to the cultural and linguistic diversity of the learner cohort, particularly given that some participants did not feel comfortable writing about the topic in English. This is discussed in the following section.

Cultural and linguistic contexts

In terms of cultural and geographical context, several comments indicated that participants would like content from more varied contexts and settings. A majority of the examples presented and discussed in the IR MOOC were from African contexts. Representative quotes include:

“Overall I found this course very useful and helpful to understand IR from a real life setting perspective. The way the concepts were explained were making the content of course really palatable and easy. Still more case studies from different settings could be used.” (Exit survey)

“The videos are great, however it would be wonderful if they include more life examples, as the ones included in module one (e.g. the women looking for health care for her child with malaria). A combination of the lecture and real life examples are a very nice combination that sensitize the learning process. It would be great to include additional case studies from other countries in Latin America and the Caribbean region.” (Exit survey)

Focus group findings also revealed that the participants from the Americas region wanted the MOOC to include examples from South America in order to make the examples and discussions more relevant to their local contexts.

The finding that there is a lack of local, contextualised content in MOOCs and online education generally has been raised in a number of recent studies (Czerniewicz et al., 2014; King, Luan, & Lopes, 2018; King, Pegrum, & Forsey, 2018; Nkuyubwatsi, 2014; Nti, 2015). It has been suggested that adapting content to local contexts may benefit learners (Castillo et al., 2015; Daniel et al., 2015; Nkuyubwatsi, 2014) and increase retention rates (Richter & McPherson, 2012).

In regard to language usage, the focus group findings revealed that participants from the Americas region were translating the MOOC video transcript into Spanish. They stated that they would prefer to use Spanish in their discussion forums as many participants struggled to write about this topic in English. Several surveyed participants likewise raised the issue of the language of instruction, both in relation to the choice of language used for the MOOC content (mainly English) and on discussion forums. Some stated that they would appreciate the course being taught in other languages. For example, one stated:

“I hope the IR MOOC will be available in Spanish some time soon.” (Exit survey)

Others appreciated the transcripts of the videos which aided their comprehension: “The texts besides the video to follow the teacher are excellent, it really helps to go back to any concept that was not clear and for people whose first language is not English is a real help” (Exit survey).

Studies have found that the language of instruction may affect learners’ willingness and ability to participate in online discussions (Garreta-Domingo et al., 2015; Liyanagunawardena, Williams, & Adams, 2013) and that offering multi-lingual facilitation may positively influence online participation (Colas et al., 2016).

Access

An important consideration when developing a MOOC, though not one that was common in the data, is that of access issues. One participant studying in China raised the issue of accessing certain course materials:

“In developing countries, it is hard to access all the resources from the internet. For myself, I can access the resources but can’t access the course video in Shanghai, China.” (Exit survey)

Access issues have been raised and documented in studies of MOOCs in other LMICs (Bartholet, 2013; King, Luan, & Lopes, 2018; King, Pegrum, & Forsey, 2018; Wildavsky, 2015) and is an important consideration for MOOC developers. For this MOOC cohort, however, access to a stable internet connection and a computer or device were most likely not relevant issues, as the participants were all working professionals who were invited to participate in the MOOC.

Conclusion and recommendations

The MOOC pilot study offered important insights into how to develop, or in the IR MOOC case, further refine, a professional development MOOC for culturally and linguistically diverse learners from LMICs. The findings from the pilot IR MOOC reveal that participants generally felt that the MOOC had improved their understanding of the topic considerably, was useful as a professional learning tool, would be of benefit to their careers, and that it had inspired them to pursue the topic further in future. In this respect, the MOOC was successful in achieving its outcomes.

The most commonly cited challenges focused on time requirements, the discussion forums, language and contextual issues with the most cited challenge pertaining to time. Working professionals link realistic time requirements and deadlines with MOOC participation and completion. For this reason, current iterations of the MOOC have replaced the peer assessments in several modules with quizzes, which are less time-intensive.

The pilot highlights the importance of adapting content to local and linguistic contexts. Based on feedback from MOOC participants and research citing the importance of providing content for local context, future iterations of the MOOC will increase the use of example from regions other than Africa.

Current iterations of the IR MOOC are offered in English with subtitles in English, French and Spanish. Furthermore, based on the pilot study, future iterations of the IR MOOC will offer online discussion groups in several languages (where feasible) and allow participants to select to join a group in their own language or English. In this case, allowing learners to self-select a language group may be one possible approach to encourage online engagement and discussion. Moreover, conversation topics may be grouped according to country or region as particular geographical areas have their own unique concerns.

Education designers faced with the issue of facilitating online discussions between students of culturally and linguistically diverse backgrounds need to identify how best to encourage participants to contribute to online discussions in a way that is meaningful to their learning. Whether this includes tailoring content to local contexts and language to suit learners’ needs are important considerations for MOOC developers, particularly in non-western contexts.

Based on these findings, this paper provides recommendations for those developing MOOCs in similar contexts and outlines the steps that were taken to improve the subsequent offering of the MOOC. This iteration of the MOOC took place in 2018 with approximately 1600 participants from 106 countries registered.

Further research is planned to investigate how the IR MOOC participants have used the knowledge gained from the course, by for example exploring whether they have initiated any relevant research projects as a result of participating in the MOOC.

References

1. Albelbisi, N., Yusop, F. D., & Salleh, U. K. M. (2018). Mapping the factors influencing success of massive open online courses (MOOC) in Higher Education. Eurasia Journal of Mathematics, Science and Technology Education, 14(7), 2995–3012. doi:10.29333/ejmste/91486
2. Altbach, P. (2014). MOOCs as neocolonialism: Who controls knowledge? International Higher Education, 75, 5-7. doi:10.6017/ihe.2014.75.5426
3. Bartholet, J. (2013). Hype and hope. Scientific American, 309(2), 53–61. doi:10.1038/scientificamerican0813-53
4. Belanger, Y., & Thornton, J. (2013). Bioelectricity: A quantitative approach, Duke University’s First MOOC. Duke Center for Instructional Technology. Retrieved from http://hdl.handle.net/10161/6216
5. Bozkurt, A., Akgün-Özbek, E., & Zawacki-Richter, O. (2017). Trends and patterns in massive open online courses: Review and content analysis of research on MOOCs (2008-2015). The International Review of Research in Open and Distributed Learning, 18(5). doi:10.19173/irrodl.v18i5.3080
6. Breslow, L., Pritchard, D. E., DeBoer, J., Stump, G. S., Ho, A. D., & Seaton, D. T. (2013). Studying learning in the worldwide classroom: Research into edX’s first MOOC. Research & Practice in Assessment, 8, 13–25. doi:10.19173/irrodl.v18i5.3080
7. Brouns, F., Serrano Martínez-Santos, N., Civera, J., Kalz, M., & Juan, A. (2015). Supporting language diversity of European MOOCs with the EMMA platform. In M. Lebrun, M. Ebner, I. de Waard, & M. Gaebel (Eds.), Proceedings of the 3rd European MOOCs Stakeholder Summit (eMOOCs2015) (pp. 157–165). Mons, Belgium. Retrieved from http://hdl.handle.net/1820/6026
8. Bulger, M., Bright, J., & Cobo, C. (2015). The real component of virtual learning: Motivations for face-to-face MOOC meetings in developing and industrialised countries. Information, Communication & Society, 18(10), 1200–1216. Retrieved from https://ssrn.com/abstract=2641798
9. Castillo, N. M., Lee, J., Zahra, F. T., & Wagner, D. A. (2015). MOOCS for development: Trends, challenges, and opportunities. International Technologies & International Development, 11(2), 35–42. Retrieved from https://itidjournal.org/index.php/itid/article/view/1396/517
10. Christensen, G., Steinmetz, A., Alcorn, B., Bennett, A., Woods, D., & Emanuel, E. J. (2014, April 18). The MOOC phenomenon: Who takes massive open online courses and why? SSRN. doi:10.2139/ssrn.2350964
11. Colas, J., Sloep, P., & Garreta-Domingo, M. (2016). The effect of multilingual facilitation on active participation in MOOCs. The International Review of Research in Open and Distributed Learning, 17(4). doi:10.19173/irrodl.v17i4.2470
12. Crosslin, M., Dellinger, J., Joksimović, S., Kovanović, V., & Gašević, D. (2018). Customizable modalities for individualized learning: Examining patterns of engagement in dual-layer MOOCs. Online Learning, 22(1), 19–38. doi:10.24059/olj.v22i1.1080
13. Czerniewicz, L., Deacon, A., Small, J., & Walji, S. (2014). Developing world MOOCs: A curriculum view of the MOOC landscape. Journal of Global Literacies, Technologies, and Emerging Pedagogies, 2(3), 122–139. Retrieved from http://www.joglep.com/files/7614/0622/4917/2._Developing_world_MOOCs.pdf
14. Daniel, J., Cano, E. V., & Cervera, M. G. (2015). The future of MOOCs: Adaptive learning or business model? International Journal of Educational Technology in Higher Education, 12(1), 64–73. doi:10.7238/rusc.v12i1.2475
15. D’Antoni, S., & Savage, C. (Eds.). (2009). Open educational resources: Conversations in cyberspace. Clamecy, France: UNESCO Publishing. Retrieved from https://unesdoc.unesco.org/ark:/48223/pf0000181682
16. Deacon, A., Jawitz, J., Small, J., & Walji, S. (2017). MOOCs and transitions: Pathways in and out of learning and work. Proceedings of the 4^th Biennial Conference of the South African Society for Engineering Education, 60–69. Retrieved from https://www.sasee.org.za/wp-content/uploads/Proceedings-of-the-4th-Biennial-SASEE-Conference-2017.pdf
17. DeBoer, J., Ho, A. D., Stump, G. S., & Breslow, L. (2014). Changing “course” reconceptualizing educational variables for massive open online courses. Educational Researcher, 43(2), 74–84. doi:10.3102/0013189X14523038
18. Emanuel, E. J. (2013). Online education: MOOCs taken by educated few. Nature, 503(7476), 342. doi:10.1038/503342a
19. Eriksson, T., Adawi, T., & Stöhr, C. (2017). “Time is the bottleneck”: A qualitative study exploring why learners drop out of MOOCs. Journal of Computing in Higher Education, 29(1), 133–146. doi:10.1007/s12528-016-9127-8
20. Fahmy, S. S. Y., Bygholm, A., & Jæger, K. (2013). Exporting a Scandinavian learning model to Egypt and Vietnam: Challenges and implications. Changing education through ICT in developing countries. In M. Georgsen & P. O. Zander (Eds.), Changing education through ICT in developing countries (pp. 83–101). Aalborg, Denmark: Aalborg Universitetsforlag. Retrieved from https://vbn.aau.dk/ws/portalfiles/portal/202169415/Changing_Education_OA_version.pdf
21. Fitzgerald, A., Wu, S., & Witten, I. (2014). Flexible open language education for a multilingual world. Proceedings of OpenCourseWare Consortium Global 2014: Open Education for a Multicultural World, Ljubljana, Slovenia. Retrieved from http://cdlh7.free.fr/OCWC_2014/Final_papers/Paper_65.pdf
22. Garreta-Domingo, M., Hernández-Leo, D., Mor, Y., & Sloep, P. (2015). Teachers’ perceptions about the HANDSON MOOC. In G. Conole, C. Rensing, J. Konert, & D. Hutchison (Eds.), Design for Teaching and Learning in a Networked World: Proceedings of the 10^th European Conference on Technology Enhanced Learning, Toledo, Spain (pp. 420–427). Berlin/Heidelberg: Springer. Retrieved from http://hdl.handle.net/1820/9070
23. Garrido, M., Koepke, L., Andersen, S., Mena, A., Macapagal, M., & Dalvit, L. (2016). An examination of MOOC usage for professional workforce development outcomes in Colombia, the Philippines & South Africa. Seattle: Technology & Social Change Group, University of Washington Information School. Retrieved from https://tascha.uw.edu/publications/an-examination-of-mooc-usage-for-professional-workforce-development-outcomes/
24. Georgsen, M., & Zander, P.-O. (Eds.) (2013). Changing education through ICT in developing countries. Aalborg, Denmark: Aalborg Universitetsforlag. Retrieved from https://vbn.aau.dk/ws/portalfiles/portal/202169415/Changing_Education_OA_version.pdf
25. Hakami, N., White, S., & Chakaveh, S. (2017). Motivational factors that influence the use of MOOCs: Learners’ perspectives. Proceedings of the 9^th International Conference on Computer Supported Education (CSEDU 2017), 323–331. Porto, Portugal. doi:10.5220/0006259503230331
26. Hew, K. F., & Cheung, W. S. (2014). Students’ and instructors’ use of massive open online courses (MOOCs): Motivations and challenges. Educational Research Review, 12, 45–58. doi:10.1016/j.edurev.2014.05.001
27. Hrdličková, Z., & Dooley, G. (2017). AuthorAID capacity development impact study 2017 (Summary Report). Retrieved from https://www.inasp.info/AuthorAIDstudy
28. International Research & Exchanges Board (2019). Advancing MOOCs for Development Initiative. Retrieved from https://www.irex.org/project/advancing-moocs-development-initiative-amdi
29. Jansen, D., Schuwer, R., Teixeira, A., & Aydin, C. (2015). Comparing MOOC adoption strategies in Europe: Results from the HOME project survey. The International Review of Research in Open and Distributed Learning, 16(6), 116–136. doi:10.19173/irrodl.v16i6.2154
30. Joksimović, S., Poquet, O., Kovanović, V., Dowell, N., Mills, C., Gašević, D., Dawson, S., Graesser, A., & Brooks, C. (2018). How do we model learning at scale? A systematic review of research on MOOCs. Review of Educational Research, 88(1), 43–86. doi:10.3102/0034654317740335
31. Kanwar, A., Kodhandaraman, B., & Umar, A. (2010). Toward sustainable open education resources: A perspective from the Global South. American Journal of Distance Education, 24(2), 65–80. doi:10.1080/08923641003696588
32. King, M., Luan, B., & Lopes, E. (2018). Experiences of Timorese language teachers in a blended massive open online course (MOOC) for continuing professional development (CPD). Open Praxis, 10(3), 279–287. doi:10.5944/openpraxis.10.3.840
33. King, M., Pegrum, M., & Forsey, M. (2018). MOOCs and OER in the Global South: Problems and potential. The International Review of Research in Open and Distributed Learning, 19(5). doi:10.19173/irrodl.v19i5.3742
34. Kirschner, A. (2012, October 5). A pioneer in online education tries a MOOC. The Chronicle of Higher Education [Blog post]. Retrieved from http://chronicle.com.ezproxy.lib.monash.edu.au/section/About-the-Chronicle/83
35. Koutropoulos, A., Gallagher, M. S., Abajian, S. C., de Waard, I., Hogue, R. J., Keskin, N. O., & Rodriguez, C. O. (2012). Emotive vocabulary in MOOCs: Context and participant retention. European Journal of Open, Distance and E-Learning, 2012(I). Retrieved from http://www.eurodl.org/materials/contrib/2012/Koutropoulos_et_al.pdf
36. Laurillard, D. (2016). The educational problem that MOOCs could solve: Professional development for teachers of disadvantaged students. Research in Learning Technology, 24. doi:10.3402/rlt.v24.29369
37. Liyanagunawardena, T. R., Adams, A., & Williams, S. A. (2013). MOOCs: A systematic study of the published literature 2008–2012. The International Review of Research in Open and Distributed Learning, 14(3), 202–227. doi:10.19173/irrodl.v14i3.1455
38. Liyanagunawardena, T. R., Lundqvist, K. Ø., & Williams, S. A. (2015). Who are with us: MOOC learners on a FutureLearn course. British Journal of Educational Technology, 46(3), 557–569. doi:10.1111/bjet.12261
39. Liyanagunawardena, T. R., Williams, S. A., & Adams, A. (2013). The impact and reach of MOOCs: A developing countries’ perspective. E-Learning Papers, 33. Retrieved from http://centaur.reading.ac.uk/32452/
40. MacGregor, K. (2014, December 2). MOOC providers versus sceptics at ‘going global’. University World News [Blog post]. Retrieved from http://www.universityworldnews.com/article.php?story=20140502101632495
41. McGreal, R. (2017). Special report on the role of open educational resources in supporting the sustainable development goal 4: Quality education challenges and opportunities. The International Review of Research in Open and Distributed Learning, 18(7). doi:10.19173/irrodl.v18i7.3541
42. Milligan, C., & Littlejohn, A. (2017). Why study on a MOOC? The motives of students and professionals. The International Review of Research in Open and Distributed Learning, 18(2). doi:10.19173/irrodl.v18i2.3033
43. Mori, K., & Ractliffe, L. (2016). Evaluating the use of a MOOC within higher education professional development training. Proceedings of the 25^th International World Wide Web Conference, 831–833, Montreal, Canada. Retrieved from https://hartpury.pure.elsevier.com/en/publications/evaluating-the-use-of-a-mooc-within-higher-education-professional
44. Murugesan, R., Nobes, A., & Wild, J. (2017). A MOOC approach for training researchers in developing countries. Open Praxis, 9(1), 45–57. doi:10.5944/openpraxis.9.1.476
45. Nkuyubwatsi, B. (2014). Cultural translation in massive open online courses (MOOCs). eLearning Papers, 37(March), 1–10. Retrieved from https://oerknowledgecloud.org/content/cultural-translation-massive-open-online-courses-moocs
46. Noukakis, D., Escher, G., & Aebischer, P. (2016). De Lausanne à Yaoundé: l’aventure des MOOCs Pratiques et recherche sur les MOOCs, à Lausanne et en Afrique francophone. Réalités Industrielles, Mai. Retrieved from http://www.annales.org/ri/2016/ri-mai-2016/RI-mai-2016-Article-NOUKAKIS-ESCHER-AEBISCHER.pdf
47. Nti, K. (2015). Supporting access to open online courses for learners of developing countries. International Review of Research in Open and Distance Learning, 16(4), 156–171. doi:10.19173/irrodl.v16i4.2328
48. van de Oudeweetering, K., & Agirdag, O. (2018). MOOCs as accelerators of social mobility? A systematic review. Educational Technology and Society, 21(1), 1–11. Retrieved from https://www.jstor.org/stable/26273863
49. Palin, A. (2014, March 9). MOOCs: Young students from developing countries are still in the minority. Financial Times [Blog post]. Retrieved from http://www.ft.com/cms/s/2/8a81f66e-9979-11e3-b3a2-00144feab7de. html#axzz3B6v5aDnA
50. Richter, T., & McPherson, M. (2012). Open educational resources: Education for the world? Distance Education, 33(2), 201–219. doi:10.1080/01587919.2012.692068
51. Salmon, G., Pechenkina, E., Chase, A. M., & Ross, B. (2017). Designing massive open online courses to take account of participant motivations and expectations. British Journal of Educational Technology, 48(6), 1284–1294.
52. Shah, D. (2018, December 11). By the numbers: MOOCs in 2018. Class Central [Blog post]. Retrieved from https://www.class-central.com/report/mooc-stats-2018/
53. Sukhbaatar, O., Choimaa, L., & Usagawa, T. (2018). Students’ perception and experience of massive open online courses in Mongolia. Creative Education, 9(12), 1818. doi:10.4236/ce.2018.912132
54. Watson, S. L., Loizzo, J., Watson, W. R., Mueller, C., Lim, J., & Ertmer, P. A. (2016). Instructional design, facilitation, and perceived learning outcomes: An exploratory case study of a human trafficking MOOC for attitudinal change. Educational Technology Research and Development, 64(6), 1273–1300. doi:10.1007/s11423-016-9457-2
55. Weybrecht, (2018, September 3). An overview of MOOCs offered by United Nations Agencies. Primetime [Blog post]. Retrieved from https://primetime.unprme.org/2018/09/03/an-overview-of-moocs-offered-by-united-nations-agencies-part-1-of-3/
56. Wildavsky, B. (2015). MOOCs in the developing world: Hope or hype? International Higher Education, 80, 23–25. doi:10.6017/ihe.2015.80.6154
57. Yuan, L., & Powell, S. (2013). MOOCs and open education: Implications for higher education. London: JISC Centre for Educational Technology & Interoperability Standards. Retrieved from https://publications.cetis.org.uk/wp-content/uploads/2013/03/MOOCs-and-Open-Education.pdf
58. Zhenghao, C, Alcorn, B., Christensen, G., Eriksson, N., Koller, D., & Emanuel, E. (2015, September 22). Who’s Benefiting from MOOCs, and Why. Harvard Business Review [Blog post]. Retrieved from https://hbr.org/2015/09/whos-benefiting-from-moocs-and-why