Open and Distance Learning: Does IT (Still) Matter?
Vitor Cardoso [vcardoso@univ-ab.pt]
Department of Exact Sciences and Technology
Universidade Aberta, Rua da Escola Politecnica, 147
1269-001 Lisboa, Portugal [http://www.univ-ab.pt]
Jose Bidarra [bidarra@univ-ab.pt]
Department of Exact Sciences and Technology
Universidade Aberta, Rua da Escola Politecnica, 147
1269-001 Lisboa, Portugal [http://www.univ-ab.pt]
English Abstract
The evident standardization of Information Technology (IT) in education, by way of the widespread adoption of robust and dependable Learning Management Systems (LMS), has made distance learning courses possible and easy to mount, even in institutions or universities that had no previous experience in online education. However, based on evidence gathered through the ODL-NET Experience project, we were able to identify major usability problems, quite a number of user difficulties and many LMS limitations. The results of the study show that technology is the solution, on one hand, and that technology is [still] the problem, so the saying technology matters is up to date, even if it represents only one component of a complex blend, involving others like educational organizations, learning content, pedagogical strategies, etc.
Portuguese Abstract
A estabilização evidente das Tecnologias de Informação (TI) na educação, por via da adopção generalizada de Sistemas de Gestão da Aprendizagem (LMS) robustos e seguros, tornou possível e fácil implementar cursos de aprendizagem a distância, até mesmo em instituições ou universidades que nunca tiveram experiência prévia de ensino online. Porém, baseados em dados recolhidos através do projecto ODL-NET Experience, pudemos identificar problemas de usabilidade relevantes, várias dificuldades ao nível do utilizador e muitas limitações de LMS. Os resultados do estudo mostram que a tecnologia é a solução, por um lado, mas que a tecnologia é (ainda) o problema, pelo que a tecnologia continua a ser um tema relevante, mesmo se representa só uma das componentes da mistura complexa que abarca as organizações educacionais, os conteúdos de aprendizagem, as estratégias pedagógicas, etc.
Keywords
Open and Distance Learning (ODL), e-learning, Information Technology (IT), Learning Management System (LMS).
Introduction
The importance of Information Technology (IT) in modern life is normally taken as a fact by common sense reasoning, yet in academic literature the debate on the importance of technology is somewhat aged, either aiming at the society in general or at more restricted fields like business, science or education.
In a 1996 study of 468 important international corporations, Paul Strassmann showed how IT Investment and Return are not statistically related (Strassmann, 1996), this was called the The Productivity Paradox. In education, according to Ringstaff and Kelley (2002), measuring the impact of technology use on student success is filled with difficulties thus it is not surprising that the impact of technology on education continues to be debated by educators and researchers alike.
Following this line of thinking, it's not rare to see scholars and educators waning publicly about the importance of IT, in favour of other aspects (pedagogy, content, processes, etc.), either referring to campus-based learning or Open and Distance Learning (ODL). Yet, in spite of these somehow misleading words, most speakers agree that although not necessarily the most important factor, technology is still a very important component of the e-learning mix.
In a 2003 article "IT Doesn't Matter" (Carr, 2003), although not specifically addressing education, Nicholas G. Carr brought a new and interesting argument in favour of the "popular" stand against IT dominance. Carr examined the evolution of information technology in business and tried to show that it follows a pattern strikingly similar to that of earlier technologies, like railroads and electric power. In the beginning, as they are being built into the infrastructure of commerce, these ‘infrastructural technologies', open opportunities for companies to gain strong competitive advantages. But as their availability increases and their cost decreases they become commodity inputs. From a strategic standpoint, they become invisible; they no longer matter. In other words, Carr says that technologies are (or could be) fundamental aspects in the first stages of innovation when only a few have access to them (competitive advantage).
Carr's argument is innovative and powerful for its consequences: if we still are in the first stages of IT innovation "IT still does mater" if not, technology loses its central important role (as a "competitive advantage") and becomes invisible. If we dare to transpose this idea to Education, we can't avoid relating it with an effect often attributed to IT use: the increase in effectiveness, motivation and engagement for the learner. The standardization of IT in education, by way of the widespread use of "standard" IT learning systems, making them "a common commodity for all institutions and courses", could compromise the success of many ODL systems.
According to Carr, we would have already attained the late ever-present stage of IT standardization in ODL. But we can hardly agree, since technology is still evolving fast around us. In ODL we welcome new technologies (RSS, blogs, webcasts, mobile communications, synchronous, etc.) supporting a brand new generation of ODL tools. So, using Carr's argument we could easily say, although against him, that technology does still matter in the realm of ODL.
In modern ODL the importance of having adequate, innovative and robust technology is unquestionable, as we realized after concluding the ODL-NET Experience project. We were able to identify major usability problems, quite a number of user difficulties and many LMS limitations, when dealing with communication systems connecting the students, the educators and the educational organization (Figure. 1).
Figure 1. IT, as the communication
system that connects Students, Educators and the Organization.
The ODL-NET Experience
The main objective of the Socrates/Minerva Project known as ODL-NET Experience (Open and Distance Learning Network for Exchange of Experiences) was to create and consolidate a network for exchanging tried and tested experiences of using Web-based distance learning, particularly in higher education institutions. Unsuccessful experiences were also analysed to discover the reasons why they failed. The decisive aim of the project was to increase the knowledge and know-how of the organisations and people who currently participate in Internet-based distance education.
The specific objectives were:
- To report, analyse and comment on the diversity of, and innovation in, online ODL throughout Europe by a substantive set of representative case studies
- To recommend to policy makers and resource holders actions to promote and support best practice in ODL
- To make contact with distance learning practitioners (teachers, producers, technologists, administrators) to encourage collaboration and innovation
- To inform the wider higher education community of the diversity and ingenuity of the educational approaches and enabling technology used throughout Europe
The project tried to identify the variables that determine the success or failure of Internet-based ODL and produce examples of best practice to circulate amongst the European higher education community. Although a wide range of good examples of this kind already exists, in most cases their quality has not been externally evaluated so the results of the project helped in reviewing existing initiatives. This was perhaps the most innovative aspect of the project.
The final results were:
- A network of project partners with expertise and experience in the analysis of ICT supported ODL in Europe
- A substantive resource of case studies that represent European best practice during 2003-2005
- A comprehensive report of the partners' national activity and analysis of the following issues: distance tutoring, content development, learning evaluation accreditation, technology and management
- Recommendations and conclusions for higher education management and policy makers
- Seminars/workshops in each partner country and other dissemination activities.
In the specific area of technology many general questions emerged: how should we integrate learning activities with technology? What are the actual strengths of the different technologies? Much has been said about the potential for interactive learning that is sustained by the "right" technology based on the "most current" tools (Bidarra and Dias, 2003). The ODL-NET project tried to get some comprehensive answers to these questions.
The methodology used in ODL-NET Experience was based on a continuous exchange of information between partners, by gathering information on cases and experiences at a regional and national level, but also through the analysis of areas of interest across all experiences.
The modus operandi was based on seven stages:
- Each partner identified four or five potential cases or experiences in their country
- The partners reviewed and approved the proposals for ODL type coverage and balance
- The partners identified the horizontal areas of interest and formed working groups to define each area and develop questions to guide an interview
- A composite questionnaire was developed
- Interviews were conducted with each case or experience guided by the questionnaire. Interview data was archived.
- Results were analysed to produce both national (vertical) and area of interest (horizontal) reports
- The three ODL NET Report volumes were compiled.
The project partners initially identified potential ODL experiences in their region or country, classifying them under the following characteristics:
- Ownership data
- Introduction / background
- Description and objectives
- Technical Media / Platform used
- Learning Methodology
- Design of Contents / Materials
- Interactivity
- Results
- Evaluation and Conclusions
In total, 36 cases and experiences from eight different countries (Cyprus, Czech Republic, Finland, Greece, Ireland, Portugal, Spain and United Kingdom) were analysed.
In parallel, working groups of three or four partners were formed in areas of horizontal interest that were of particular relevance to, or were problematic in, the process of Internet-based ODL.
An agreed brief definition and description of each area was evolved through an iterative process of suggestion, inclusion, modification and improvement, each partner contributing to the evolution of three or four interest areas.
Results of the Technology Study
Within the ODL-NET Experience project, the technology study was based on data collected by means of a questionnaire. It focused on 6 main aspects: identification of technologies, access by learners, operational strengths, innovation factor, swiftness of operation and critical aspects.
Identification of technologies
As a background to the following analysis it's important to refer that, for the courses analysed in this report (36 courses), the distribution between online[1] and blended is almost equal (Figure.2).
Figure 2. Nature of the analysed courses
Decision makers and educators running online courses require stable and proven IT solutions, so it is not surprising that in the majority of the cases (33 or ~92%) a Learning Management System (LMS) was used. Most LMSs used (25 cases, ~69%) were well established commercial systems (WebCT, TopClass, Learning Space, Intralearn, etc.) in 6 cases (~17%) universities/institutions had developed their own LMS (Merlin, Odisseia) and in 5 cases (~14%) Open Source LMSs were used (Moodle).
Despite the LMS dominance in 92% of the courses (Figure.3), in the remaining 8% are used other IT systems such as classic Web server technologies or IT systems/software developed specifically for the course.
Figure 3. Percentage of Courses using Learning Management Systems (LMS).
It is also important to acknowledge that in a course where a certain LMS is the "official" tool, other systems/tools can be used as an alternative or a complement in situations where the LMS solutions are considered less adequate. One example is the "poor" chat feature of some platforms that is often substituted by more popular "messengers". One other aspect we should mention is the "change of platform" from one course edition to another (as referred to in some of the Irish courses analysed) motivated by dissatisfaction or cost. High prices of commercial LMS are stimulating the adoption of open source software, so costs do matter too! In some cases this change has led to user dissatisfaction with the limitations of the new platform, so management decisions to change LMS must be carefully planned and take into account pedagogical aspects and users habits.
On the server side, most respondents could not answer clearly about the Operating System used (Unix/Linux/Windows) by the LMS server, since most of LMS area administrators only deal with the LMS via the Web Interface[2] and don't have access to the lower level of the server administration (usually controlled by the IT centre of the institution).
On the client side, Microsoft Windows/Internet Explorer dominates absolutely with a small penetration of Linux/OSS Web Browser as an alternative.
e-learning standards (SCORM, IMS, etc.) are referred to in some responses, but in none of the courses were considered to be an important aspect, although they are present in the LMS and some users are aware of that.
Access by learners
A multimedia PC and a dial-up Internet connection is the minimum necessary equipment the students need to follow the online component of the courses.
Any reasonably modern PC meets the multimedia PC requirement, but we doubt if a dial-up connection is adequate. The fact that most courses (~64%) use exclusively computer mediated asynchronous technology[3] (Figure.4) explains such a low specification. In any case there where some reports of complains about connection troubles when rich multimedia or synchronous communication where used.
Figure 4. Courses using essentially
asynchronous technologies or mixed
On the other hand, more and more students are profiting from the more general availability of high bandwidth throughout Europe, which facilitates the access to courses with rich media content (video/audio) and synchronous communication. This is a current trend in ODL systems development showing interesting innovations (Cardoso, 2005).
In most cases, the student must have some software (MS Word, PowerPoint, media plug-ins, etc.) to be able to use the digital contents, but these programs are widespread and some of them are free or have free viewers. So we can easily conclude that the access for the learner is fairly available and reasonably guaranteed in the universe of courses studied.
Operational strengths
Probably because educators often fear IT failures, the technical robustness of IT systems is pointed out as the most positive aspect and it is reasonable to say that most IT systems and LMS presently in use are trustworthy.
The second operational strength most invoked is the user's habit or previous experience in using a specific LMS platform. This may drive us to the conclusion that, unless special circumstances occur, a platform will remain the same for more than one edition of the course.
The third aspect most referred to as positive is the technical support for IT. A guarantee of IT technical support (helpdesk) is very important in the decision to create, or keep running, an Internet-based ODL course.
Innovation Factor
Since most courses run on an LMS platform, the respondents tend to assume that innovation is an issue concerning the LMS creators and will be implemented "automatically" in the next version of the learning platform. This could be a negative effect of using learning platforms; innovation may be stifled because the LMS can't grow out of its confines.
Fortunately, it is reported that some innovation is going on by way of course content (rich media, 3D, etc.) or by complementary software acquired specifically for innovative communication areas (web conferencing, synchronous systems, virtual classroom, etc.)
Swiftness of operation
Besides some respondents confusion of terms between technical swiftness versus organizational swiftness (which is slower), we can conclude that from the data available about most current LMSs, the creation of new courses is technically quick and simple.
Critical aspects
Figure 5 represents the respondent's view of critical aspects and their relative importance.
Figure 5. Critical aspects reported (1 per course, although a course may report several types of problems)
The three most important are thus: User difficulties, LMS problems and Cost.
User difficulties are by far the most significant. In the 36 courses studied 27 (~75%) reported user problems. These include difficulties experienced by tutors (mostly) and students in dealing with the online system (usually an LMS). There are also LMS limitations, technical or pedagogical, which form the second greatest problem, reported in ~40% of the cases.
The third problem reported is the high cost of some commercial LMS; this situation has already triggered a rapid change towards free and/or open source LMS solutions. This change if unplanned may have negative effects on users' satisfaction, habits and performance. Most of the time usability and pedagogical aspects are not taken into account, as reported by some respondents.
Other critical aspects reported are technical difficulties with the system (interruptions, firewalls, security, etc.) and problems with Internet connections.
An ODL course for students living in an African country (Angola) had to undergo some fundamental changes in crucial working conditions, e. g. disabling of Internet access, which led ultimately to a course failure. This one case (out of a 36 cases study) shows that, although a course failure due to technology issues is not common, it happens, particularly if associated with other critical aspects (here political).
Conclusion
This study on the technology supporting Internet-based ODL reveals that:
- The widespread adoption of Learning Management Systems (LMS), that are robust and dependable (and some even free), has made them possible and easy to mount, even in organisations that had no previous experience of online courses.
- There are major usability problems associated with the implementation and use of LMS systems for ODL, evident by the number of reported user difficulties and platform limitations.
- The gap between the LMS promise and the users' expectations has to be faced seriously and dealt with.
In the end, the results of the study showed that technology is the solution, on one hand, and that technology is [still] the problem, so the saying technology matters is up to date, even if it represents only one component of a complex blend, involving others like educational organizations, learning content, pedagogical strategies, etc.
Endnotes
[1]Rarely courses are 100% online. A very exceptional face-to-face component (support, meetings, evaluation etc.) exists in some courses but this does not put them in the category of blended, because they remain in essence online courses.
[2]The most widely used LMS in these courses (WebCT) may run in several Operating Systems (Unix/Linux/Windows).
[3]It is sometimes very difficult to draw a line. For the aims of this study, when synchronous technologies where considered absent or not important we considered the course to be asynchronous.
References
[1] Bidarra, Joséand Ana Dias (2003). From Cognitive Landscapes to Digital Hyperscapes. In International Review of Research in Open and Distance Learning (Athabasca University - Canada), Volume 4, Number 2, November 2003. Available online article at: http://www.irrodl.org/
[2] Cardoso, Vitor
(2005). New Roles for Synchronous Communication in On-line Education: the
Odisseia model for site structure integration. Proceedings of the First
International Conference of Innovative Views of .NET Technologies, 121-132.
ISEP & Microsoft, Porto, June 2005. Available online article at:
http://w2ks.dei.isep.ipp.pt/labdotnet/files/IVNET/Odisseia_p.pdf
[3] Carr, N.G. (2003). IT Doesn't Matter. Harvard Business Review, May 2003.
[4] Ringstaff, C. and Kelley, L. (2002). The Learning Return On Our Educational Technology Investment. West Ed. Online report at: http://rtecexchange.edgateway.net/learningreturn.pdf
[5] Strassmann, P. (1996). The value of computers. Information and Management. Online report at: http://www.strassmann.com/pubs/cik/cik-value.shtml
Acknowledgements
This research was part of the European Project ODL-NET EXPERIENCE (Open and Distance Learning Network for Exchange of Experiences), Ref. 110345-CP-1-2003-1-ES-MINERVA-MPP. http://www.adeit.uv.es/odlnetexperience