Implementation of Quality Information Systems
for E-Learning Applications
Daniel Sommer
Institute AIFB, University of Karlsruhe, Germany
Key words:
E-Learning, Quality Management, Quality Information Systems
Abstract:
Quality information systems (QIS) are providing all the quality-relevant information during the whole life cycle of a product to all the people involved. With regard to e-learning applications this means that a QIS should provide quality-relevant information to the authors of e-learning materials as well as to the instructors and the learners using them. In this work it is described how such a QIS can be implemented in the context of e-learning applications. In this connection a prototypic system design and strategies for the integration of a QIS into the information system infrastructure of e-learning providers are presented.
1 Introduction
Many concepts of quality management are not only applicable during the production
and usage of material products, but also in the context of immaterial ones -
like for example e-learning applications. Nevertheless quality management concepts
are hardly used in this regard today. As a basis for quality management so-called
quality information systems (QIS) can be used. They are providing all the quality-relevant
information during the whole life cycle of a product to all the people involved.
In the context of e-learning this means that a quality information system should
provide all the quality-relevant information to the authors of e-learning materials,
to the instructors and tutors, and also to the learners.
In this work after a look on the State of the Art some fundamental terms and
ideas of quality information systems will be introduced, and it will be pointed
out how they can be used in the special case of e-learning. In particular it
will be described which information could be quality-relevant and how it should
be prepared for the different users of the system. Then the focus of the work
will be on the implementation of such a quality information system. After a
detailed requirements analysis a system design will be presented. Moreover it
will be shown how an information system like this could be integrated into the
existing information system infrastructure of e-learning providers. Here the
importance of standards for all the types of data involved, like for example
IEEE LOM for the learning objects, will be pointed out.
The results presented in this work have been obtained in the context of the
research project "ViKar - Virtual University Group Karlsruhe" [1].
This project is being financed by the state of Baden-Württemberg as a part
of the research program "Virtual University of Baden-Württemberg"
[2].
2 State of the Art
In the last few years a lot of endeavors have been made in several fields of
e-learning research and development. In the beginning the authoring of e-learning
materials was in the spotlight. As a basis for all further e-learning activities
contents have been developed in a multitude of different technical formats.
Since special programs for the authoring of e-learning materials have been missing,
most of the authors used well-known software like word processing tools or HTML
editors. Since these programs don't support any special requirements for the
authoring of e-learning content at all, it's also hardly possible to support
quality management concepts using these tools.
At the same time, while the first e-learning materials have been created, many
so-called e-learning platforms have been developed to provide e-learning materials
to the learners and for the handling of communication and organization. Comparisons
of such e-learning platforms (like for example in [3]) show that there are wide
differences in their functionalities, but nearly all of them have one in common:
they don't force or support a continuous quality management.
Furthermore till now there is a big gap between the authoring tools used for
the development of e-learning content and the e-learning platforms. Since most
of the platforms don't support the authoring satisfactorily the authors have
to use tools widely independent from the platforms, which obviously complicates
quality management, too. A possible solution for this problem might be the usage
of standards for the modeling of e-learning content such as LOM [4], but they
are not used widely enough so far. In this situation the first step to a continuous
quality management might be the implementation of a quality information system.
3 Quality Information Systems
3.1 Terms and Definitions
For the design and implementation of a quality information system it is necessary
to understand what "quality information" really is. For this purpose
some fundamental terms and definitions, which will be used in the succeeding
sections, will be explained here.
The ISO 9000:2000 standard defines quality as the "degree to which a set
of inherent distinguishing features fulfils needs or expectations that are stated,
generally implied or obligatory" [5]. In accordance with this definition
a quality information comes from the comparison of the actual and the nominal
condition of a quality criterion [6]. Then a quality information system obviously
is an instrument for handling such quality information. In many cases quality
information is not available directly, but it has to be derived from other raw
data. Therefore in the following data is regarded as quality-relevant, if it
can be used to obtain quality information.
All these definitions can be transferred directly to the special case of e-learning.
The most important expectation for an e-learning application and therefore the
most interesting quality criterion should be that a certain subject can be learnt
using the application. But different users of a quality information systems
might have different other expectations: generally the authors of e-learning
material for example want to improve the quality of their product and so they
should be interested in many kinds of information about the usage of their material;
and on the other hand for the management of a company using a certain e-learning
application also cost savings may be of interest. For these reasons all the
data that is quality-relevant for an e-learning application should be collected
completely in a special quality information system.
3.2 Quality-Relevant Data in E-Learning Applications
To obtain a complete compilation of quality-relevant data all the different data types occurring in the whole life cycle of the application have to be considered. A detailed analysis of many different data types can be found in [7]. Figure 1 shows only a simplified overview of the principal constituents every quality information systems for e-learning applications should contain.
Figure 1: Quality-relevant data
The overview is structured according the different phases in the life cycle of the application:
- In the analysis phase only general specifications of the planned e-learning application are generated. For example the learning objectives of the application and its target group should be defined here.
- In the design phase the e-learning application is modeled in detail, in particular the course content and its structure has to be defined. It's very reasonable to specify all this data in a formalized way. Then it is obviously much more adapted for an automatic data processing than the implemented e-learning content itself. In later phases this data can be used to survey, whether the implemented e-learning application does suit its specifications or if it's used in a way according to them.
- In the implementation phase the e-learning application has to be
realized pursuant to the modeling in the preceding phases. The corresponding
specifications should be transformed into the meta information of multimedia
elements, modules and courses. Of course it makes sense to use a standard
for e-learning meta information here. If the learning modules for example
are written using XML only some special tags according to the standard have
to be inserted.
It should be mentioned again that also the learning content itself is quality-relevant of course, but it's not as accessible for an automatic processing as its formalized meta information. Therefore it's not reasonable to store the whole learning content in a quality information system.
- In the phase of integration and testing and in the phase of operation and maintenance essentially the same types of quality-relevant data can occur. Examples for these types of data are user-tracking data, results coming from direct questionings of the users and personal user data including the results of learner assessments. Obviously the learners are involved in the generation of all this data and therefore their privacy has to be taken into account. For this reason all the personal data in a quality information system should be anonymized or at least pseudonymized, but for a later only statistical analysis usually that's no problem. Some techniques respecting at least the German data protection acts have been described for example in [8].
All these types of quality-relevant data might be interesting separately, but they'll develop their significance only if they are associated with each other. If for example many users coming from the same module search for a certain word in the glossary, this might indicate that the learning module is ambiguous at a certain point. If this is not the case, some information about the learners might indicate that some knowledge the learners don't have has been presupposed in that learning module. As another example also questionings of the users will become more significant if they are related to certain parts of the content. If these relations are specified in a formalized way, an automatic analysis will be much simplified. Therefore in any implementation of a quality information system the different types of quality-relevant data and, what is more, their relationships have to be regarded.
4 Implementation Strategies
Generally a quality information system cannot be implemented strictly separated
from the other information systems of an e-learning provider, but it's very
important to consider how close the connection between these information systems
should be. One extreme would be to consider the quality information system only
as a view on the whole information system architecture, the other one would
be to implement the quality information system widely independent from the other
systems and to use only some small interfaces. Of course the decision which
implementation strategy to use depends on the situation of the existing information
systems, but there are some reasons for using only a loose coupling between
these different systems.
This can be seen in the following comparison of two different approaches for
the integration of a quality information system into an e-learning application.
In Figure 2 an integration with a close coupling is shown.
Figure 2: Integration of QIS and e-learning application (I)
In this example the architecture is shown in a UML class diagram. It should be assumed that the classes Application and QIS are the "main classes" of the e-learning application and the quality information system respectively. The other classes should be assumed as interfaces to different types of quality-relevant data the quality information system is based on:
- The Content (in general its meta information only) is used by the class QIS,
- the personal data of the Learner is included in the analysis,
- all the Tracks the learners left while using the e-learning application are available,
- and the results of Polls are also included in the quality information system.
These relationships are shown in the UML diagram by associations between the
class QIS on the one hand and the four classes standing for the different
types of quality-relevant data mentioned above on the other hand. As shown in
the figure also the e-learning Application needs to have associations
to all these classes providing access to the quality-relevant data. Therefore
this approach has some disadvantages: with regard to data protection issues
for example it is critical that the personal data of learners can be accessed
from the quality information system. The class QIS is also related to
classes representing types of data that are not of interest to a quality evaluation
(e. g. parts of the personal data or parts of the content, which are not accessible
for an automatic evaluation). Furthermore the implementation of the e-learning
application and the quality information system in a distributed way should be
taken into account: using this architecture with its very close coupling a distributed
implementation is made difficult.
For all these reasons an architecture with a looser coupling between the e-learning
application and the quality information system, like the one in Figure 3, should
be considered.
Figure 3: Integration of QIS and e-learning application (II)
Using an approach like this the e-learning application and the quality information system are much more separated from each other. Here the class QIS doesn't have access to all the personal data of the learners, since Anonymous represents an interface to some anonymized data only. Other types of quality-relevant data in this model, namely the results of polls and the tracking information, are only related to the anonymized learner profiles and not to the other personal data. In consideration of data protection that's a big advantage in comparison to the other approach. Since the associations between Application on the one hand and Poll and Track on the other hand can be realized using some small interfaces, the quality information system is much more independent from the e-learning application. Nevertheless there are some types of quality-relevant data that are also extremely important for the e-learning application, for example the Meta Information of the learning content. Here it's obviously not reasonable to introduce an artificial separation, since both parts need a close connection to the meta information, even though they need it for different purposes. But this quite close coupling here shouldn't be problematic, since in contrast to the other types of quality-relevant data the standardization of this data type has advanced significantly. If a standard for e-learning content and its meta information is used, like for example LOM [4], the data exchange between the e-learning application and the quality information system described above is quite uncritical. Therefore both systems are sufficiently independent from each other and may also be implemented in a distributed way.
5 Conclusion and Outlook
As described in the preceding sections a quality information system for an
e-learning application should be designed very carefully. There are many different
types of quality data occurring in the context of e-learning. To obtain a quality
information system, which is usable during all phases of an e-learning application's
life cycle, all the relevant data should be stored in one information system.
In the long run there are many advantages, if both systems - the quality information
system and the e-learning application - are working quite independently from
each other. The implementation, the operation, and the maintenance of the systems
will be simplified very much, if they have only some small interfaces realizing
a standardized communication. This strategy can be used, if both systems are
being designed from scratch, but it should also be considered, when it is planned
to implement a quality information system in addition to an existing e-learning
application. The increasing importance of e-learning standards is another factor
supporting this approach.
The implementation of a quality information system includes further interesting
problems of course, e. g. in the context of the analysis of quality data. There
are some existing approaches, which could be transferred from the field of the
evaluation of other web content, but nevertheless also special concepts for
the quality management in the emerging area of e-learning will be developed
to achieve a higher quality of e-learning.
References:
[1] ViKar - Virtual University Group Karlsruhe, retrieved August 7, 2002, from http://www.vikar.de
[2] Virtual University of Baden-Württemberg, retrieved August 7, 2002,
from http://www.virtuelle-hochschule.de
[3] Landon, B.: Online educational delivery applications: a web tool for comparative
analysis, retrieved July 25, 2002, from http://www.c2t2.ca/landonline
[4] IEEE Learning Technology Standards Committee: Standard for Learning Object
Metadata, retrieved August 1, 2002, from http://ltsc.ieee.org/wg12
[5] International Organization for Standardization: ISO 9000:2000. Quality management
systems - Fundamentals and vocabulary. Geneva: ISO, 2000.
[6] Scheer, A. W.; Trumpold, H.: Qualitätsinformationssysteme. Heidelberg:
Springer, 1996.
[7] Sommer, D; Stucky, W.: Quality-Relevant Data in the Context of E-Learning
Applications. Proceedings of the Second Research Workshop of the European Distance
Education Network. E. Wagner; A. Szücs (ed.). Budapest: EDEN, 2002.
[8] Schaar, P.; Möller, F.: Der Hamburgische Datenschutzbeauftragte: Orientierungshilfe
Tele- und Mediendienste, retrieved January 18, 2002, from http://www.hamburg.de/fhh/behoerden/datenschutzbeauftragter/veroeffentlichungen.htm
Author:
Daniel Sommer
Institute AIFB
University of Karlsruhe
76128 Karlsruhe
Germany
sommer@aifb.uni-karlsruhe.de