Designing online learning assessment through alternative approaches: facing the concerns
Dr. Joan Mateo [jmateo@ub.edu]
Chair professor
Universitat de Barcelona
Facultat de Pedagogia
Campus Vall d'Hebron Edifici de Migdia, 4a planta
Passeig de la Vall d'Hebron, 171
08035 Barcelona, Spain
Prof. Albert Sangrà [asangra@uoc.edu]
Full professor
Universitat Oberta de Catalunya
Estudis de Psicologia i Ciències de l'Educació
Rbla. del Poblenou, 156
08018 Barcelona, Spain
Abstract
Catalan
Aquest article presenta alguns dels problemes que l'avaluació dels aprenentatges es troba en l'àmbit de l'aprenentatge en línia, i analitza les limitacions dels models clàssics d'avaluació que han d'ésser superades per un nou paradigma per al disseny de l'avaluació en aquest nou context. A més, es presenten algunes consideracions i recomanacions per al disseny d'enfocaments alternatius, entre els quals es destaca l'e-portfolio com un dels més viables.
Spanish
Este artículo presenta algunos de los problemas que la evaluación de los aprendizajes encuentra en el ámbito del aprendizaje en línea, y analiza las limitaciones de los modelos clásicos de evaluación que deben superarse por medio de un nuevo paradigma para el diseño de la evaluación en este nuevo contexto. Además, se presentan algunas consideraciones y recomendaciones para el diseño de enfoques alternativos, entre los cuales destaca el e-portfolio como uno de los más viables.
Italian
Questo articolo presenta alcuni dei problemi che la valutazione dell'apprendimento incontra nell'ambito dell'e-learning, e analizza i limiti dei modeli classici di valutazione. Questi devono essere superati attraverso di un nuovo paradigma per la progettazione della valutazione in questo nuovo contesto. Inoltre, propone alcune considerazioni e raccomandazioni per la progettazione degli approcci alternativi, tra i quali l'e-portfolio emerge come uno dei più realizzabile.
English
This article tackles the concerns assessment is facing in online learning and considers the limitations of the classic assessment models should be overcome by a new paradigm for assessment design in online learning. Considerations and recommendations on designing alternatives approaches are given, pointing out the eportolio as one of the most feasible ones.
Keywords
Assessment, online learning, distance education, alternative approaches to assessment
Assessment, distance and online education
Every evaluative model or practice responds to the conceptions that we have about the teaching and learning process which become visible in strategic decisions we make concerning the design of the educational model and the related assessment model.
We have to distinguish between the evaluative strategies and instrumentations we apply in order to gather information about one relevant moment within an activity on one hand and, on the other hand, the evaluative model that has the main function of establishing the guidelines for the integration of all gathered information in order to respond to the different objectives of the evaluative action.
Nowadays, the multidimensional and complex character of the domains to be assessed is clearly recognized. And so is the need to assess them in combination with the application contexts.
Generally, big strategic decisions in distance education are made taking into consideration two of its main aims: to support adult student learning needs and to optimize the use of learning resources.
However, and as a part of the request based on the mentioned conceptions of assessment, learning has been traditionally measured quantitatively, basically through written examinations, objectives and contents. Brown, Bull & Pendlebury (1997) show the differences among the mentioned approaches in the table below:
Table 1. Shifting the paradigm in assessment
From |
Towards |
Written examinations |
Coursework |
Tutor-led assessment |
Student-led assessment |
Implicit criteria |
Explicit criteria |
Competition |
Collaboration |
Product assessment |
Process assessment |
Objectives |
Outcomes |
Content |
Competences |
(Source: Brown, Bull & Pendlebury, 1997)
Under the basis of the mentioned emerging trends in distance education, we would assume these trends are making the scenario change when using technology in education (Badia, 2002).
Therefore, theorists tried to find the most recommendable assessment approaches to open and distance education (Morgan & O'Reilly, 1999). Some of these changes are referred to the nature of assessment, i.e. to assess to promote learning vs. to assess to certify, to assess competencies vs. to assess discipline content. If assessment is based on the academic disciplines, it is centred on the understanding of general structures of knowledge, while if it is based on competencies, it will be centred on the search of scenarios similar to the real working life, and will choose techniques of project assessment, case studies or field observation, closer to the students' reality.
On the other hand, the use of ICT and, particularly, the Internet, let us introduce a very important improvement in learning and, by extension, in assessment: the possibility of relationship among the students. Learning virtual environments widen the options to develop a real learning community in cyberspace, in which formative assessment should be the most important.
Table 2. Shifting the paradigm in distance education assessment
From |
Towards |
Tool to certify |
Tool to promote learning |
Academic disciplines |
Professional competencies |
Uniformity of techniques |
Diversity of techniques |
Final assessment |
Continuous assessment |
Issues and concerns about assessing in online education
Several issues are still arising around the appropriateness of organizational systems and models of distance education and e-learning institutions and experiences: faculty organization, electronic platforms feasibility, multimedia teaching materials … are elements of online distance education which are not only under permanent debate, but being deeply discussed from some hypothesis which consider face-to-face educational practices are more efficient than distance and online ones.
Within the framework of learning assessment, these trends are stronger. Even if some studies have been able to doubt of the apriorism of face-to-face methods are better than online ones (Russell, 1999), there is the need of developing solutions to increase not only the quality of online learning, but also its social perception.
Quality of learning should be measured not only in terms of student learning achievements or success, but also related to the effectiveness of the teaching action and to the satisfaction towards the assessment process. For this reason, one of the most important concerns of distance education institutions is to develop a formative assessment system. Assessment must help to identify and apply improvements in the student learning process, through a permanent feedback.
Likewise, the eternal debate in relation with the recognition of online distance education at a social level, make us not only to be careful and accurate in developing all the learning activities, but also in the rationale of the actions we do, defending with rigour and with the confidence that can respond to those parameters which have been broadly accepted.
At the current moment, assessing competences instead of contents is one of the major challenges facing the European Higher Area (EHEA). Obviously, to develop a good assessment model of competences means to implement an appropriate course design.
Another related issue is the coherence with the model. If you are confident with an online learning model to teach and train people because it is a more flexible way to face the current needs of lifelong learners, it is needed to be coherent by developing any online learning assessment system, capable to maintain this high level of flexibility.
Finally, another of the bigger concerns in terms of assessment in online education is the issue about the identity of the student. How can we make sure of the student providing assignments, essays, exercises … is the student she or he says to be?
The evolution of the concept of assessment
Limitations to the classic model of educational assessment
The classical approach to educational assessment came to the fore at the beginning of the 20th century, and adopted the main principles of psychometric research. In spite of the evolution in educational assessment that has taken place over the past century, there is evidence to show that the modern approaches are still lacking or insufficient, therefore establishing a need to change the paradigm.
The open crisis in respect to traditional assessment procedures is already very profound and professionals understand that they no longer respond to the current needs and expectations and for this we are faced with having to do everything necessary to have a global re–conceptualisation of assessment and its most significant constituent elements.
Over the past few decades this crisis has also been appearing in the context of distance education that has been subject to an important development (Barberà & Badia, 2004; Marcelo & Perera, 2004; García Areito, 2004) due to the constant progress of information technology and the growing need for knowledge access and lifelong learning.
What is evident is the substantial growth that has been seen in the whole world, by the number of institutions offering complete studies or courses via distance learning (Sangrá, 2004; Allen & Seaman, 2007), the increase in the diversity of those and the ever growing interest of the students to continue their studies or training in this way, (Silva, 2007).
However, the impact in the field of distance education is still somewhat obscure, and we need to spend more time on understanding its perspective and collecting data necessary to quantify and interpret its place in society and education.
After all, it is in the field of distance education where, as result of the singularity of its instructional approach, we can find the highest level of contradiction between the demands of the system and the answers given by the classical approaches to assessment.
In our article, we discuss three contexts of reflection which we consider fundamental to begin to orientate the educative answers to the doubts and insufficiencies highlighted up till now.
- Firstly, we analyse the natural limitations inherent to the classic model of assessment, still widely used today in the majority of educational institutions.
- Secondly we will propose the fundamental changes that will need to be introduced to overcome the aforementioned insufficiencies.
- Finally, we will discuss how these new approaches can be managed from the side of distance education.
The classic pretence of measuring has historically consisted in treating to quantify the attributes which are present in objects or individuals. This means that we attempt to standardize and objectify our judgement in respect to these attributes and therefore facilitate their study and knowledge.
With the measuring techniques there is an attempt to compare individuals in relation to thee properties which have been previously analysed. The measurement started to be applied in Humanities from the 19th century when it tried to introduce this type of knowledge in the same metaphor in which the so called scientific disciplines had already been placed. Measurement is the prerequisite in order to apply the scientific method to whichever area specific to knowledge that pretended to gain scientific status.
Considering all this, the analogy applied to Humanities in understanding that the measurable characteristics of this 'knowledge' are naturally similar to scientific knowledge is today in full crisis. There is evidence that there are a large number of limitations of which some are conceptual, others instrumental and finally methodological ones as well.
Conceptual, because of the dynamism of measurement in education (Mateo, 2006), universality and unidimensionality (Berklac, H. et al., 1992; Goldstein, 1992; 1993). Instrumental, because applying the strategies and instruments of the traditional approach, we are unable to determine in scientific terms to what extent and how our students are learning. And methodological, because several controversial arguments have been discussed about validity and reliability (Messick, 1996; Mateo, 2006; Gipps, 2000).
Major trends for alternative assessment approaches from distance education
Currently, higher education is being transformed from the intensive introduction of ICT (Sangrà & González-Sanmamed, 2004). In the same way, the emergence of online distance education and e-learning has forced to rethink teaching and learning models in conventional universities, too.
This transformation is particularly perceived when talking about assessment in online environments (Morgan & O'Reilly, 1999), and needs a new set of procedures and actions to be adapted to the new teaching and learning context emerging from virtuality. This implies to reflect on the assessment experiences we can offer to the students.
There is a clear trend in engaging online distance education students in significant assessment experiences. These experiences should be rooted in openness and flexibility, as according with the kind of learning activities. This also means to provide an online environment wit alternative ways of assessment, to which the students should access by their choice and in a flexible way, taking into consideration any personal need or process (Morgan & O'Reilly, 1999).
On the other hand, Rossman & Rossman (1990) point out on the necessary relationship between theory and practice: the pertinence of the assessment activities, real life-based, and useful for the students as learning activities too.
A new paradigm for assessment design in distance and online learning
Considerations on the course design
Our starting point is the synthesis of the instructional principles of Savery, J.R. & Duffy, T.M. (2006) alongside with our own contributions (Mateo, 2007; Guàrdia, Sangrà & Maina, 2007) in order to identify the way the online model should incorporate in its instructional design the basic defining models derived from the new approach. Also the Sluijmans et al. (2006) approach on competence-based performance assessment has been taken into account.
- Online learning is more efficient and effective when its curricular structure is fundamentally based on the execution of activities.
To renounce to structure instruction based on topics is, in our opinion, fundamental in the field of distance education in general and particularly in online education. We are of the opinion that the most plausible alternative is the model of "learning based on activities".
- The activities have to be authentic.
Authentic experiences are very important for the individual to build mental structures that work in significant situations (Herrington, Oliver & Reeves, 2003). Even so, our concept of autheticism refers to the type of cognitive challenge proposed by the activity more than to the genuine character of the context.
- All activities that are part of the educational program are related to activities with greater range.
The student has to be aware of the fact that the representations of reality are very complex and that all the activities are part of an enormously wide macro-universe. Inserting knowledge in a wider context establishing a "learning situation" (Sangrà & Guàrdia, 2006) in line with the already mentioned "application situation" helps to provide our closest learning situations with real significance.
- The designed activities have to be a real challenge for the student's development of thinking.
Everything related to knowledge, abilities and skills that have to be applied, as well as the contexts in which that application takes place, has to be relevant and significant to the student if we really want to motivate him.
- The instructive process has to be designed in a way that it offers possibilities that the student takes over the development of the activity execution.
It is fundamental that the student takes over the resolution of the problematic elements embedded in the activity. At this point we would like to underline the role of the teacher in the sense that he can offer support to the student but can never provide solutions to the problem.
- The activities have to imply some type of social negotiation and intervention
Knowledge is a socially negotiated product. The importance of the social contexts in the learning process is a clear contribution of constructivism based on Vigotsky's (1978) interpretation.
The use of collaborative learning groups or the most recent and innovative experiences in communities of online learning (Carrol, 2001) provide very clear elements in regarding the positive character of the issue.
- It is necessary to create learning situations facilitating a group analysis regarding the acquisition of knowledge and of the processes that support this acquisition.
One of the most important objectives of whatever instructional model is the development of skills that allow the self-regulation of the students. To do so, the teachers should foster working models that activate the simultaneous group reflection about "how" and "what" students are learning. (Clift, Houston & Pugach, 1990).
- It is necessary to gather assessment information of every activity and introduce activities designed based on the assessment logic. Assessment has to allow the dynamization and guarantee the students' individual and collective learning processes.
All activities have to be designed so that they can provide information that allows analysing them from a position of evaluative reflection. This consists in highlighting the key elements about how the own learning is taking place.
Considerations on alternatives assessment approaches
As we have mentioned before at the beginning of this article, in this new framework the predominant strategy is the one we introduced as alternative or genuine assessment based on executions. This new approach can be defined – in line with the Office of Technology Assessment (OTA) (cited in Elliott, 1995) - as a set of methods that require the students to generate an answer or product the demonstrates their level of control over a knowledge, ability or skill.
It is possibly the context of distance duration where this type of assessment has been accepted in a relatively easy way. The experiences described in the related literature (Barry, 2003) or the deep reflections offered by experts in the field of DL who highlight the relation between this type of assessment and the technical medium, (Rockwell et al., 2000; Lamon et al., 2003) can prove this assumption.
Even so, when we analyse the most widely used online teaching platforms it is surprising that the majority of them are designed to implement basically transmissive pedagogical models related to a cognitive learning paradigm (Wise & Quelay, 2006).
At the same time the proposals of assessment services given by most of them are basically limited to self-assessment or objective online tests (Zúñiga, 2001; Silva, 2007). There are very few exceptions among the commercial platforms, as for example AulaNet, that shyly introduce the possibility of using other types of reactives (projects) that are more in line with the previously presented principles.
It will possibly be in the new open source platforms (Atutor, Moodle, .LRN, Claroline, etc.) where we can hope for some change. In reference to them we are going to mention some of the possibilities opened by Moodle. This platform has been recently adopted by the University of Barcelona where one of the authors of this article currently works.
Moodle appears to be one of the best designed platforms to support instructive processes that are closer to the new paradigm (Dougiamas & Taylor, 2003). Its tools allow analysing not only how to manage the information and the communication but also how learning is taking place.
The portfolio approach
Moodle version 1.6 includes Hot Potatoes for the construction of assessments. It also offers the creation of blogs that can be individual or group/course blogs. What we consider most important is the fact that in the future versions there will be an e-portfolio service that allows students to construct their own electronic portfolio, perfectly integrated in the global context of the platform which adds the possibility of managing the contents of this portfolio in a general way.
From our point of view, the portfolio possibly represents the most important tool of integrated management we have in educational contexts in general and particularly in distance education.
Initially the portfolio consisted (Martin, 1997) in compiling student works that have been gathered throughout his learning process and that offer evidence regarding his knowledge, skills and competences, even regarding him being prepared to act in determined ways.
One key advantage of a portfolio is that it allows the student to present his creations to the teacher so that this one gets a better understanding of the student's work in a global perspective, not fragmented and disconnected of other aspects of his personality.
Another big advantage is that the fact of having to choose which products are included in the portfolio, the student is obliged to decide consciously on the convenience of one or the other piece and its capacity of reflecting the evolution of his most significant learning processes. Inevitably this will lead him to reflect on his level of knowledge and understanding regarding the intentionality of the educational intervention on one hand and his level of achievement on the other hand.
It also provides students and teachers with key elements to be discussed in a concrete and intelligent way. Debating the contents of the portfolio, all parties have a clear awareness of the student's level of development as a person and as a participant in an educational program.
The last five years have shown a considerable growth of interest expressed by designers of electronic tools within the digital environments (IMS, 2005). This explains the rising numbers of specific proposals and the immense amount of research being undertaken in the field of e-portfolios (Ali Safari, 2006).
Even so, our position is that the creation of tools should be carried out within the specific context of a platform like in the case of Moodle, so that integration and combined management of all the other elements of the system can be guaranteed.
From the digital point of view, technicians tend to understand the e-portfolio as a digital container that is able to store texts, images, sound and even software instruments in order to manage a big and diverse amount of information (Abrami & Barret, 2005). However, from our point of view it is much more than that. It forms a dynamic and comprehensive meeting space for the processes of instructional, evaluative and personal development of the student (Barberà & Ahumada, 2007).
More and more often it is conceived as a powerful tool able to extend its comprehensive management to the three big fields of student performance: learning, assessment and personal development (IMS, 2005).
As we have defended throughout our article, the new educational concept unites the processes of learning and assessment. Both belong to models of intervention that work under the same logic and share the same global objective: to educate the subject in a comprehensive way.
However, education is understood as an activity that lasts the whole life. Students have to acquire continual awareness of their achievements and their needs that emerge due to the continuous interaction with reality. This reflection is what allows them to plan their development in an adequate way and build their personal biography.
The acquisition of the necessary competences to face these challenges can not be left at random. It is possibly the most substantial contribution made by modern education and it requires, as we have seen, substantial changes in the processes and agents that operate in educational contexts. But they are also in need of new forms of management.
It is at this point where we place the importance of the evaluative change. The new assessment fundamentally provides a model of managing the learning processes and the education of the subjects on a much more rational basis than the traditional assessment models. But it needs new tools to promote itself. The new instrumentation based on alternative testing of genuine character guarantees the quality of the evaluative information. The portfolio provides an unlimited capacity of managing all this information, especially in the case of an electronic tool.
It is clear that in the context of platforms that consider the new instructional approaches, systems of genuine assessment can be developed and incorporated. At the same time, a well designed and powerful portfolio can envisage its comprehensive management.
In this virtual management environment it is also possible to include elements based on a consensus reached between students and teachers and that gather relevant aspects of the personal development of the subject, of his learning processes seen in a continuum that goes beyond the time of compulsory education, and of all the elements that allow a completely comprehensive education through distance teaching and learning.
Conclusions
Alternatives approaches to assessment in online distance education should be appraised as a means of changing the educational paradigm towards a more constructivist one. The portfolio approach seems to be the one with more potential because allows facing some of the most important issues and concerns about assessment in online environments
Alternative assessment approaches in conventional face-to-face teaching and learning are much more difficult to implement. But online distance education can play a major role as a facilitator: it is placed in a privileged position to carry out a pioneer function for change. It has the whole set of characteristics that make it indispensable in the case it wants to relaunch itself and overcome the function of a complementary model in order to occupy a central place in the socio-educational system. It possesses all the technical conditions to make this endeavour perfectly feasible.
Bibliography
[1] Abrami, P.C..& Barret, H. (2005) Directions for research and development on electronic portfolios. Canadian Journal of Learning and Technology 31 (3)
[2] Ali Safari & Kaufman, C. (Eds.) (2006) Hanbook of Research on e-Portfolios. Indianapolis: Indiana University- Purdue University
[3] Allen, E. & Seaman, J. (2007) Online Nation. Five Years of Growth in Online Learning. New York: The Sloan Consortium.
[4] Badía, A. (2002). L'avaluació virtual dels aprenentatges dels estudiants. Alguns conceptes clau Barcelona: UOC Universitat Oberta de Catalunya.
[5] Barberà, E. & Ahumada, M. (2007) "Assessment of Research Competences Using e-Portfolio". In In Bernath, U. & Sangrà, A. (Eds.) Research on Competence Development in Online Distance Education and E-Learning. Oldenburg: BIS-Verlag der Carl Von Ossietzky Universität Oldenburg.
[6] Barberá, E. & Badía, A. (2004) Educar con aulas virtuales: Orientaciones para la innovaciones el proceso de enseñanza y aprendizaje, Madrid: A. Machado.
[7] Barry, W. (2003) Distance Education at a glance, Paper avalaible in http://www.uidaho.edu/eo/distglan.html
[8] Berlack; H., Newmann, F., Adams, E. Archibald, D., Burgess, T., Raven, J. & Romper, T. (1992) Towards a New Science of Educational Testing and Assessment, New York: State University Press.
[9] Brown, G.; Bull, J.; Pendlebury, M. (1997) Assessing Student Learning in Higher Education. New York: Routledge.
[10] Carrol, T. (2001) Do today's evaluation meet needs of tomorrow's networked learning communities?, paper published in Vision Vision Quest: Evaluation and Assessment. Los Angeles, Ca:
[11] Clift, R., Houston, W., & Pugach, M.(Eds.), (1990). Encouraging Reflective Practice in Education. New York: Teacher's College Press.
[12] Dougiamas, M & Taylor, P. (2003) Moodle: Using Learning Communities to Create and Open Source Course Management System, World conference on educational multimedia, hypermedia and telecommunications, 2003 (1), 171-178.
[13] Elliott, S.N. (1995). Creating Meaningful Performance Assessments. ERIC Digest E531.
[14] García Areito, L. (2004) Características de la producción de materiales para la formación a distancia. En Salinas, J., Aguaded, J. & Cabero, J. (Coords.), Tecnologías para la educación: Diseño, producción y evaluación de medios para la formación docente (pp. 249-268), Madrid: Alianza.
[15] Gipps, C. (2000) Beyond Testing, London: The Falmer Press.
[16] Guàrdia, L.; Sangrà, A. & Maina, M. (2007) "Case-based learning in VTLE: An Effective Strategy for Improving Learning Design". In Bernath, U. & Sangrà, A. (Eds.) Research on Competence Development in Online Distance Education and E-Learning. Oldenburg: BIS-Verlag der Carl Von Ossietzky Universität Oldenburg.
[17] Herrington, J., Oliver, R., & Reeves, T. C. (2003). Patterns of engagement in authentic online learning environments. Australian Journal of Educational Technology, 19 (1), 59–71.
[18] Herrington, J., Reeves, T. C., Oliver, R., & Woo, Y. (2004). Designing authentic activities in Web-based courses. Journal of Computing in Higher Education, 16 (1), 3–29.
[19] IMS (2005) ePortfolio Best Practice and Implementation Guide. IMS Global Learning Consortium, Inc. http://www.insglobal.org/ep/epv1p0/imsep_bestv1p0.html
[20] Job, R. (2007) "Evaluation of learning in eLearning environments". In Ghislandi, P. (ed.) Towards eUniversity. Trento: Editrice Università degli Studi di Trento.
[21] Lamon, M., Reeve, R. & Scardamalia, M. (2001) Mapping Learning and the Growth of Knowledge Building Community in Scardamalia, M. New Directions in Knowledge Building. Annual meeting of the American Educational Research Association. Seattle, WA.
[22] Marcelo, C. & Perera, H. (2004) El análisis de la interacción didáctica en los nuevos ambientes de aprendizaje virtual. Bordón, 5 (3 y 4), 533-558.
[23] Martin, D.J. (1997). Elementary science methods, a constructivist approach. Albany (NY): Delmar Publishers
[24] Mateo, J. (2006) Claves para el diseño de un nuevo marco conceptual para la medición y evaluación educativa, Revista de Investigación Educativa, 24 (1), 165-186.
[25] Messick, S. (1996) Validity of Performance Assessment. In G. Philips, Technical Issues in Long-Scale Assessment. Washington, D.C.: National Center for Educational Statistics.
[26] Morgan, C. & O'Reilly (1999). Assessing open and distance learners. Sterling, Va: Stylus Publishing.
[27] Reeves, T.C. (2000) "Alternative Assessment Approaches for Online Leraning Environments in Higher Education", Journal of Educational Computing Research, 23, 1, 101-111.
[28] Rockwell, K., Furgason, J., & Marx, D. (2000) Research and evaluation needs for distance education: A delphi study. Nebraska Lincoln, NE: University of Nebraska.
[29] Rossman, M. E., & Rossman, M. H. (1990). The Rossman Adult Learning Inventory: Creating awareness of adult development. In M. H. Rossman & M. E. Rossman (Eds.), Applying adult development strategies (New Directions for Adult and Continuing Education, number 45). San Francisco: Jossey-Bass.
[30] Russell, T. (1999) The No Significant Difference Phenomenon. Raleigh: North Carolina State University, Office of Instructional Telecommunications.
[31] Sangrà, A.; Guàrdia, L. & González-Sanmamed, M. (2007) "Planning for Quality Improvement: Educational Design as a Key Issue". In Bullen, M. & Janes, D.P. Making the Transition to E-learning: Strategies and Issues. Hershey, PA: Idea Group, Inc.
[32] Sangrà, A. & Guàrdia, L. (2006) "El rediseño de programas por competencias: una estrategia sistémica". En Gisbert, M. et al. La educación en entornos virtuales: calidad y efectividad en el e-learning. Tarragona: EDUTEC 2006.
[33] Sangrà, A. & González-Sanmamed, M. (2004) La transformación de las universidades a través de las TIC: discursos y prácticas. Barcelona: Ediuoc.
[34] Sangrà, A. (coord.) (2004) La universidad española y las Tecnologías de la Información y la Comunicación. Oferta formativa de 1r. y 2° ciclo impartida por medios electrónicos. Edición digital.
[35] Savery, J.R. & Duffy, T.M. (2006) Problem Based Learning: An instructional model and its constructivists framework. In B. Wilson (Ed.) Constructivist Learning Environments: Case Studies in Instructional Design. Englewood Clifs, NJ : Educational Technology Publications
[36] Silva, J. (2007) Las interacciones en un entorno virtual de aprendizaje para la formación continua de docentes de enseñanza básica. Barcelona: Universidad de Barcelona, tesis inédita.
[37] Sluijsmans, D.M.A.; Prins, F.J. & Martens, R.L. (2006) "The design of competency-based performance assessment in e-learning", Learning Environments Reasearch, 9, 45-66.
[38] Vygostky, L.S. (1978), Mind in Society: The development of higher psychological processes. Cambridge MA: Harvard University Press.
[39] Wise, L. & Quelay, J. (2006) Report for the Melbourne-Monash, Collaboration in Educational Technologies
[40] Zhi-Feng Liu, E. (2007) "Developing a personal and group-based learning portfolio system", British Journal of Educational Technology, 38, 6, 1117-1121.
[41] Zúñiga,G. (2001) Tecnologías Colaborativas para la Realización de Eneseñanza Basada en Web, Interactive Educational Multimedia, 2, 1-18. http://www.ub.es/multimedia/iem.