Learner Generated Content: Quality Criteria in online Collaborative Learning
Maria Pérez-Mateo [mperez-mateo@uoc.edu]
Marcelo F. Maina [mmaina@uoc.edu]
Montse Guitert [mguitert@uoc.edu]
Marc Romero [mromerocar@uoc.edu]
eLearn Center (UOC), Roc Boronat 117, 08018, Barcelona [http://www.uoc.edu/portal/english/elearncenter/index.html]
Abstract
This study focuses on quality in a Learner Generated Content (LGC). The main objective is to identify and describe the criteria supporting the quality of the creation of content by those learners working together in an online environment. Contrasting a literature review and learners’ perception, we propose a quality criteria framework for LGC organized in three clusters: content, format and process. Emphasis on both process and end product highlights the LGC’s twofold intention of being useful as a creative new pedagogical strategy and as a way to share educational resources imbued with the learner’s voice and perception.
Key words: Learner Generated Content; quality criteria; Collaborative learning; wiki; online learning.
List of topics
Introduction
Experts in the field of education claim that fulfilling current educational needs require strong modifications in the traditional roles of educational agents. That is, rethinking the educational space and envisaging new roles for both teachers and students (Lynch, 2002, Richardson and Swan, 2003). For this situation to change, some issues appear to be relevant: active learning, collaboration, creativity and using technology to enhance learning. The common element in all cases is quality, which should be commonplace in all educational experiences.
Learning through Learner Generated Content
In an online learning context, student activity becomes a key element. According to Bonwell & Eisen (1991:2), students’ active learning can be defined as instructional activities involving “students doing things and thinking about what they are doing”. Research on educational technology highlights ICT’s privileged position in the promotion of active learning opportunities (Williams & Chann, 2009) as well as collaboration among students. According to this, the online learner is not a passive agent anymore. This also implies a change in the teachers’ role, becoming a “guide on the side” (Palloff & Pratt, 2001), a mentor or facilitator of the said process.
The introduction of ICT in education has not only changed student and teacher roles, but also the context in which learning takes place. Conole (2008) emphasizes speed and immediacy as the main characteristics of today’s digital environment and at the same time, the ability to access a vast amount of information at the click of a mouse. The author also mentions as part of this new learning landscape the multiplicity of communication channels and social networks. These new features shape the context in which learning takes place nowadays.
Starting from this situation, students’ relationship with knowledge is being significantly challenged: they are becoming active participants in the creation of educational content primarily facilitated by the so-called Web 2.0 (O'Reilly, 2005), which provides vast availability and variety of tools that support the educational process. Web 2.0 is seen as a trend in technology innovation that increases participation and promotes bidirectional communication. Users are not limited to reading or viewing. Instead, they are encouraged to increase their level of involvement by producing and publishing their own content through simple tools (Dans, 2009). There is no doubt about it that in the Web 2.0 context, we are witnessing a change in the role of users who now becoming the true “protagonists” of a process that includes active reading and creative writing (Balagué & Zayas, 2007; Cabero et al., 2009). Applying the description of this new reality to the educational context signifies recognizing the learner “as an active constructor of learning materials (co-creator), personal learning environments and initiator of his or her own learning processes” (QMPP, 2009:32).
The web 2.0 framework and the concepts associated with social software are favoring the emergence of new open virtual learning environments for higher education (Shirky, 2003). The use of open and collaborative tools like blogs, wikis, social bookmarking, podcasts, video channels, etc. is becoming an increasingly feasible alternative to reshape the traditional e-Learning scenario that is currently based on closed, proprietary, institutionalized systems (Grodecka et al., 2008). At the same time, Web 2.0 tools and social software increase collaboration, communication and knowledge production (Rhoades, Friedel, & Morgan, 2009).
As part of the web 2.0 social phenomenon, the concept of what knowledge means and how it is created is significantly changing. In this sense, Franklin & Harmelen (2007) note that Net users take advantage of the web as a platform to generate, re-purpose, and consume shared content. This is what some authors call “wisdom of the crowds”, or in other words, “that user-generated content and mass participation enable new ways of co-constructing ideas” (Conole, 2008). As other authors observe, the debate over both what we teach and how we teach it is being reshaped by the UGC movement (Lee & McLoughlin, 2007).
Given that the term "User" in the notion of UGC refers to a broad set of actors and contexts, this paper adopted the concept "Learner Generated Content" (LGC) since our focus of content generation develops within an educational setting. Learner generated content is a relatively new concept that is still being explored. The notion of LGC entails two main features: the context of content development within a learning situation, and the direct implication of learners in their elaboration. LGC relates to the notion of student “performance” content which is dynamically generated by students within the process of learning (Boettcher, 2006). It may include “completed project/assignment work or deliverables (i.e. end products) as well as evidence of the process of learning, such as successive drafts of solutions, descriptions of mistakes made, or difficulties encountered” (Lee & McLoughlin, 2007).
Creativity and online learning
According to Cebrián (2011), creativity and innovation are becoming key values for the development of the current society. Starting from the conceptualization about creativity (among others, Herrmann, 1996; Csikszentmihalyi, 1997; Ardaiz-Villanueva et al., 2011; Cebrián, 2011), novelty and originality stand out as defining elements. According to Vidal (2009:414), “creativity occurs when a person (or a group) makes a change in a domain, a change that will be transmitted through time”.
Vidal (2009) identifies four essential skills that determine creativity: a) Fluency (or fluidity in terms of Ardaiz-Villanueva et al., 2011), as the ability to generate or produce multiple problems, ideas, alternatives or solutions; b) Flexibility or the ability to process ideas or objects in many different ways given the same stimulus; c) Originality, that is, getting away from the obvious and commonplace or breaking away from routine bound thinking or, in other words, to produce something new or unique (Cubukcu & Dündar, 2007; George, 2007, quoted by Ardaiz-Villanueva et al., 2011); and d) Elaboration, associated to content structuring.
Ardaiz-Villanueva et al. (2011) highlight that creativity is not free in terms of teaching but it can be enhanced by addressing these skills. For this to happen, it is important to take into account two elements.
On the one hand, diverse sets of technological tools can be used to support individual as well as group creativity (Ardaiz-Villanueva et al., 2011). In the Web 2.0 context, creative development possibilities are increased by a number of features: mashingup, embedding, syndication, personalization,…
On the other hand, the pedagogical proposal can promote or stifle creativity. Groth & Peters (1999) suggest that “discovered” learning –as opposed to “taught” learning- could improve creativity and innovation. Open proposals are also relevant in this sense. At the same time, it is important that these proposals will be creative in themselves.
Quality issues in a Learner Generated Content scenario
The active, open and creative nature of online learning does not mitigate the need to reflect on quality issues. Indeed, content creation by learners using Web 2.0 tools is not free in terms of quality. Educational activities oriented towards the generation of content should also consider quality assessment and not just constrained to content and furthermore, being opened to the entire process of collaborative construction of new knowledge.
According to QMPP (2009:34), “in eLearning 2.0, quality cannot be tied to the evaluation of a pre-determined learning environment or learning contents produced by an expert. It is not just reception but active participation which is most important, and thus brings to light the question of how far a learning scenario stimulates the creation of individual personal learning environments, the compilation of individual learning contents and sharing them with others”. This situation, however, raises an important need: to highlight and rethink quality criteria. Ehlers (2009:296) states in this sense that “even though the question of quality had already been controversially discussed when e-learning 1.0 appeared on the market; e-learning 2.0 creates even more insecurity”. Liu & Johnson (2005) argue that the accelerating quantity and complexity of online resources focus attention on their inconsistent quality.
Responding to the fact that the vast amount of socially produced content is available online, an increasing number of researchers are addressing the issue of quality in their elaboration within the context of learning situations. This is the case of the on-going European project Content Creation through Dialogue in Education (CONCEDE). Among its objectives, include the development of a quality framework for UGC/LGC. As part of this initiative, we present a preliminary quality criteria approach for LGC by looking at those related to creativity.
Methodology
This paper outlines and describes criteria for supporting the quality of content creation by learners working together in an online environment. Starting from an interpretative research approach, we adopt a qualitative point of view. Based on these considerations, we have used the case study (Stake, 1998, Yin, 2003) as a way to develop our study.
Scenario
The educational environment, which becomes the general scenario of our research, is the Open University of Catalonia Virtual Campus. The UOC is a fully online university, with a Virtual Campus where all learning activities and communication take place. The average profile of the UOC student is an adult, aged between 25 and 40 who works more than 30 hours a week.
Our study is carried out within the “ICT competences” course of the Social Education undergraduate program. The 6 classrooms of this course were involved in the study. The average number of students in each classroom is about 60. “ICT competences” (6 ECTS) is a cross-curricular course common to all UOC undergraduate programs. Its main objective is to initiate students in the use of ICT for learning purposes and to gradually support the development of a specific competence defined by the UOC: the “Use and application of ICT in academic and professional development”. The course is recommended to be taken in the first semester of the online program, at the first contact that the students experience with this new learning environment.
As pointed out by Ardaiz-Villanueva et al. (2011:701), “several studies have provided evidence for the idea that creative potential is improved in social environments where group work with freedom and autonomy exists or in settings that encourage positive attitudes toward creative behavior (Hunder, Bedell, & Mumford, 2007; Niu & Sternberg, 2003)”. The “ICT competences” pedagogical proposal starts from a creative approach and, at the same time, it takes into account the relevance of student activity as well as collaboration processes and creative skills. Specifically, it is based on project-based learning, PBL (Han & Bhattacharya, 2001; Railsback, 2002). Starting from the activeness and collaboration among students, the PBL methodology influences their creativity and innovation development (Yang and Cheng, 2010).
The students form small groups of 3 to 4 participants and choose a topic of interest related to their discipline (Social Education). This topic is then discussed and collaboratively elaborated throughout the learning process (LGC). As a result of this process, each group develops its own Virtual Project. Groups use a wiki as the main tool in order to develop the content. The wiki was chosen by us as a tool which fosters interaction in content creation as well as collaboration and an architecture for participation (Wheeler et al., 2008) within a shared and openly accessible digital space. It also facilitates some features in order to develop creativity.
The Virtual Project methodology involves 4 phases, each of which proposes a set of interrelated activities. Across the different phases, students build their project within a wiki specific to each group and following the stages shown in Figure 1. Student evaluation is based on continuous assessment, including self, peer and tutor evaluation.
Figure 1 Virtual Project steps (ICT competences course)
According to the PBL approach, through Virtual Project elaboration, students develop the four skills associated to creativeness (fluency, flexibility, originality and elaboration). Indeed, the Virtual Project is open in its proposal: starting from Social Education issues, students choose a topic which is organized and developed in an original and own way for each group. This process is also supported by wiki’ features, which facilitates creativeness in its format by organizing the content, including visual aspects, embedding objects..., i.e. fostering multiliteracy. Although the teacher provides some general guidelines, it is a creative process from the beginning (choosing the topic, organizing the group...) to the end (developing and presenting the content).
Data collection and analysis methods
The procedure for data collection and analysis developed in this paper is composed of the following phases:
- Identification of quality criteria for LGC on the basis of a literature review on the matter. Given that this is still an emerging subject, we have also considered other studies related to multimedia resources. As a result of this process, we composed a list of quality criteria organized into three categories: content, format and process. These criteria are intended for the drawing up of an LGC quality framework.
- Exploration of students’ perception of
quality content generated by themselves in relation to quality criteria in LGC. This perception is based on their own experience. This process was
conducted through an online questionnaire. The questionnaire was voluntary,
anonymous and non-assessable. The questionnaire included three open questions
associated to format, content and process in LGC. These questions are presented
in this paper.
The questionnaire was sent to 6 classrooms (378 students) and answered by 114 students.
Taking into account the data collected in this survey, the student profile involved in this study is female (74.76%), between 26 and 35 (45.79%), who worked while taking the course (95.32%). Most of them (86.91%) used the Net daily, beyond the activities related to the UOC. Before taking part of the course, their domain of ICT was medium (53.27%) or low (36.44%). - Contrasting and validating quality criteria based on the literature review with students’ perception. The last step was to compare what students perceived from their own experience with the established quality criteria from the literature review. This step was based on the open answers of the questionnaire through a non-systematic coding but the identification of labels, annotations and selection (Blaxter et al., 2000).
Results
Boettcher (2006) posits that the key focus of LGC is on the process of content generation and knowledge construction, and not the end product itself. The process’s main purpose is on the development of competences and the achievement of learning outcomes. This viewpoint emphasizes the learners’ benefits. Nevertheless, we have identified two complementary aspects related to the content itself that raise the importance of the output or end product: towards the teacher, as concrete learning evidence for evaluation purposes, and to other learners or the public in general, as an information source for use and reuse. Output, besides learners’ learning and awareness of good content, is also concerned with content for reuse (creating for others). Sener (2007) provides a collection of LGC examples as evidence that supports the approach in terms of learning improvement, increasing learner engagement and added “lasting” value of end products. This last feature is presented according to three different LGC reuse purposes: as a learning resource for future learners, as a beta product useful for professional or societal further development, and as an output serving the learner itself (evidence for their portfolio, a resource for the elaboration of a byproduct like a journal article, and other publications).
Quality issues regarding LGC should then pay attention to both the process of content creation as well as the output. A quality framework has been drawn up which focuses on these two dimensions (see Figure 2).
Figure 2 Dimensions of LGC quality framework
According to this general framework, we proceeded to disaggregate each of these dimensions into a series of indicators organized into three clusters: process, content, and format.
The development of a first version of the quality framework is based on literature. Aware of the non existence of a specific quality framework for LCG, we identified neighbor concepts and selected authoritative sources developing quality criteria including articles in journals, book chapters, but also university guides and web sites on the matter as follows: Learning Resources (Van Assche, 2006), Multimedia Learning Resources (Leacock & Nesbit, 2007, from Nesbit & Belfer, 2004), Digital Learning Resources (BECTA, 2007), Learning Materials at a Distance (Endean, 2003), e-Learning Materials (Ruiz, Candler, & Teasdale, 2007), Course Materials (KMD, 2007), Learning Materials (University of California Regents, 2004), Instructional Media (FGCU, 2006), Multimedia Learning (Clark & Mayer, 2008), Internet Research Sources (Harris, 2010), Internet Resources (Johnson& Lamb, 2011), Internet Information (Virginia Tech University Library, 2011), and Peer Production of eLearning (Ehlers, 2009).
We then listed the criteria and classified them according to our three clusters and two dimensions of our quality criteria structure organization. Specifically, we organized the quality criteria into these categories: content, format and process (see Figure 3). These three categories as well as the linked indicators help in practice to identify to what extent quality is addressed in a LGC from a holistic point of view. It bears mentioning that it does not have evaluation purposes but is rather a tool for supporting LGC development in order to ensure quality from the process itself.
Figure 3 quality categories and indicators for LGC
It is important to mention that it is a generic proposal: it includes all the items identified in the literature of the field. Indeed, depending on the context as well as the pedagogical proposal and the educational objectives, it can be referred or appreciated as relevant in the experience or not.
These criteria reflect how creativity is involved in LGC through PBL methodology and the skills that take part in the process. Using italics, we highlight those indicators in which creativity plays an important role.
In the following sections, we address each category focusing on their definition as well as the voice of the students (evidence).
Content
LGC content refers to the topic or issue that students develop. Many researchers highlight the importance of quality criteria related to content in such processes (Leacock & Nesbit, 2007; Van Assche, 2006; BECTA, 2007; Endean, 2003; Ruiz, Candler & Teasdale, 2007; KMD, 2007; University of California Regents, 2004; Harris, 2010; Johnson & Lamb, 2011; Virginia Tech University Library, 2011; FGCU, 2006; and Clark & Mayer, 2008).
Table 1 shows some quality criteria associated with content in LGC.
CRITERIA |
DEFINITION |
EVIDENCE |
Adequacy |
Adequacy and relevance of information in relation to the subject developed. |
“Contents have to be focused on the topic”. “Most important information must be selected and prioritized” |
Consistency |
Content and discourse consistency. |
“The project’s coherence is basic”. “I think contents have to be related to the project’s global concept”. |
Organization |
Content organization and sequence. |
“The content has to be properly laid out. If it’s not organized, readers cannot understand the project and they can lose interest in going on reading.” “There has to be an index with its sections””. |
Reliability |
Accuracy and reliability in using information sources. |
“Information resources must be reliable and fact-checked”. “I always read the references in order to find out where the information has been taken from”. |
Thoroughness |
Thoroughness and correct argumentation of statements and content. |
“Thorough content”. “Thoroughness in information processing”. |
Agility |
Content presentation rhythm. |
“The project has to be easy to read, and be understandable”. “Ensuring a dynamic reading”. |
References |
Citations and references, including the reuse of UGC. |
“References are important to show where information has been taken from”. Information resources (...) being referenced”. |
Argumentation |
Quality of the arguments developed. |
“When all information is gathered, we can elaborate the content, but we can’t copy and paste it. It’s necessary to write a new text, adding our ideas. Projects need our essence”. “...arguments being constructively written”. |
Currency |
Content is up-to-date. |
“Data was up-to date”. “Current information sources”. |
Recipient |
Adapting content to the recipient. |
“We have to think in the audience of our projects”. “Readers who have to read our projects have to be taken into account”. |
Summary |
Content synthesis capacity. |
“Projects should only contain useful information”. “The lesser the quantity of information, the better!”. |
Clarity |
Ease and simplicity of content understanding. |
“Expressions and writing have to be clear, understandable and sexy”. “content concise, without abstract thinking” |
Style |
Communication style, including spelling. |
“Correct and stylish writing”. “Reading can be easy with short sentences, clear and well expressed ideas, ...” |
Prospective |
How content encourages the generation of new content: further research on that issue, opening questions, formulating new questions, providing clues to research, etc. |
“Interesting content that promotes reflection”. “Additional Information complementing project’s content”. |
License |
License type in order to reuse content (CreativeCommons, GNU FDL, etc.). |
(No evidence) |
Dissemination |
Indexed in search engines, embedding, RSS, Tweeter and /or Facebook, etc. |
“Being based on dissemination maintaining the seriousness and quality of the text”. “Common language based Projects focusing on working methods and dissemination (using ICT) but not focused on content”. |
Rating |
Assessment made by visitors (users). |
(No evidence) |
Visits |
The number of visits. |
(No evidence) |
Validation |
Explicit validation and/or evaluation process (explanation of content creation process as well as validation. It should also include some elements such as authorship, date, editing and versions date and context where it is developed). |
(No evidence) |
Studying Table 1 in depth, there are some criteria that have not been taken into account by students as relevant for content quality in LGC. These are License, Rating, Visits and Validation. This situation is probably related to the course proposal in itself, which omits these elements in response to the context in which they occur.
However, we observe some trends which are considered essential, being mentioned by most students. These are Organization, Reliability, Summary, Clarity and Style. This is logical given that it is based on an academic use of LGC. Hence, students value those criteria that facilitate easy location of information, in an organized and synthetic way, which, in turn, provides reliability to their work.
As a result of this analysis, we conclude that students appreciate those elements that, from their point of view, provide quality to project content. This perception is based on the context of the course and so they are not aware of the value of their work beyond the course, i.e. its dissemination or reuse. However, it is important to mention that students access to similar projects that are used as examples in order to build their own.
Format
Format refers to those formal elements associated with the display of information. Literature in the field also highlights some elements related to format for UGC quality (Van Assche, 2006; Leacock & Nesbit, 2007; BECTA, 2007; Ruiz, Candler & Teasdale, 2007; KMD, 2007; University of California Regents, 2004; FGCU, 2006; Clark & Mayer, 2008).
Table 2 illustrates some quality criteria associated to LGC format.
CRITERIA |
DEFINITION |
EVIDENCE |
Adequate |
The format is consistent with the content presented. |
“Visually attractive and adequate”. |
Structure |
Sequence and clarity of the structure. |
“A Wiki’s well distributed structure makes it understandable at first sight”. “Visual aspects, navigability, structure, etc., are important, they facilitate reading and understanding. Readers can analyze different parts of the wiki quickly and easily”. |
Design |
Design and presentation. |
“A visually attractive design makes reading more fun”. “It has to be appealing and easy to read, avoiding reader’s fatigue. Visual aspects are very important”. |
Diversity |
Images, links, graphs, multimedia, videos, etc, and the proper treatment of those resources. |
“Attaching all types of resources (photos, video, etc.) to help project’s understanding, in order to make reading clearer and more concise.” “Adequate paragraph separation, inserting pictures and videos make reading more dynamic and fun.” |
Interaction usability |
Ease of navigation and access to all content. |
“Navigation has to be easy. Readers must navigate through project’s pages without thinking. If they have to think about how to read the next sections, they lose concentration”. “It’s necessary to access to all sections in no more than 2 clicks”. |
Typography |
Typographic resources used (frames, text attributes, titles, etc.) |
“The use of different font types, colors, bolds… Facilitate and stimulates reading” “Typography has to be clear and clean”. |
Resources |
Using the different resources provided by the tool (optimal use). |
“Using more Web 2.0 tools”. “Inserting links and other resources will make wikis more attractive”. |
Reuse |
Ease or degree of reuse of content format. |
(No evidence) |
Features |
Features to facilitate searching, printing, republishing content, etc. |
(No evidence) |
Accessibility |
The degree to which content can be used by people of all abilities and disabilities. |
“Content has to be visual, easily accessible and synthetic. It has to be all clear and easy”. “Visual and audio elements make content easier to use for more people”. |
Multiplatform |
Suitability for different types of devices (web, mobile, etc.). |
(No evidence) |
According to Table 2, students are not aware of the importance of the following criteria: Reuse, Features and Multiplatform. Furthermore, Adequate criterion was referred by only one student. As in the case of Content, the preliminary analysis of the results suggests that this is due to the subject approach in itself in that it is more oriented towards project building than its diffusion: generated content by learners is for its internal use only.
Concerning those criteria associated with format that are perceived as relevant by students, we observe a trend towards Structure, Design, Diversity, Interaction usability and Typography. This trend reinforces the importance of visual aspects in LGC as well as usability concerns, which is also addressed by the course proposal. In the ICT competences course, students are encouraged to take care of their projects from a formal point of view in order to be attractive to recipients.
Process
Process refers to the dynamics of development, i.e. how students work together in the virtual environment to develop the LGC. According to Van Assche (2006) and QMPP (2009), some LGC quality items associated with the process were defined. While some authors have suggested that process is a key element to LGC, the literature review has shown that most of the quality criteria refer to content and format. In an LGC scenario, we support the relevance of process in order to address quality in all dimensions.
Table 3 contains some LGC quality criteria associated with process.
CRITERIA |
DEFINITION |
EVIDENCE |
Collaboration |
Content elaborating in a collaboration way (not as a sum of parts). |
“"It is important that, although each team member has an assigned role or task, everyone can give their opinion in order to enrich the process.” “All group members have to contribute with their ideas to the entire project and to other member’s contributions”. |
Critical |
Critical skills through the process. |
“Constructive criticism is basic in order to carry out a common project”. “A <<devil’s advocate>> is necessary to provide criticism and help improve the projects” |
Consensus |
Ability to reach agreements in relation to any aspect associated with the work group. |
“It is necessary to reach a consensus on emerging comments and ideas during the process”. “Group members need to learn to leave behind their own ideas to get new ones through consensus.” |
Exchange |
Dialogue, exchange of different points of view and constructive criticism during the content elaboration process. |
“…Exchange of opinions and discuss them” “Debating ideas when all partners share their opinions in order to focus on workload uniformly.” |
Effectiveness |
Developing the workgroup process in an optimal way. |
“Coordination and time planning are basic issues to the groups working”. “Clear objectives must be defined from the beginning, but there has to be some flexibility.” |
Involvement |
Participation of all group members in the process. |
“Involvement, participation and motivation of all the group’s members”. “All members have to be aware of their own tasks for the whole project”. |
Constancy |
Regular and frequent participation. |
“Regular and active participation”. “Be connected daily to group’s virtual space in order to be continuously working”. |
Equitable |
Fair collaboration of group members in the process. |
“Implication of all the group members working at the same intensity”. “Trying to be fair and not to load all the weight of the project on one only member”. |
Attitude |
The civic and proactive attitude of all members within the workgroup. |
“Active attitude and facilitating communication”. “Availability, desire to work, empathy, assertiveness, patience, effort and desire of things working fine”. |
Technology |
Knowledge and use level of digital technology. |
“Digital literacy of group members". |
Reflection |
Reflecting on the process developed (intra and inter-evaluation). |
(No evidence) |
Teaching |
Monitoring, motivating, dynamizing and evaluating the process. |
“Teacher guidelines”. "Consultations with teachers." |
As we can see in Table 3, all criteria identified in the literature have been referred by students as relevant criteria for the quality of LGC, with the exception of Reflection. However, and according to Ehlers (2009) and QMPP (2009) research, the ICT competences course provides an activity aimed at reflecting on the development process of LGC for each of its four phases. Data analysis demonstrates that, while they not considered “Reflection” as a quality criterion in itself, they do rely on it to make their ratings within the survey. For example, a student says about the process referring to the group reflection activity: “As we commented in the group, we think (and I think) that the key elements in the process are: (…)”.
Technology criterion was referred by just one student. Given that the UOC is a fully virtual university, the relevance of ICT is clearly important. This issue, however, deserves more investigation.
Regarding the trends in students’ perception of the quality criteria associated with the process, they highlight Collaboration, Consensus, Effectiveness, Involvement, Equitability and Attitude. It should be noted that, in this case, we see a better balance on the value they attached to these criteria. Again, this trend is explained by the subject’s approach, focused on a very exhaustive way in how students develop their projects.
Conclusions
We have chosen to reflect on creativity aspects of open educational resources not only from the end product itself but from the very process and context of creation. The learner generated content approach to OER provides a rich setting to explore new ways in which knowledge may be created, targeted and shared. OER seen from an LGC perspective results in an innovative way to learn and to transcend the learning situation.
We are currently implementing this new learning methodology of content generation by learners through the use of web 2.0 tools. It is precisely because of the novelty of this approach, quality issues regarding practices of the kind are still emerging. LGC that focuses on the learner experience and on output, either for repurposing or reuse, should benefit from some guidelines that ensure both the quality of the process and of the end product.
Being aware of the relevance of quality in LGC processes, we have identified a set of quality criteria from neighbor concepts. Based on the literature and the concept of LGC, we have developed a quality criteria proposal organized into three categories (content, format and process); each category is defined by a number of indicators. These indicators are useful in identifying to what extent quality is addressed in an LGC from a holistic point of view.
As we suggest in the findings, many indicators of quality for LGC are related to creativity and especially those associated to format but also content ones. They include or refer to some skills like diversity, flexibility, novelty, challenging, curiosity, inspiration, motivation to seek new ways or solutions, new ways to meta learning, etc.
Process indicators provide less leeway to creativeness probably because the teacher has more control of this in the “ICT competences” course. The ways of collaboration and group organization are maybe the most creative elements in this sense.
The opportunity for each group to create its own project becomes a challenge in terms of creativity. It also increases the variety of resources. “ICT competences” students are aware that their Virtual Project will be shared with other students. In fact, all the projects constitute a repository which will serve as an example for new “ICT competences” students. Beyond the UOC, other users on the Net may also make use of these projects. For these new students, finding ways to improve them becomes a challenge and a motivating issue.
The results of this first study into the research and development of a quality framework for LGC provides insights on the issue and add new elements that help define the continuity of this research. The traced path includes: Firstly, revising the current quality criteria proposal according to the students’ perception and introduce those elements emerging from the experience. In this paper, we have pointed out some quality criteria that have not been taken into consideration by students. As we noted, this fact probably relates to the course approach. However, it is important to mention that learners have referred to other criteria, such as accessibility to content for example, that have not been directly addressed in the course object of study. As a result of this process, we will develop and validate the quality framework, redefining and adding new quality criteria for LGC.
Secondly, reorganizing the criteria establishing levels of inclusiveness and an index of relevance. This more detailed but more operational quality framework should facilitate its implementation. It is intended to be used as a support tool for the teacher at design time, when outlining the LGC methodology, as well as for the learner at developing time, when elaborating the new content. In this sense, the quality criteria are intended not only for evaluation purposes but mainly for scaffolding the generation of content.
Thirdly, the quality criteria should also be supplemented with an implementation guide that highlights critical aspects of LGC output intended as an OER. The contrast between the literature review and the students’ perception has shown the need to be more explicit in aspects that make up an OER and from which students seem to be not fully aware of. In general, those criteria which have not been mentioned by the students are associated with LGC use beyond the context of the course, i.e., dissemination, social appraisal, and reuse.
Learning through ICT increases the ways to enhance creativity. ICT not only provides a number of resources or technologies with many opportunities but extends the variety and possibilities of pedagogical proposals. LGC based on a PBL approach appears as a learning methodology serving learning purposes at the same time that embraces the principles of OER by giving the learner the status of knowledge creator and disseminator in a creative way. Quality issues presented in this paper point to supporting the task of generating content to ensure reuse including aspects of reliability and verifiability.
References
- Ardaiz-Villanueva, O.; Nicuesa-Chacón, X.; Brene-Artazcoz, O.; Sanz de Acedo, M.L.; Sanz de Acedo, M.T. (2011). Evaluation of computer tools for idea generation and team formation in project-based learning. Computers & Education, 56(3), 700-711.
- Balagué, F.; Zayas, F. (2007). Usos educatius dels blogs. Recursos, orientacions i experiències per a docents. Barcelona: Ed. UOC.
- Blaxter, L.; Hughes, C.; Tight, M. (2000). Cómo se hace una investigación. Barcelona: Gedisa.
- BECTA (2006). Quality principles for digital
learning resources.
Retrieved from: http://publications.becta.org.uk/download.cfm?resID=32112 - Boettcher, J.V. (2006, February 28). The rise of student performance content. Campus Technology. Retrieved from: http://campustechnology.com/articles/2006/02/the-rise-of-student-performance-content.aspx
- Bonwell, C. C; Eison, J. A. (1991). Active Learning: Creating Excitement in the Classroom. Washington, DC: George Washington Press.
- Cabero, J.; López, E.; Ballesteros, C. (2009). Experiencias universitarias innovadoras con blogs para la mejora de la praxis educativa en el contexto europeo. Revista De Universidad y Sociedad Del Conocimiento (RUSC), 6(2).
- Clark, R. C. & Mayer, R. E. (2008). e-Learning and the Science of Instruction: Proven Guidelines for Consumers and Designers of Multimedia Learning. San Francisco: Preiffer.
- Conole, G. (2008). New schemas for mapping pedagogies and technologies, Ariadne, 56. Retrieved from: http://www.ariadne.ac.uk/
- Csikszentmihalyi, M. (1997). Creativity: Flow and the Psychology of Discovery and Invention. New York: HarperCollins.
- Dans, E. (2009). Educación online: Plataformas educativas y el dilema de la apertura. Revista De Universidad y Sociedad Del Conocimiento, 6(1).
- Ehlers, U. D. (2009). Web 2.0 – E-Learning 2.0 – Quality 2.0? Quality for new learning cultures. Quality Assurance in Education 17, 296-314.
- Endean, M. (2003). Learning Materials at a Distance (Guide). UK Centre for materials Education. UK: Ashley Printers. Retrieved from: http://www.materials.ac.uk/guides/5-distancelearning.pdf
- FGCU (2006). Principles of online design: Instructional media. Florida Gulf Coast University. Retrieved from: http://www.fgcu.edu/onlinedesign/mediadev.html
- Franklin, T.; Van Harmelen, M. (2007). Web 2.0 for content for Learning and Teaching in Higher Education. Bristol: JISC. Retrieved from http://www.jisc.ac.uk/media/documents/programmes/digitalrepositories/Web 2.0-content-learning-and-teaching.pdf
- Grodecka, K.; Wild, F.; Kieslinger, B. (2008). How to Use Social Software in Higher Education. Retrieved from: http://www.icamp.eu/wp-content/uploads/2009/01/icamp-handbook-web.pdf
- Groth, J.C.; Peters, J. (1999). What blocks Creativity? A managerial perspective. Creativity and Innovation Management, 8(3), 179-187.
- Harris, R. (2010). Evaluating Internet research sources. Retrieved from http://www.virtualsalt.com/evalu8it.htm
- Herrmann, N. (1996). The whole brain business book. New Yourk: Mc Graw-Hill.
- Johnson, L.; Lamb. A. (2011). Evaluating Internet Resources. Teacher Tap. Retrieved from http://eduscapes.com/tap/topic32.htm
- (KMD, 2007) The 10 core quality principles. Deakin Knowledge Media Division. Retrieved from http://www.deakin.edu.au/quality/10-principles.php
- Leacock, T. L., & Nesbit, J. C. (2007). A Framework for Evaluating the Quality of Multimedia Learning Resources. Educational Technology & Society, 10 (2), 44-59.
- Lee, M. J. W., & McLoughlin, C. (2007). Teaching and learning in the Web 2.0 era: Empowering students through learner-generated content. International Journal of Instructional Technology and Distance Learning, 4(10), 21-34.
- Liu, L.; Johnson, D. L. (2005). Web-based resources and applications: Quality and influence. Computers in the Schools, 21, 131–146.
- Lynch, P. J.; Horton, S. (2008). Web style guide: Basic design principles for creating web sites. Yale University, USA.
- O’Reilly, T. (2005). What Is Web 2.0 Design Patterns and Business Models for the Next Generation of Software. Retrieved from: http://oreilly.com/web2/archive/what-is-web-20.html
- Palloff, R. M.; Pratt, K. (2001). Lessons from the cyberspace classroom. The realities of online teaching. San Francisco, EUA: Jossey-Bass.
- QMPP Project (Auvinen, A.M.; Ehlers, U. et al.): QMPP Handbook Quality Management of Peer Production. Downloadable at http://www.qmpp.net
- Rhoades, E. B.; Friedel, C. R.; Morgan, A. C. (2009). Can web 2.0 improve our collaboration? Techniques, 84(1).
- Richardson, J. C.; Swan, K. (2003). Examining social presence in online courses in relation to students’ perceived learning and satisfaction. Journal of Asynchronous Learning Networks, 7, 68-88.
- Ruiz, J.G.; Candler, C.; Teasdale, T. A. (2007). Peer Reviewing E-Learning: Opportunities, Challenges, and Solutions. Academic Medicine 82(5), 503-507
- Sener, J. (2007). In Search of Student-Generated Content in Online Education. E-mentor, 4(21). Retrieved from http://e-mentor.edu.pl/_xml/wydania/21/467.pdf
- Shirky, C. (2003). Social Software and the Politics of Groups. Available from « http://shirky.com/writings/group_politics.html » [Last accessed: 05/01/09).
- Stake, R. E. (1998). Investigación con estudios de casos. Madrid: Morata.
- University of California Regents (2004). Criteria: Constructing High Quality Learning Materials—a Checklist. Retrieved from http://missinglink.ucsf.edu/lm/lm_toolbox/quality_criteria_checklist.pdf
- van Assche, F.; Vuorikari, R. (2006). A framework for quality of learning resources. In U-D. Ehlers & J. M. Pawlowski (eds) Handbook on quality and standardization in e-learning (pp. 443-456). Germany: Springer Berlin - Heidelberg.
- Vidal, R.V.V. (2009). Creativity for problem solvers. AI & Soc, 23, 409-432.
- Virginia Tech University Library (2011). Evaluating internet information Retrieved from http://www.lib.vt.edu/instruct/evaluate/
- Wheeler, S.; Yeomans, P.; (2008). The good, the bad and the wiki: Evaluating student-generated content for collaborative learning. British Journal of Educational Technology, 39(6), 987-995.
- Williams, J. & Chann, S. (2009). Using Web 2.0 to Support the Active Learning Experience. Journal of Information Systems Education, 20(2), 165-174.
- Yang, H.-L., & Cheng, H. H. (2010). Creativity of student information system projects: from the perspective of network embeddedness. Computer & Education, 54(1), 209–221.
- Yin, R. K. (2003). Applications of case study research. London: Sage Publications Inc.